Pith. sign in

REVIEW 3 major objections 5 minor 51 references

TINED: GNNs-to-MLPs by Teacher Injection and Dirichlet Energy Distillation

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A layer-by-layer distillation method copies a trained GNN's feature transformations into an MLP, approximates graph propagation, and matches smoothing ratios; the students beat prior distillations and, with graph access, the teacher.

desk verdict Clever FT-injection and DE-ratio distillation; the paper overclaims its advantage over NOSMOG, which beats TINED on the large OGB datasets. read the letter →

arxiv 2412.11180 v3 pith:BPK2Y47C submitted 2024-12-15 cs.LG cs.SI

Ziang Zhou , Zhihao Ding , Jieming Shi , Qing Li , Shiqi Shen This is my paper · ORCID
classification cs.LGcs.SI MSC 68T0768R10
keywords GNN-to-MLPdistillationknowledgeteacherinjectionDirichletenergygraphneuralnetworksnodeclassificationinferenceaccelerationover-smoothing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TINED is a layer-by-layer distillation method that turns a trained graph neural network into a plain MLP, so node classification can run without repeatedly gathering multi-hop neighbourhoods at inference. The paper's central claim is that a GNN layer's feature transformation is formally identical to a fully-connected layer, so the teacher's trained weights can be copied directly into the student, and that the remaining graph-propagation step can be approximated by an extra FC layer whose smoothing behaviour is matched to the teacher through a Dirichlet-energy ratio. Across seven datasets the resulting students outperform prior GNN-to-MLP distillation methods, and the graph-aware variant TINED+ is reported to exceed the teacher GNN on all seven datasets while running 94 times faster.

What carries the argument

Two mechanisms carry the argument. Teacher injection copies the trained transformation $W^{(\ell)}, b^{(\ell)}$ of each FT operation in the teacher GNN into a corresponding FC layer of the student MLP, because both compute $\sigma(hW + b)$; a gradient modifier $\eta$ limits how far fine-tuning moves the injected weights. Dirichlet energy distillation defines the DE ratio $R_{\text{op}} = E(\text{op}(H))/E(H)$ with $E(H) = \frac{1}{n}\operatorname{tr}(H^\top L H)$, and adds a loss that matches student FC-layer ratios to the teacher's GP and FT ratios, preserving the observation that GP smooths aggressively while FT smooths conservatively. Theorem 4.1 supplies the theoretical support for replacing propagation with a linear layer: for a sparse Laplacian $L$ and full-column-rank $H$, some $W^*$ gives $\|LH - HW^*\|_F / \|H\|_F \le \lambda_{\max}(L)$.

What would settle it

For a frozen GraphSAGE teacher on a chosen dataset, compute the per-layer relative error between the actual GP output and the best linear FC approximation $\|GP^{(\ell)}(H) - HW^*\|_F/\|H\|_F$. If this error exceeds $\lambda_{\max}(L)$, or if ablating the DE-ratio loss and randomizing injected weights still yields the reported accuracy, the method's stated justification for replacing propagation with an FC layer would be contradicted on those layers.

Watch

Extended reading notes

Core claim

The central discovery is that fine-grained, layer-wise knowledge—trained feature-transformation parameters and per-layer smoothing ratios—is more transferable from a GNN to an MLP than output soft labels alone. With a two-layer GraphSAGE teacher, TINED without any graph access reaches 74.43% on Citeseer, improving on GLNN by 3.21 percentage points and on NOSMOG by 1.54, and on the OGB Products dataset it gains 9.42 points over a plain MLP. When graph structure is allowed at inference, TINED+ exceeds the GraphSAGE teacher on all seven datasets and achieves 75.39% on Citeseer. The same recipe works when the teacher is GCN, GAT, or APPNP, and on heterophilic graphs.

Load-bearing premise

The practical success rests on an unproven premise: that a single fully-connected layer can closely approximate the specific graph-propagation operation of the chosen GNN, since the proof's linear-map assumption does not cover the concatenation used by GraphSAGE or the learned attention used by GAT.

Editorial extensions

If this is right

  • If TINED's results hold, a graph-free student MLP can replace a message-passing GNN for node classification without sacrificing accuracy, and TINED+ can even exceed the teacher's accuracy.
  • Inference time drops from 153 ms for the 2-layer GraphSAGE teacher to about 1.6 ms for the student, a 94x speedup and 733x against the 3-layer teacher.
  • The ablation results attribute the gains to both components: removing teacher injection or removing DE distillation lowers accuracy on every dataset.
  • The method transfers across teacher architectures (GraphSAGE, GCN, GAT, APPNP), so layer-wise injection is not tied to one GNN design.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the paper leaves implicit is that matching per-layer Dirichlet energy ratios could serve as a general regularizer against over-smoothing in deeper MLP students, since it forces each FC layer to keep the same smoothing profile as the teacher's corresponding operations.
  • The injection principle extends to other architecture pairs: whenever a teacher module is computationally identical to a student module, copying parameters and then fine-tuning with a small learning rate may transfer more knowledge than matching outputs alone.
  • A testable extension is to replace the single FC approximation of graph propagation with a small set of feature-dependent linear maps when the teacher uses attention; the paper's DE-ratio matching would then have an even closer target.
  • On graphs where the unnormalized Laplacian has a large largest eigenvalue, the theoretical bound becomes loose; measuring the actual approximation error on such graphs would show whether the practical method still holds where its proof is weakest.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes TINED, a GNN-to-MLP distillation method that operates layer-wise rather than treating teacher and student as monolithic models. TINED injects the feature-transformation weights of a trained GNN teacher into corresponding FC layers of an MLP student, adds an FC layer to emulate graph propagation, and adds a Dirichlet-energy-ratio loss to match layer-wise smoothing behavior. A variant TINED+ additionally uses graph structure at inference. The authors report a theoretical bound on the graph-propagation approximation error and present experiments on seven benchmark datasets under transductive and production settings, with additional results on two heterophilic datasets, ablations, parameter sensitivity studies, and inference-time measurements. The main empirical claim is that TINED outperforms prior GNN-to-MLP distillation baselines (GLNN, NOSMOG, KRD, FFG2M) and that TINED+ exceeds the teacher GNN on all seven datasets.

Significance. If the comparative results survive a controlled hyperparameter search, the layer-wise distillation idea is a useful and nontrivial contribution to the GNN-to-MLP literature. The paper ships source code, reports means and standard deviations over 10 trials, includes ablation and sensitivity analyses, and extends evaluation to heterophilic and large OGB datasets. The Dirichlet-energy-ratio observation (FT tends to be more conservative than GP) is interesting and gives a principled reason for layer-wise matching. The theoretical bound in Theorem 4.1 is, however, a relatively weak linear-algebra statement and is not clearly connected to the actual GP operations of the experimental teachers. The empirical comparison, not the theorem, is the load-bearing evidence, and that comparison currently suffers from an asymmetric hyperparameter search budget.

major comments (3)
  1. [§5.2, Appendix A.7] The central empirical claim that TINED outperforms existing distillation methods is not fully supported by the current comparison because the search effort is unequal. In Appendix A.7, GLNN and NOSMOG use hyperparameters from their original papers, and GLNN* and NOSMOG* only match TINED's number of layers and hidden dimension, while TINED is selected over a much larger per-dataset search space (learning rate, weight decay, lambda, beta, eta, zeta, normalization, dropout, batch size, and DE-ratio smoothing function). The reported margins are small on several datasets, for example 0.14% over GLNN* on Pubmed in Table 1, and Table 5 shows that varying only eta can move Citeseer accuracy by more than 7 percentage points. A matched hyperparameter search for the baselines, with model selection on validation data for all methods, is needed before the superiority claim can be accepted.
  2. [Abstract, §5.2] The abstract's statement that TINED 'outperforms GNNs' is too broad. In the graph-independent setting of Table 1, TINED is below the GraphSAGE teacher on Arxiv (64.44 vs 70.73) and Products (69.48 vs 77.17); only TINED+ exceeds the teacher on all seven datasets. The claim should be qualified to state which variant is meant, for example 'TINED+ exceeds the teacher when graph structure is available at inference, while TINED matches or improves on the teacher on the smaller datasets without graph access.'
  3. [Theorem 4.1, §4.1, Appendix A.2] The approximation bound in Theorem 4.1 is not connected to the GP operations actually used in the main experiments. For GraphSAGE, Eq. (4) defines GP(l) as a concatenation of the node embedding with an aggregation of neighbor embeddings, so the output is not a square matrix product LH with L in R^{n x n}, and lambda_max(L) in the theorem is not defined for that operation. For GAT, Eq. (16) defines GP through a data-dependent attention matrix, and the student FC layer in Eq. (5) includes a bias and nonlinearity that the theorem does not model. In addition, for the unnormalized Laplacian L = D - A used in Definition 4.2, lambda_max(L) can be large, making the bound vacuous. The theoretical claim should be restricted to GNNs whose propagation is exactly a fixed square Laplacian product, such as GCN in Eq. (15), or explicitly presented as a heuristic motivation.
minor comments (5)
  1. [Table 2] The 'prod' rows in Table 2 are reported without standard deviations, unlike the 'ind' and 'tran' rows; please report the variation of the weighted average across seeds.
  2. [Eq. (6)] The notation hat-nabla in Eq. (6) is not defined; please clarify that it denotes the modified gradient used to update the injected parameters.
  3. [Figure 4] Figure 4 aggregates results over multiple teacher GNNs and datasets, but the caption does not state what is being averaged and no error bars are shown; please clarify the aggregation protocol.
  4. [References] The reference to Hinton et al. (2015) is given as arXiv:2211.02989, which is the identifier of a different paper; please correct the arXiv number.
  5. [Conclusion] There is a typo in the conclusion: 'grpahs' should be 'graphs'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: TINED's student is trained against external teacher signals (fixed GNN soft labels and per-layer DE ratios); the theoretical bound is an independent linear-algebra statement.

full rationale

All load-bearing training signals in TINED come from a fixed, pre-trained teacher GNN: soft labels z_v (Eq. 3), the injected FT parameters W^(l), b^(l) (Eqs. 4-5), and the per-layer DE ratios R_GP^(l), R_FT^(l) (Definition 4.3 and Eq. 8). These targets are computed from the teacher and from the graph Laplacian; they are not functions of the student MLP's own parameters or outputs. The student's DE ratios are matched to the teacher's in Eq. 8, but matching a statistic is not equivalent to defining it. Theorem 4.1 is a generic linear-algebra bound (proof in Appendix A.2) with W* = H†LH; it holds independently of TINED and is not derived from the method's fitted values, although its applicability to GraphSAGE's concatenation GP is a soundness concern rather than a circular one. The authors' self-citations (SlotGAT ICML 2023, Info. Sci. 2023, CIKM 2024) appear only as related work and are not load-bearing. No equation in the paper reduces by construction to a fitted input or to a self-citation chain; reported improvements are against external benchmarks and fixed teachers.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method's central claim rests on the assumptions that FT maps to FC exactly, that GP can be approximated by an FC layer within acceptable error, and that matching DE ratios transfers useful knowledge. The free parameters eta, beta, lambda, zeta, and mu are all tuned per dataset and affect performance materially. No new physical or mathematical entities are introduced.

free parameters (5)
  • eta (gradient modifier) = searched in [0.01, 0.1, 0.5, 1, 3, 10]; best varies by dataset, e.g., 1.0 on Cora, 0.5 on Citeseer
    Controls how much injected teacher FT parameters are fine-tuned; directly affects performance (Table 5).
  • beta (DED loss weight) = searched in [1e-6, 5e-5, 1e-5, 0.05, 0.1, 0.5, 1, 5, 10]; best varies, e.g., 1.0 on Cora, 10.0 on Citeseer
    Balances Dirichlet Energy Distillation loss; sensitivity shown in Table 7.
  • lambda (KL loss weight) = searched in [0.1, 0.4, 0.5, 0.6, 1]
    Balances soft-label distillation loss from Eq (3).
  • zeta (DE sampling ratio) = searched in [0.001, 0.005, 0.1, 0.4, 1]; used 0.7 on Arxiv
    Subgraph sampling ratio to approximate DE ratios on large graphs; results in Appendix A.8.
  • mu (DE ratio smoothing function) = chosen from sqrt, log, identity
    Empirical smoothing on DE ratios in MSE loss (Appendix A.7).
assumptions (5)
  • standard math Standard linear algebra: SVD, pseudo-inverse, spectral norm bound
    Used in proof of Theorem 4.1, Appendix A.2.
  • domain assumption FT operation in GNN layers has the same functional form as an FC layer in MLP
    Basis of Teacher Injection; holds for GraphSAGE/GCN/GAT when FT is linear transformation plus activation (Eq 4, 15, 16).
  • domain assumption GP operation in a GNN layer can be approximated by a linear map HW* with error bounded by lambda_max(L)
    Theorem 4.1 assumes GP output is LH for fixed sparse L; for GraphSAGE GP includes concatenation, for GAT the attention matrix depends on H and W, so the fixed-linear-map assumption is violated in general.
  • ad hoc to paper Matching DE ratios between teacher and student layers improves student accuracy
    No theoretical justification; validated only empirically by ablation Table 4 and Figure 6.
  • domain assumption Transductive soft labels from teacher on all nodes, including test nodes, are legitimate training targets
    Standard in GLNN/NOSMOG; can be seen as using teacher predictions on unlabeled nodes, not ground truth.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TINED: GNNs-to-MLPs by Teacher Injection and Dirichlet Energy Distillation." pith.science (2026). https://pith.science/paper/BPK2Y47C

@misc{pith2026241211180,
  author       = {Pith},
  title        = {Pith review of: TINED: GNNs-to-MLPs by Teacher Injection and Dirichlet Energy Distillation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BPK2Y47C}},
  note         = {Machine review of arXiv:2412.11180}
}
read the original abstract

Graph Neural Networks (GNNs) are pivotal in graph-based learning, particularly excelling in node classification. However, their scalability is hindered by the need for multi-hop data during inference, limiting their application in latency-sensitive scenarios. Recent efforts to distill GNNs into multi-layer perceptrons (MLPs) for faster inference often underutilize the layer-level insights of GNNs. In this paper, we present TINED, a novel approach that distills GNNs to MLPs on a layer-by-layer basis using Teacher Injection and Dirichlet Energy Distillation techniques. We focus on two key operations in GNN layers: feature transformation (FT) and graph propagation (GP). We recognize that FT is computationally equivalent to a fully-connected (FC) layer in MLPs. Thus, we propose directly transferring teacher parameters from an FT in a GNN to an FC layer in the student MLP, enhanced by fine-tuning. In TINED, the FC layers in an MLP replicate the sequence of FTs and GPs in the GNN. We also establish a theoretical bound for GP approximation. Furthermore, we note that FT and GP operations in GNN layers often exhibit opposing smoothing effects: GP is aggressive, while FT is conservative. Using Dirichlet energy, we develop a DE ratio to measure these effects and propose Dirichlet Energy Distillation to convey these characteristics from GNN layers to MLP layers. Extensive experiments show that TINED outperforms GNNs and leading distillation methods across various settings and seven datasets. Source code are available at https://github.com/scottjiao/TINED_ICML25/.

Figures

Figures reproduced from arXiv: 2412.11180 by the authors.

Figure 1
Figure 1. The DE ratios of FTs and GPs in the layers of GraphSAGE. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. (a) TINED with Teacher Injection and Dirichlet Energy Distillation; (b) Inference settings student models without message passing. GLNN (Zhang et al., 2022b) trains an MLP student using node features as input and soft labels from a GNN teacher as targets. NOSMOG (Tian et al., 2022) incorporates graph structure, adversarial feature augmentation, and node similarity rela￾tions into distillation. VQGraph learns a struc… view at source ↗
Figure 3
Figure 3. shows the trade-off between accuracy and infer￾ence time on Citeseer. Methods closer to the top-left cor￾ner achieve a better balance of accuracy and speed. Our methods, TINED and TINED+, achieve the highest accuracy 10 0 10 1 10 2 10 3 10 4 Inference Time (ms) 70 72 74 76 Accuracy GLNN GLNN* SAGE-L1 SAGE-L2 SAGE-L3 NOSMOG NOSMOG* TINED+ TINED KRD FFG2M [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (5 more)
Figure 5
Figure 5. Figure 5: t-SNE of model embeddings at different training stages on Citeseer. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Learned DE ratios in MLPs v.s. ground truth of [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: The layer-wise DE ratio of trained GCN, GraphSAGE and GAT, with 2 layers architecture on different datasets. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: The layer-wise DE ratio of trained GCN, GraphSAGE and GAT, with 5 layers architecture on different datasets. [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Layer-wise t-SNE visualization of the embeddings for various models (GraphSAGE, [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 40 canonical work pages

  1. [1]

    Distilhubert: Speech representation learning by layer-wise distillation of hidden-unit bert

    Heng-Jui Chang, Shu-wen Yang, and Hung-yi Lee. Distilhubert: Speech representation learning by layer-wise distillation of hidden-unit bert. In ICASSP, pages 7087--7091. IEEE, 2022

    work page 2022

  2. [2]

    Measuring and relieving the over-smoothing problem for graph neural networks from the topological view

    Deli Chen, Yankai Lin, Wei Li, Peng Li, Jie Zhou, and Xu Sun. Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. In AAAI, pages 3438--3445, 2020 a

    work page 2020

  3. [3]

    On graph neural networks versus graph-augmented mlps

    Lei Chen, Zhengdao Chen, and Joan Bruna. On graph neural networks versus graph-augmented mlps. In ICLR, 2021

    work page 2021

  4. [4]

    Simple and deep graph convolutional networks

    Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. Simple and deep graph convolutional networks. In ICML, 2020 b

    work page 2020

  5. [5]

    Graph-free knowledge distillation for graph neural networks

    Xiang Deng and Zhongfei Zhang. Graph-free knowledge distillation for graph neural networks. IJCAI, 2021

    work page 2021

  6. [6]

    Effective illicit account detection on large cryptocurrency multigraphs

    Zhihao Ding, Jieming Shi, Qing Li, and Jiannong Cao. Effective illicit account detection on large cryptocurrency multigraphs. In CIKM, page 457–466, 2024

    work page 2024

  7. [7]

    Rayleigh quotient graph neural networks for graph-level anomaly detection

    Xiangyu Dong, Xingyi Zhang, and Sibo Wang. Rayleigh quotient graph neural networks for graph-level anomaly detection. In ICLR, 2024

    work page 2024

  8. [8]

    Spacegnn: Multi-space graph neural network for node anomaly detection with extremely limited labels

    Xiangyu Dong, Xingyi Zhang, Lei Chen, Mingxuan Yuan, and Sibo Wang. Spacegnn: Multi-space graph neural network for node anomaly detection with extremely limited labels. In ICLR, 2025 a

    work page 2025

Show all 51 references
  1. [9]

    Smoothgnn: Smoothing-aware gnn for unsupervised node anomaly detection

    Xiangyu Dong, Xingyi Zhang, Yanni Sun, Lei Chen, Mingxuan Yuan, and Sibo Wang. Smoothgnn: Smoothing-aware gnn for unsupervised node anomaly detection. In Proceedings of the ACM on Web Conference 2025, pages 1225--1236, 2025 b

    2025

  2. [10]

    Freekd: Free-direction knowledge distillation for graph neural networks

    Kaituo Feng, Changsheng Li, Ye Yuan, and Guoren Wang. Freekd: Free-direction knowledge distillation for graph neural networks. In KDD, 2022

    2022

  3. [11]

    Hamilton, Rex Ying, and Jure Leskovec

    William L. Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NeurIPS, 2017

    2017

  4. [12]

    Mlpinit: Embarrassingly simple GNN training acceleration with MLP initialization

    Xiaotian Han, Tong Zhao, Yozen Liu, Xia Hu, and Neil Shah. Mlpinit: Embarrassingly simple GNN training acceleration with MLP initialization. In ICLR, 2023

    2023

  5. [14]

    Open graph benchmark: Datasets for machine learning on graphs

    Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. In NeurIPS, 2020

    2020

  6. [15]

    Graph-mlp: node classification without message passing in graph

    Yang Hu, Haoxuan You, Zhecan Wang, Zhicheng Wang, Erjin Zhou, and Yue Gao. Graph-mlp: node classification without message passing in graph. arXiv preprint arXiv:2106.04051, 2021

    2021 arXiv

  7. [16]

    A comprehensive survey of regression based loss functions for time series forecasting

    Aryan Jadon, Avinash Patil, and Shruti Jadon. A comprehensive survey of regression based loss functions for time series forecasting. arXiv preprint arXiv:2211.02989, 2022

    2022 arXiv

  8. [17]

    Redundancy-free computation for graph neural networks

    Zhihao Jia, Sina Lin, Rex Ying, Jiaxuan You, Jure Leskovec, and Alex Aiken. Redundancy-free computation for graph neural networks. In KDD, 2020

    2020

  9. [18]

    Tinybert: Distilling BERT for natural language understanding

    Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. Tinybert: Distilling BERT for natural language understanding. In EMNLP, pages 4163--4174. Association for Computational Linguistics, 2020

    2020

  10. [19]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015

    2015

  11. [20]

    Kipf and Max Welling

    Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017

    2017

  12. [21]

    Predict then propagate: Graph neural networks meet personalized pagerank

    Johannes Klicpera, Aleksandar Bojchevski, and Stephan G \"u nnemann. Predict then propagate: Graph neural networks meet personalized pagerank. In ICLR, 2019

    2019

  13. [22]

    Graph-based knowledge distillation by multi-head attention network

    Seunghyun Lee and Byung Cheol Song. Graph-based knowledge distillation by multi-head attention network. arXiv preprint arXiv:1907.02226, 2019

    1907 arXiv

  14. [23]

    Layer-level knowledge distillation for deep neural network learning

    Hao-Ting Li, Shih-Chieh Lin, Cheng-Yeh Chen, and Chen-Kuo Chiang. Layer-level knowledge distillation for deep neural network learning. Applied Sciences, 9 0 (10): 0 1966, 2019

    1966

  15. [24]

    Distance encoding: Design provably more powerful neural networks for graph representation learning

    Pan Li, Yanbang Wang, Hongwei Wang, and Jure Leskovec. Distance encoding: Design provably more powerful neural networks for graph representation learning. In NeurIPS, 2020

    2020

  16. [25]

    Less is more: Task-aware layer-wise distillation for language model compression

    Chen Liang, Simiao Zuo, Qingru Zhang, Pengcheng He, Weizhu Chen, and Tuo Zhao. Less is more: Task-aware layer-wise distillation for language model compression. In ICML, pages 20852--20867. PMLR, 2023

    2023

  17. [26]

    Towards deeper graph neural networks

    Meng Liu, Hongyang Gao, and Shuiwang Ji. Towards deeper graph neural networks. In KDD, pages 338--348, 2020

    2020

  18. [27]

    A critical look at the evaluation of gnns under heterophily: Are we really making progress? ICLR, 2023

    Oleg Platonov, Denis Kuznedelev, Michael Diskin, Artem Babenko, and Liudmila Prokhorenkova. A critical look at the evaluation of gnns under heterophily: Are we really making progress? ICLR, 2023

    2023

  19. [28]

    Multi-scale attributed node embedding

    Benedek Rozemberczki, Carl Allen, and Rik Sarkar. Multi-scale attributed node embedding. Journal of Complex Networks, 9 0 (2): 0 cnab014, 2021

    2021

  20. [29]

    A survey on oversmoothing in graph neural networks

    T Konstantin Rusch, Michael M Bronstein, and Siddhartha Mishra. A survey on oversmoothing in graph neural networks. arXiv preprint arXiv:2303.10993, 2023

    2023 arXiv

  21. [30]

    Learning mlps on graphs: A unified view of effectiveness, robustness, and efficiency

    Yijun Tian, Chuxu Zhang, Zhichun Guo, Xiangliang Zhang, and Nitesh Chawla. Learning mlps on graphs: A unified view of effectiveness, robustness, and efficiency. In ICLR, 2022

    2022

  22. [31]

    Knowledge distillation on graphs: A survey

    Yijun Tian, Shichao Pei, Xiangliang Zhang, Chuxu Zhang, and Nitesh V Chawla. Knowledge distillation on graphs: A survey. arXiv preprint arXiv:2302.00219, 2023

    2023 arXiv

  23. [32]

    Graph attention networks

    Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In ICLR, 2018

    2018

  24. [33]

    Equivariant and stable positional encoding for more powerful graph neural networks

    Haorui Wang, Haoteng Yin, Muhan Zhang, and Pan Li. Equivariant and stable positional encoding for more powerful graph neural networks. In ICLR, 2022

    2022

  25. [34]

    Deep graph library: A graph-centric, highly-performant package for graph neural networks

    Minjie Wang, Da Zheng, Zihao Ye, Quan Gan, Mufei Li, Xiang Song, Jinjing Zhou, Chao Ma, Lingfan Yu, Yu Gai, Tianjun Xiao, Tong He, George Karypis, Jinyang Li, and Zheng Zhang. Deep graph library: A graph-centric, highly-performant package for graph neural networks. arXiv prepr...

    1909 arXiv

  26. [35]

    Lirong Wu, Haitao Lin, Yufei Huang, Tianyu Fan, and Stan Z. Li. Extracting low-/high- frequency knowledge from graph neural networks and injecting it into mlps: An effective gnn-to-mlp distillation framework. In AAAI, pages 10351--10360, 2023 a

    2023

  27. [36]

    Lirong Wu, Haitao Lin, Yufei Huang, and Stan Z. Li. Quantifying the knowledge in gnns for reliable distillation into mlps. In ICML, pages 37571--37581, 2023 b

    2023

  28. [37]

    A comprehensive survey on graph neural networks

    Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems, 2020

    2020

  29. [38]

    Tinygnn: Learning efficient graph neural networks

    Bencheng Yan, Chaokun Wang, Gaoyang Guo, and Yunkai Lou. Tinygnn: Learning efficient graph neural networks. In KDD, 2020

    2020

  30. [39]

    Extract the knowledge of graph neural networks and go beyond it: An effective knowledge distillation framework

    Cheng Yang, Jiawei Liu, and Chuan Shi. Extract the knowledge of graph neural networks and go beyond it: An effective knowledge distillation framework. In WWW, 2021

    2021

  31. [40]

    Vqgraph: Rethinking graph representation space for bridging gnns and mlps

    Ling Yang, Ye Tian, Minkai Xu, Zhongyi Liu, Shenda Hong, Wei Qu, Wentao Zhang, Bin CUI, Muhan Zhang, and Jure Leskovec. Vqgraph: Rethinking graph representation space for bridging gnns and mlps. In ICLR, 2024

    2024

  32. [41]

    Distilling knowledge from graph convolutional networks

    Yiding Yang, Jiayan Qiu, Mingli Song, Dacheng Tao, and Xinchao Wang. Distilling knowledge from graph convolutional networks. In CVPR, 2020

    2020

  33. [42]

    Position-aware graph neural networks

    Jiaxuan You, Rex Ying, and Jure Leskovec. Position-aware graph neural networks. In ICML, 2019

    2019

  34. [43]

    Diving into unified data-model sparsity for class-imbalanced graph representation learning

    Chunhui Zhang, Chao Huang, Yijun Tian, Qianlong Wen, Zhongyu Ouyang, Youhuan Li, Yanfang Ye, and Chuxu Zhang. Diving into unified data-model sparsity for class-imbalanced graph representation learning. In GLFrontiers, 2022 a

    2022

  35. [44]

    Heterogeneous graph neural network

    Chuxu Zhang, Dongjin Song, Chao Huang, Ananthram Swami, and Nitesh V Chawla. Heterogeneous graph neural network. In KDD, pages 793--803, 2019

    2019

  36. [45]

    Agl: A scalable system for industrial-purpose graph machine learning

    Dalong Zhang, Xin Huang, Ziqi Liu, Zhiyang Hu, Xianzheng Song, Zhibang Ge, Zhiqiang Zhang, Lin Wang, Jun Zhou, Yang Shuang, et al. Agl: A scalable system for industrial-purpose graph machine learning. arXiv preprint arXiv:2003.02454, 2020

    2003 arXiv

  37. [46]

    Graph-less neural networks: Teaching old mlps new tricks via distillation

    Shichang Zhang, Yozen Liu, Yizhou Sun, and Neil Shah. Graph-less neural networks: Teaching old mlps new tricks via distillation. In ICLR, 2022 b

    2022

  38. [47]

    Cold brew: Distilling graph node representations with incomplete or missing neighborhoods

    Wenqing Zheng, Edward W Huang, Nikhil Rao, Sumeet Katariya, Zhangyang Wang, and Karthik Subbian. Cold brew: Distilling graph node representations with incomplete or missing neighborhoods. In ICLR, 2022

    2022

  39. [48]

    Graph neural networks: A review of methods and applications

    Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. Graph neural networks: A review of methods and applications. AI Open, 2020

    2020

  40. [49]

    Slotgat: Slot-based message passing for heterogeneous graphs

    Ziang Zhou, Jieming Shi, Renchi Yang, Yuanhang Zou, and Qing Li. Slotgat: Slot-based message passing for heterogeneous graphs. In ICML , volume 202, pages 42644--42657, 2023 a

    2023

  41. [50]

    Effective stabilized self-training on few-labeled graph data

    Ziang Zhou, Jieming Shi, Shengzhong Zhang, Zengfeng Huang, and Qing Li. Effective stabilized self-training on few-labeled graph data. Inf. Sci., 631: 0 369--384, 2023 b

    2023

  42. [51]

    Interpreting and unifying graph neural networks with an optimization framework

    Meiqi Zhu, Xiao Wang, Chuan Shi, Houye Ji, and Peng Cui. Interpreting and unifying graph neural networks with an optimization framework. In WWW, page 1215–1226, 2021

    2021

  43. [52]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.