Pith. sign in

REVIEW 3 major objections 6 minor 69 references

Graffe: Graph Representation Learning via Diffusion Probabilistic Models

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that the negative log of the conditional denoising score matching loss is a tractable lower bound on conditional mutual information between data and learned representation, and it uses this Diff-InfoMax principle to…

desk verdict Solid empirical graph-SSL paper with a real exponentiation error in its main mutual-information theorem; worth peer review for the method, not for the theory as stated. read the letter →

arxiv 2505.04956 v1 pith:INLQW6RN submitted 2025-05-08 cs.LG cs.AI

Dingshuo Chen , Shuchen Xue , Liuji Chen , Yingheng Wang , Qiang Liu , Shu Wu , Zhi-Ming Ma , Liang Wang This is my paper · ORCID
classification cs.LGcs.AI MSC 68T0768R1094A17
keywords diffusionprobabilisticmodelsgraphrepresentationlearningself-superviseddenoisingscorematchingconditionalmutualinformationInfoMaxprincipleneuralnetworks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that diffusion models can be repurposed from generators into representation learners for graphs. Its central theoretical claim is that the negative logarithm of the conditional denoising score matching loss is a lower bound on the conditional mutual information between the data and a learned representation given the noisy observation. The paper calls this the Diff-InfoMax principle and uses it to justify training a graph encoder jointly with a conditional diffusion decoder. Empirically, Graffe achieves the best reported linear-probing accuracy on 9 of the 11 node and graph classification datasets tested.

What carries the argument

The central object is the conditional denoising score matching loss $L_{x_0,\mathrm{DSM},\phi,t} = \mathbb{E}[\|x_\theta(x_t, t, E_\phi(x_0)) - x_0\|^2]$, the reconstruction error of a diffusion decoder that receives the noisy data $x_t$, the time step $t$, and the representation $z = E_\phi(x_0)$ as a condition. Theorem 1 identifies the minimum of this loss with the trace of the conditional covariance $\mathrm{Tr}(\mathrm{Cov}[x_0 \mid x_t, E_\phi(x_0)])$; Lemma 3 bounds that trace from below by entropy; Theorem 3 assembles these into the Diff-InfoMax inequality. The Diff-InfoMax principle states that a representation $f(x)$ should maximize $\int \lambda(t)\, I(x; f(x) \mid x_t)\, dt$, the conditional mutual information between data and representation given the noisy observation, and the paper argues that the denoising objective is a tractable surrogate for it. In Graffe, random feature masking before the encoder prevents the trivial identity shortcut, and the decoder is a Graph-UNet that is GNN-based for graph-level tasks and MLP-based for node-level tasks, fusing $x_t$, $t$, and $z$ to reconstruct the original features.

What would settle it

Re-derive Eq. (27) from Eq. (26) symbolically: exponentiating $(d/2)\log(2\pi e/d \cdot \mathbb{E}[\mathrm{Tr}]) \geq h(x_0 \mid x_t) - I$ gives $\mathbb{E}[\mathrm{Tr}] \geq (d/2\pi e)\exp(2h/d)\exp(-2I/d)$. If the paper's $\exp(-I)$ is used instead, plug in any concrete joint distribution with known conditional mutual information, such as a Gaussian linear model, and check numerically whether the claimed inequality $I(x_0; E_\phi(x_0) \mid x_t) \geq -\log L + C$ actually holds; if it fails, Theorem 3 as stated is false.

Watch

Extended reading notes

Core claim

At the center of the paper is Theorem 3: for $x_0 \in \mathbb{R}^d$, the conditional denoising score matching loss at time $t$, $L_{x_0,\mathrm{DSM},\phi,t} = \mathbb{E}[\|x_\theta(x_t, t, E_\phi(x_0))-x_0\|^2]$, satisfies $I(x_0; E_\phi(x_0) \mid x_t) \geq -\log L_{x_0,\mathrm{DSM},\phi,t} + C$, with $C = \log(d/(2\pi e)) + \frac{2}{d}h(x_0 \mid x_t)$. Interpreted plainly, pushing down the denoising loss is pushing up a lower bound on the conditional mutual information, so the diffusion reconstruction objective is a principled surrogate for representation quality. The paper also proves that conditioning on an extra representation strictly decreases the minimal achievable denoising loss (Theorem 2), and that the standard InfoMax objective arises as the special case where the noisy observation is pure noise (Remark 1). The Graffe model converts these statements into an algorithm: a graph encoder produces the conditioning representation, and a Graph-UNet decoder reconstructs masked, noised node features.

Load-bearing premise

The load-bearing premise is a single algebra step in the proof of Theorem 3: exponentiating inequality (26) gives a factor $\exp(-2I/d)$, not $\exp(-I)$, so the stated lower bound depends on that step being repaired.

Editorial extensions

If this is right

  • If Theorem 3 holds, the standard diffusion reconstruction loss is not just a generation objective: minimizing it is equivalent to maximizing a lower bound on conditional mutual information, giving a principled justification for using diffusion probabilistic models as representation learners.
  • Because conditioning on a more informative representation lowers the denoising loss (Theorem 2), the same training signal pushes the encoder toward richer representations, which explains why the representation condition outperforms label-conditioned and unconditional denoising in the paper's experiments.
  • The Diff-InfoMax principle subsumes the classical InfoMax objective as the special case where the noisy observation is pure Gaussian noise, so diffusion-based self-supervised learning can be seen as a multi-scale generalization of InfoMax rather than a competing paradigm.
  • Graffe's linear-probing results on 9 of 11 datasets imply that generative self-supervision can be competitive with or better than contrastive and masked-autoencoder methods on graphs, at least under the linear evaluation protocol.
  • For node-level tasks, using an MLP rather than a GNN in the decoder works better because GNN message passing mixes nodes with different noise levels; this suggests that decoder design should isolate noise per node.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the exponent step in the proof of Theorem 3 is repaired, the same Diff-InfoMax logic would likely extend to any differentiable encoder, suggesting the theoretical result is not graph-specific even though Graffe's architecture is.
  • A natural testable extension is to condition the decoder on a set of representations extracted at multiple noise levels, potentially forcing the encoder to preserve both low- and high-frequency information simultaneously.
  • The masking ratio could be made adaptive and learned rather than tuned per dataset, since the paper's own ablation shows the optimal mask ratio varies widely across tasks, which is an empirical knob the theory does not currently explain.
  • A direct falsification of the proof's algebra would be to compute both sides of Theorem 3 on a simple synthetic distribution with known conditional mutual information; if the inequality fails, the theorem as stated cannot be salvaged by the given proof.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Graffe, a self-supervised graph representation learning method based on conditional diffusion models. A GNN encoder maps a masked graph into node or graph representations, and these representations condition a GraphU-Net or MLP denoising decoder that reconstructs node features. The paper's central theoretical claim is Theorem 3: the negative logarithm of a per-time denoising score matching loss lower-bounds the conditional mutual information I(x0; E_phi(x0) | xt) up to a constant, which the authors call the Diff-InfoMax principle. The paper also reports extensive node and graph classification experiments on 11 datasets, several of which achieve state-of-the-art accuracy, together with ablations on masking, reconstruction target, encoder, and decoder design.

Significance. If the theoretical claim were correct, the paper would offer a principled information-theoretic justification for diffusion-based representation learning and a strong new graph self-supervised method. The empirical evaluation is a clear strength: 11 datasets, reported error bars, many baselines, and thorough ablations. However, the proof of the central theorem contains an algebraic error at Eqs. (26)--(27) that invalidates the stated bound, and the claimed 'tractable' constant is not actually computable without estimating an unknown entropy term. The practical method may still be sound, but the paper's main theoretical contribution is not established in its current form.

major comments (3)
  1. [Section IV-B, Eqs. (26)--(30)] The step from Eq. (26) to Eq. (27) is algebraically incorrect. With A = (2*pi*e/d) E[Tr(Cov[x0|xt,E_phi(x0)])], Eq. (26) states (d/2) log A >= h(x0|xt) - I, where I = I(x0; E_phi(x0)|xt). Exponentiating gives A >= exp(2h/d) * exp(-2I/d), not A >= exp(2h/d) * exp(-I). This missing factor of 2/d propagates through Eqs. (29)--(30), so Theorem 3 as stated is not proven. A corrected derivation would yield a different bound, e.g. I >= h(x0|xt) - (d/2) log((2*pi*e/d) E[Tr]), and the coefficients on the loss and the entropy terms would differ from those in Eq. (30). The theorem, the Diff-InfoMax principle, and the discussion in Section IV-B must be reworked accordingly.
  2. [Abstract and Section IV-B, Eq. (20)] The paper repeatedly describes the mutual-information lower bound as 'tractable,' but the constant C in Eq. (20) contains the differential entropy h(x0|xt), which is not known in closed form for graph feature distributions and is not estimated anywhere in the paper. This quantity is independent of phi, so it is constant with respect to the encoder parameters, but it is not a computable constant. The claim that the negative log of the denoising score matching loss is a 'tractable lower bound' is therefore not supported without an additional estimation procedure.
  3. [Section IV-B, Figure 3] The empirical evidence in Figure 3 plots the negative logarithm of the training loss against linear probing accuracy on the same dataset, which shows only that lower training loss is associated with higher downstream accuracy. This is in-sample correlation and does not test the mutual-information inequality in Theorem 3; in particular, it cannot distinguish the proposed bound from a generic effect of improved reconstruction. The paper should present this as indirect motivation rather than as validation of the theoretical claim.
minor comments (6)
  1. [Section III-A] The subsection title contains a typo: 'Backgound' should be 'Background.'
  2. [Section II-A] There are misspellings in the related work section, including 'constractive' and 'constrastive'; these should be corrected to 'contrastive.'
  3. [Appendix A] In the proof of Theorem 1, the phrase 'objective objective' appears twice; one occurrence should be removed.
  4. [Appendix B] The section title 'HYPER-PARAMTER CONFIGURATIONS' should be 'HYPER-PARAMETER CONFIGURATIONS.'
  5. [Section IV-B, Eq. (20)] Theorem 3 is stated for a per-time loss L_{x0,DSM,phi,t}, while the main training objective in Eq. (6) integrates over time. The paper should clarify explicitly how the per-time theorem relates to the integrated training objective.
  6. [Overall] No source code is released. Given the number of implementation choices (noise schedules, mask ratios, decoder details), a public implementation would substantially improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: Theorem 3 is derived from standard max-entropy and chain-rule arguments without fitted parameters; the Diff-InfoMax principle is a consequence, and the proof gap at Eq. (26)-(27) is a correctness issue, not circularity.

full rationale

The central derivation is self-contained. Theorem 3 (Eq. 20/30) is obtained from Lemma 3, the standard Gaussian max-entropy bound for a fixed covariance trace, together with Jensen's inequality, the chain rule I(x0;xt,Ephi(x0)) = I(x0;xt) + I(x0;Ephi(x0)|xt), and Theorem 1's characterization of the DSM minimum as an expected conditional covariance. The loss L_{x0,DSM,phi,t} is a measured objective; no parameter is fitted to the target mutual information to force the inequality, and the theorem holds for arbitrary phi. The Diff-InfoMax principle is introduced after the theorem ('Motivated by Theorem 3, we propose the Diff-InfoMax principle'), so it is a restatement of the derived bound rather than an input used to derive the bound. Figure 3's correlation between the training loss and linear-probing accuracy is soft confirmation, but it is not a circular step because the theorem already fixes the direction of the claimed relationship. The self-citations [34]-[36] appear only in related-work context and are not load-bearing for any theorem or empirical claim. A separate non-circularity concern: the proof of Theorem 3 appears to have an arithmetic gap between Eq. (26) and Eq. (27), since exponentiating (d/2)log B >= h - I gives B >= exp(2h/d - 2I/d) rather than exp(2h/d - I); as written, the theorem's bound does not follow. Similarly, the constant C includes the unknown differential entropy h(x0|xt), so calling C 'tractable' is overstated. These are correctness/repair issues, not reductions of the claim to its inputs.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

The theoretical claims rely on standard probability theorems and an ideal-capacity assumption. The empirical method depends on several per-dataset hyperparameters, especially the mask ratio and noise schedule, which are chosen by performance. No new entities are postulated.

free parameters (7)
  • mask_ratio m = 0.7 for node datasets; 0.3 for graph datasets except MUTAG 0
    Tuned per dataset to prevent the encoder from copying features while retaining reconstructable information.
  • noise_schedule = sigmoid for most datasets; inverted for Ogbn-arxiv, quad for Photo
    Selected per dataset based on performance among sigmoid, linear, inverted, and quad.
  • learning_rate = 1e-4 or 3e-4/1e-3 per dataset
    Set per dataset; encoder LR is twice decoder LR.
  • num_hidden = 1024/512/256/32 per dataset
    Chosen per dataset to balance capacity and overfitting.
  • feat_drop = 0.1 to 0.4 per dataset
    Dropout on input features, tuned per dataset.
  • att_drop = 0.1 to 0.3 per dataset
    Dropout on attention weights in GAT, tuned per dataset.
  • num_head = 2 or 4 per dataset
    Number of attention heads, tuned per dataset.
assumptions (6)
  • standard math Denoising score matching loss minimization at infinite capacity converges to the conditional expectation E[x0|xt] or E[x0|xt,Eφ(x0)].
    Used in Theorems 1 and 2; follows from L2 projection properties of conditional expectation.
  • standard math For a fixed trace of covariance, the Gaussian distribution maximizes entropy (Lemma 3).
    Foundational to Theorem 3's bound.
  • standard math The diffusion forward process is a linear Gaussian SDE; xt|x0 ~ N(α_t x0, σ_t^2 I).
    Background assumption from Eq (1).
  • domain assumption The theoretical analysis assumes infinite model capacity and optimal training, so the actual DSM loss equals the population conditional variance lower bound.
    Section IV-A: 'we eliminate the effects of limited network capacity or optimization errors'.
  • ad hoc to paper Random feature masking (Bernoulli) is sufficient to prevent the identity shortcut without losing the information needed for reconstruction.
    Section V-A; the masking strategy is a design choice, not derived from a principle.
  • domain assumption Reconstructing node features is a better learning signal than reconstructing topology or both.
    Section V-B and Table III: empirical decision made after testing, not derived from theory.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Graffe: Graph Representation Learning via Diffusion Probabilistic Models." pith.science (2026). https://pith.science/paper/INLQW6RN

@misc{pith2026250504956,
  author       = {Pith},
  title        = {Pith review of: Graffe: Graph Representation Learning via Diffusion Probabilistic Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/INLQW6RN}},
  note         = {Machine review of arXiv:2505.04956}
}
read the original abstract

Diffusion probabilistic models (DPMs), widely recognized for their potential to generate high-quality samples, tend to go unnoticed in representation learning. While recent progress has highlighted their potential for capturing visual semantics, adapting DPMs to graph representation learning remains in its infancy. In this paper, we introduce Graffe, a self-supervised diffusion model proposed for graph representation learning. It features a graph encoder that distills a source graph into a compact representation, which, in turn, serves as the condition to guide the denoising process of the diffusion decoder. To evaluate the effectiveness of our model, we first explore the theoretical foundations of applying diffusion models to representation learning, proving that the denoising objective implicitly maximizes the conditional mutual information between data and its representation. Specifically, we prove that the negative logarithm of the denoising score matching loss is a tractable lower bound for the conditional mutual information. Empirically, we conduct a series of case studies to validate our theoretical insights. In addition, Graffe delivers competitive results under the linear probing setting on node and graph classification tasks, achieving state-of-the-art performance on 9 of the 11 real-world datasets. These findings indicate that powerful generative models, especially diffusion models, serve as an effective tool for graph representation learning.

Figures

Figures reproduced from arXiv: 2505.04956 by the authors.

Figure 1
Figure 1. The overall framework of Graffe. (Left) The input graph has certain nodes corrupted and is subsequently fed into a GNN encoder to obtain node representations as the condition. The decoder then receives both the noisy graph features xt and the condition z as inputs to perform denoising, aiming to restore the original node features x0. (Right) The diffusion process of graph features and the architecture of GraphU-Net … view at source ↗
Figure 2
Figure 2. The comparison of denoising losses using different conditions on Cora datasets. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The correlation between the negative logarithm of diffusion loss [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The effect of mask ratio m on Cora, Computer and MUTAG dataset. Computer and MUTAG dataset and consider a candidate list covering the value ranges of m: [0, 0.1, 0.3, 0.5, 0.7, 0.9]. As shown in [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

69 extracted references · 35 canonical work pages

  1. [1]

    Reducing the dimensionality of data with neural networks,

    G. E. Hinton and R. R. Salakhutdinov, “Reducing the dimensionality of data with neural networks,” science, vol. 313, no. 5786, pp. 504–507, 2006

    2006

  2. [2]

    Auto-encoding variational bayes,

    D. P. Kingma, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114, 2013

    arXiv 2013

  3. [3]

    Improving language understanding by generative pre- training,

    A. Radford, “Improving language understanding by generative pre- training,” 2018

    2018

  4. [4]

    Generative adversarial networks,

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y . Bengio, “Generative adversarial networks,” Communications of the ACM , vol. 63, no. 11, pp. 139–144, 2020

    2020

  5. [5]

    Generative pretraining from pixels,

    M. Chen, A. Radford, R. Child, J. Wu, H. Jun, D. Luan, and I. Sutskever, “Generative pretraining from pixels,” in International conference on machine learning. PMLR, 2020, pp. 1691–1703

    work page 2020

  6. [6]

    Large scale adversarial representation learning,

    J. Donahue and K. Simonyan, “Large scale adversarial representation learning,” Advances in neural information processing systems , vol. 32, 2019

    work page 2019

  7. [7]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems , vol. 33, pp. 6840– 6851, 2020

    2020

  8. [8]

    Score-based generative modeling through stochastic differential equations,

    Y . Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differential equations,” arXiv preprint arXiv:2011.13456 , 2020

    arXiv 2011

Show all 69 references
  1. [9]

    A revision bloom’s taxonomy: An overview,

    D. Krathwohl, “A revision bloom’s taxonomy: An overview,” Theory into Practice, 2002

    2002

  2. [10]

    Image generation from scene graphs,

    J. Johnson, A. Gupta, and L. Fei-Fei, “Image generation from scene graphs,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 1219–1228. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 11

    2018

  3. [11]

    Infodiffusion: Representation learning using information maximizing diffusion models,

    Y . Wang, Y . Schiff, A. Gokaslan, W. Pan, F. Wang, C. De Sa, and V . Kuleshov, “Infodiffusion: Representation learning using information maximizing diffusion models,” in International Conference on Machine Learning. PMLR, 2023, pp. 36 336–36 354

    2023

  4. [12]

    Soda: Bottleneck diffusion models for representation learning,

    D. A. Hudson, D. Zoran, M. Malinowski, A. K. Lampinen, A. Jaegle, J. L. McClelland, L. Matthey, F. Hill, and A. Lerchner, “Soda: Bottleneck diffusion models for representation learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 202...

    2024

  5. [13]

    A survey of graph meets large language model: Progress and future directions,

    Y . Li, Z. Li, P. Wang, J. Li, X. Sun, H. Cheng, and J. X. Yu, “A survey of graph meets large language model: Progress and future directions,” arXiv preprint arXiv:2311.12399 , 2023

    2023 arXiv

  6. [14]

    Gslb: The graph structure learning benchmark,

    Z. Li, L. Wang, X. Sun, Y . Luo, Y . Zhu, D. Chen, Y . Luo, X. Zhou, Q. Liu, S. Wu et al., “Gslb: The graph structure learning benchmark,” Advances in Neural Information Processing Systems , vol. 36, pp. 30 306–30 318, 2023

    2023

  7. [15]

    Denoising diffusion autoencoders are unified self-supervised learners,

    W. Xiang, H. Yang, D. Huang, and Y . Wang, “Denoising diffusion autoencoders are unified self-supervised learners,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 15 802–15 812

    2023

  8. [16]

    Deconstructing denoising diffusion models for self-supervised learning,

    X. Chen, Z. Liu, S. Xie, and K. He, “Deconstructing denoising diffusion models for self-supervised learning,” arXiv preprint arXiv:2401.14404 , 2024

    2024 arXiv

  9. [17]

    Directional diffusion models for graph representation learning,

    R. Yang, Y . Yang, F. Zhou, and Q. Sun, “Directional diffusion models for graph representation learning,” Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  10. [18]

    Diffusion-based representation learning,

    K. Abstreiter, S. Mittal, S. Bauer, B. Schölkopf, and A. Mehrjou, “Diffusion-based representation learning,” arXiv preprint arXiv:2105.14257, 2021

    2021 arXiv

  11. [19]

    Self-organization in a perceptual network,

    R. Linsker, “Self-organization in a perceptual network,” Computer, vol. 21, no. 3, pp. 105–117, 1988

    1988

  12. [20]

    Learning deep representations by mutual information estimation and maximization,

    R. D. Hjelm, A. Fedorov, S. Lavoie-Marchildon, K. Grewal, P. Bach- man, A. Trischler, and Y . Bengio, “Learning deep representations by mutual information estimation and maximization,” arXiv preprint arXiv:1808.06670, 2018

    2018 arXiv

  13. [21]

    Deep graph infomax

    P. Velickovic, W. Fedus, W. L. Hamilton, P. Liò, Y . Bengio, and R. D. Hjelm, “Deep graph infomax.” ICLR (Poster), vol. 2, no. 3, p. 4, 2019

    2019

  14. [22]

    Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization,

    F.-Y . Sun, J. Hoffmann, V . Verma, and J. Tang, “Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization,” arXiv preprint arXiv:1908.01000 , 2019

    1908 arXiv

  15. [23]

    Deep graph contrastive representation learning,

    Y . Zhu, Y . Xu, F. Yu, Q. Liu, S. Wu, and L. Wang, “Deep graph contrastive representation learning,” arXiv preprint arXiv:2006.04131 , 2020

    2006 arXiv

  16. [24]

    Graph contrastive learning with adaptive augmentation,

    ——, “Graph contrastive learning with adaptive augmentation,” in Proceedings of the web conference 2021 , 2021, pp. 2069–2080

    2021

  17. [25]

    Graph contrastive learning with augmentations,

    Y . You, T. Chen, Y . Sui, T. Chen, Z. Wang, and Y . Shen, “Graph contrastive learning with augmentations,” Advances in neural information processing systems, vol. 33, pp. 5812–5823, 2020

    2020

  18. [26]

    Large-scale representation learning on graphs via bootstrapping,

    S. Thakoor, C. Tallec, M. G. Azar, M. Azabou, E. L. Dyer, R. Munos, P. Veliˇckovi´c, and M. Valko, “Large-scale representation learning on graphs via bootstrapping,” arXiv preprint arXiv:2102.06514 , 2021

    2021 arXiv

  19. [27]

    From canonical correlation analysis to self-supervised graph neural networks,

    H. Zhang, Q. Wu, J. Yan, D. Wipf, and P. S. Yu, “From canonical correlation analysis to self-supervised graph neural networks,” Advances in Neural Information Processing Systems , vol. 34, pp. 76–89, 2021

    2021

  20. [28]

    Gpt-gnn: Generative pre-training of graph neural networks,

    Z. Hu, Y . Dong, K. Wang, K.-W. Chang, and Y . Sun, “Gpt-gnn: Generative pre-training of graph neural networks,” in Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, 2020, pp. 1857–1867

    2020

  21. [29]

    Contrastive multi-view representation learning on graphs,

    K. Hassani and A. H. Khasahmadi, “Contrastive multi-view representation learning on graphs,” in International conference on machine learning . PMLR, 2020, pp. 4116–4126

    2020

  22. [30]

    Gcc: Graph contrastive coding for graph neural network pre-training,

    J. Qiu, Q. Chen, Y . Dong, J. Zhang, H. Yang, M. Ding, K. Wang, and J. Tang, “Gcc: Graph contrastive coding for graph neural network pre-training,” in Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining , 2020, pp. 1150–1160

    2020

  23. [31]

    Variational graph auto-encoders,

    T. N. Kipf and M. Welling, “Variational graph auto-encoders,” arXiv preprint arXiv:1611.07308, 2016

    2016 arXiv

  24. [32]

    Graph attention auto-encoders,

    A. Salehi and H. Davulcu, “Graph attention auto-encoders,”arXiv preprint arXiv:1905.10715, 2019

    1905 arXiv

  25. [33]

    Graphmae: Self-supervised masked graph autoencoders,

    Z. Hou, X. Liu, Y . Cen, Y . Dong, H. Yang, C. Wang, and J. Tang, “Graphmae: Self-supervised masked graph autoencoders,” in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022, pp. 594–604

    2022

  26. [34]

    Uncovering neural scaling laws in molecular representation learning,

    D. Chen, Y . Zhu, J. Zhang, Y . Du, Z. Li, Q. Liu, S. Wu, and L. Wang, “Uncovering neural scaling laws in molecular representation learning,” Advances in Neural Information Processing Systems , vol. 36, pp. 1452– 1475, 2023

    2023

  27. [35]

    Beyond efficiency: Molecular data pruning for enhanced generalization,

    D. Chen, Z. Li, Y . Ni, G. Zhang, D. Wang, Q. Liu, S. Wu, J. Yu, and L. Wang, “Beyond efficiency: Molecular data pruning for enhanced generalization,” Advances in Neural Information Processing Systems , vol. 37, pp. 18 036–18 061, 2024

    2024

  28. [36]

    Gder: Safeguarding efficiency, balancing, and robustness via prototypical graph pruning,

    G. Zhang, H. Dong, Z. Li, D. Chen, K. Wang, T. Chen, Y . Liang, D. Cheng, K. Wang et al. , “Gder: Safeguarding efficiency, balancing, and robustness via prototypical graph pruning,” Advances in Neural Information Processing Systems , vol. 37, pp. 50 285–50 312, 2024

    2024

  29. [37]

    Diffusion autoencoders: Toward a meaningful and decodable representa- tion,

    K. Preechakul, N. Chatthee, S. Wizadwongsa, and S. Suwajanakorn, “Diffusion autoencoders: Toward a meaningful and decodable representa- tion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 10 619–10 629

    2022

  30. [38]

    Unsupervised representation learning from pre-trained diffusion probabilistic models,

    Z. Zhang, Z. Zhao, and Z. Lin, “Unsupervised representation learning from pre-trained diffusion probabilistic models,” Advances in neural information processing systems , vol. 35, pp. 22 117–22 130, 2022

    2022

  31. [39]

    Diffusion models as masked autoencoders,

    C. Wei, K. Mangalam, P.-Y . Huang, Y . Li, H. Fan, H. Xu, H. Wang, C. Xie, A. Yuille, and C. Feichtenhofer, “Diffusion models as masked autoencoders,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023

    2023

  32. [40]

    Reverse-time diffusion equation models,

    B. D. Anderson, “Reverse-time diffusion equation models,” Stochastic Processes and their Applications , vol. 12, no. 3, pp. 313–326, 1982

    1982

  33. [41]

    Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps,

    C. Lu, Y . Zhou, F. Bao, J. Chen, C. Li, and J. Zhu, “Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps,” Advances in Neural Information Processing Systems , vol. 35, pp. 5775–5787, 2022

    2022

  34. [42]

    Sa-solver: Stochastic adams solver for fast sampling of diffusion models,

    S. Xue, M. Yi, W. Luo, S. Zhang, J. Sun, Z. Li, and Z.-M. Ma, “Sa-solver: Stochastic adams solver for fast sampling of diffusion models,” arXiv preprint arXiv:2309.05019, 2023

    2023 arXiv

  35. [43]

    Accelerating diffusion sampling with optimized time steps,

    S. Xue, Z. Liu, F. Chen, S. Zhang, T. Hu, E. Xie, and Z. Li, “Accelerating diffusion sampling with optimized time steps,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 8292–8301

    2024

  36. [44]

    On variational bounds of mutual information,

    B. Poole, S. Ozair, A. Van Den Oord, A. Alemi, and G. Tucker, “On variational bounds of mutual information,” in International Conference on Machine Learning . PMLR, 2019, pp. 5171–5180

    2019

  37. [45]

    Diffusion models beat gans on image synthesis,

    P. Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,” Advances in neural information processing systems , vol. 34, pp. 8780–8794, 2021

    2021

  38. [46]

    Diffusion probabilistic model made slim,

    X. Yang, D. Zhou, J. Feng, and X. Wang, “Diffusion probabilistic model made slim,” in Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition , 2023, pp. 22 552–22 562

    2023

  39. [47]

    Freeu: Free lunch in diffusion u-net,

    C. Si, Z. Huang, Y . Jiang, and Z. Liu, “Freeu: Free lunch in diffusion u-net,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 4733–4743

    2024

  40. [48]

    Diffusion is spectral autoregression,

    S. Dieleman, “Diffusion is spectral autoregression,” 2024. [Online]. Available: https://sander.ai/2024/09/02/spectral-autoregression.html

    2024

  41. [49]

    Rethinking graph neural networks for anomaly detection,

    J. Tang, J. Li, Z. Gao, and J. Li, “Rethinking graph neural networks for anomaly detection,” in International Conference on Machine Learning . PMLR, 2022, pp. 21 076–21 089

    2022

  42. [50]

    Graphmae2: A decoding-enhanced masked self-supervised graph learner,

    Z. Hou, Y . He, Y . Cen, X. Liu, Y . Dong, E. Kharlamov, and J. Tang, “Graphmae2: A decoding-enhanced masked self-supervised graph learner,” in Proceedings of the ACM web conference 2023 , 2023, pp. 737–746

    2023

  43. [51]

    Masked graph autoencoder with non-discrete bandwidths,

    Z. Zhao, Y . Li, Y . Zou, J. Tang, and R. Li, “Masked graph autoencoder with non-discrete bandwidths,” in Proceedings of the ACM on Web Conference 2024, 2024, pp. 377–388

    2024

  44. [52]

    Graph attention networks,

    P. Velickovic, G. Cucurull, A. Casanova, A. Romero, P. Lio, Y . Bengio et al., “Graph attention networks,” stat, vol. 1050, no. 20, pp. 10–48 550, 2017

    2017

  45. [53]

    How powerful are graph neural networks?

    K. Xu, W. Hu, J. Leskovec, and S. Jegelka, “How powerful are graph neural networks?” arXiv preprint arXiv:1810.00826 , 2018

    2018 arXiv

  46. [54]

    What’s behind the mask: Understanding masked graph modeling for graph autoencoders,

    J. Li, R. Wu, W. Sun, L. Chen, S. Tian, L. Zhu, C. Meng, Z. Zheng, and W. Wang, “What’s behind the mask: Understanding masked graph modeling for graph autoencoders,” in Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , 2023, pp. 1268–1279

    2023

  47. [55]

    U-net: Convolutional networks for biomedical image segmentation,

    O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III

    2015

  48. [56]

    Springer, 2015, pp. 234–241

    2015

  49. [57]

    Sigmoid-weighted linear units for neural network function approximation in reinforcement learning,

    S. Elfwing, E. Uchibe, and K. Doya, “Sigmoid-weighted linear units for neural network function approximation in reinforcement learning,” Neural networks, vol. 107, pp. 3–11, 2018

    2018

  50. [58]

    Collective classification in network data,

    P. Sen, G. Namata, M. Bilgic, L. Getoor, B. Galligher, and T. Eliassi-Rad, “Collective classification in network data,” AI magazine, vol. 29, no. 3, pp. 93–93, 2008. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 12

    2008

  51. [59]

    Pitfalls of graph neural network evaluation,

    O. Shchur, M. Mumme, A. Bojchevski, and S. Günnemann, “Pitfalls of graph neural network evaluation,” arXiv preprint arXiv:1811.05868 , 2018

    2018 arXiv

  52. [60]

    Open graph benchmark: Datasets for machine learning on graphs,

    W. Hu, M. Fey, M. Zitnik, Y . Dong, H. Ren, B. Liu, M. Catasta, and J. Leskovec, “Open graph benchmark: Datasets for machine learning on graphs,” Advances in neural information processing systems , vol. 33, pp. 22 118–22 133, 2020

    2020

  53. [61]

    Deep graph kernels,

    P. Yanardag and S. Vishwanathan, “Deep graph kernels,” in Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining , 2015, pp. 1365–1374

    2015

  54. [62]

    Libsvm: a library for support vector,

    C. Chang and C. Lin, “Libsvm: a library for support vector,” 2001

    2001

  55. [63]

    Adam: A method for stochastic optimization,

    D. P. Kingma, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  56. [64]

    Decoupled weight decay regularization,

    I. Loshchilov, “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  57. [65]

    Sgdr: Stochastic gradient descent with warm restarts,

    I. Loshchilov and F. Hutter, “Sgdr: Stochastic gradient descent with warm restarts,” arXiv preprint arXiv:1608.03983 , 2016

    2016 arXiv

  58. [66]

    Rethinking graph masked autoencoders through alignment and uniformity,

    L. Wang, X. Tao, Q. Liu, and S. Wu, “Rethinking graph masked autoencoders through alignment and uniformity,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 14, 2024, pp. 15 528–15 536

    2024

  59. [67]

    Infogcl: Information- aware graph contrastive learning,

    D. Xu, W. Cheng, D. Luo, H. Chen, and X. Zhang, “Infogcl: Information- aware graph contrastive learning,” Advances in Neural Information Processing Systems, vol. 34, pp. 30 414–30 425, 2021

    2021

  60. [68]

    Weisfeiler-lehman graph kernels

    N. Shervashidze, P. Schweitzer, E. J. Van Leeuwen, K. Mehlhorn, and K. M. Borgwardt, “Weisfeiler-lehman graph kernels.” Journal of Machine Learning Research, vol. 12, no. 9, 2011

    2011

  61. [69]

    graph2vec: Learning distributed representations of graphs,

    A. Narayanan, M. Chandramohan, R. Venkatesan, L. Chen, Y . Liu, and S. Jaiswal, “graph2vec: Learning distributed representations of graphs,” arXiv preprint arXiv:1707.05005 , 2017. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 13 APPENDIX A PROOFS A. Proof of Theor...

    2017 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.