Pith. sign in

REVIEW 4 major objections 4 minor 48 references

Graph Attention is Not Always Beneficial: A Theoretical Analysis of Graph Attention Mechanisms via Contextual Stochastic Block Models

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Graph attention boosts node classification only when structure noise outweighs feature noise; under feature noise, plain convolution wins, and a multi-layer hybrid reaches perfect classification at far weaker signals.

desk verdict The noise-regime story is interesting and probably repairable, but the central SNR formulas in Corollary 1 and Eqn 4 are wrong as written, and Lemma 5's proof does not work. read the letter →

arxiv 2412.15496 v3 pith:ZPIRCNOW submitted 2024-12-20 cs.LG stat.ML

Zhongtian Ma , Qiaosheng Zhang , Bocheng Zhou , Yexin Zhang , Shuyue Hu , Zhen Wang This is my paper · ORCID
classification cs.LGstat.ML MSC 05C8062H3068T07
keywords graphattentioncontextualstochasticblockmodelnodeclassificationsignal-to-noiseratioover-smoothingperfectneuralnetworkstheoreticalanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks when graph attention mechanisms actually earn their keep in node classification. Working on Contextual Stochastic Block Models (CSBMs), it defines structure noise as the imbalance between intra- and inter-class edge probabilities and feature noise as the inverse signal-to-noise ratio, then tracks how the signal-to-noise ratio changes through one attention layer. It concludes that attention helps when structure noise dominates feature noise, with larger attention intensity giving larger gains, and hurts when feature noise dominates, where a plain graph convolution (attention intensity zero) is the better choice. It also claims attention can stop the over-smoothing that degrades deep graph convolutional networks, and that a hybrid multi-layer design achieves perfect classification under a much weaker signal-to-noise requirement than a single layer. The practical stake is a clear rule of thumb: use attention when edges are noisy, not when node features are noisy.

What carries the argument

The load-bearing object is the sign-based graph attention mechanism of Equation 3, which assigns attention score $t$ to a neighbor when the two features have the same sign and $-t$ when they differ, then normalizes with softmax; $t$ is called the attention intensity. Because this rule is nonlinear in the features, the proof machinery computes the post-layer mean and variance through truncated Gaussian moments, yielding explicit functions $F$ and $\widehat F$ for $\mu'$ and $(\sigma')^2$. That computation converts the qualitative question "does attention help?" into a comparison of $\mu'/\sigma'$ against $\mu/\sigma$, and the SNR ratio $\delta(t)$ in the structure-noise regime is the object that carries the conclusion that attention intensity can be tuned up or down to match the noise type.

What would settle it

Compute the exact post-layer SNR for the sign rule on a CSBM with $S_{\mathrm{noise}}=O(1)$ and $F_{\mathrm{noise}}=\omega(1)$ for a family of $t>0$: if $\mu'/\sigma'$ increases with $t$ for any finite $n$ in that regime, the $\Theta$ bound in Equation 5 is wrong. Alternatively, train the learnable attention model used in the paper's Appendix L.2 on the same regime and measure whether its classification error rises above the $t=0$ convolution baseline; the paper's own results indicate it does not, which would mark the boundary of the claim.

Watch

Extended reading notes

Core claim

On a balanced two-class CSBM with $p,q=\Omega(\log^2 n/n)$ and $p>q$, the paper analyzes the attention rule $\Psi(X_i,X_j)=t$ when $X_iX_j\ge 0$ and $-t$ otherwise, followed by softmax aggregation. After one layer the output expectation and variance are computed in closed form, giving the post-layer SNR. In the regime $S_{\mathrm{noise}}=(p+q)/(p-q)=\omega(1)$ and $F_{\mathrm{noise}}=\sigma/\mu=o(1/\sqrt{\log n})$, the SNR becomes $\mu'/\sigma'=\sqrt n\,\delta(t)\,\mu/\sigma$ with $\delta(t)$ increasing in $t>0$; in the opposite regime $S_{\mathrm{noise}}=O(1)$ and $F_{\mathrm{noise}}=\omega(1)$, the SNR decreases as $t$ grows, so $t=0$ (pure convolution) is best. For over-smoothing, the paper proves that for $L=O(n)$ layers a GCN satisfies $\gamma(X^{(l)})=(1-2q/(p+q))^l\gamma(X^{(0)})$, while a GAT with $t=\omega(\sqrt{\log n})$ keeps $\gamma(X^{(l)})=\Theta(\gamma(X^{(0)}))$. Finally, for $p=a\log^2 n/n$ and $q=b\log^2 n/n$, a hybrid network that starts with convolutional layers and finishes with a high-intensity attention layer achieves perfect node classification whenever $\mu/\sigma=\omega(\sqrt{\log n}/n^{1/3})$, relaxing the single-layer requirement $\omega(\sqrt{\log n})$.

Load-bearing premise

Everything in Theorems 1-4 is proven for homophilic graphs with $p,q=\Omega(\log^2 n/n)$ and $p>q$, a density condition the paper explicitly says is chosen for the proof technique, and the attention analysis uses a fixed sign rule rather than learned attention weights.

Editorial extensions

If this is right

  • When $S_{\mathrm{noise}}=\omega(1)$ and $F_{\mathrm{noise}}=o(1/\sqrt{\log n})$, each attention layer multiplies the SNR by $\sqrt n\,\delta(t)$, so increasing $t$ strictly improves separability, up to the ceiling $\sqrt{np}\,\mu/\sigma$.
  • When $S_{\mathrm{noise}}=O(1)$ and $F_{\mathrm{noise}}=\omega(1)$, every positive attention intensity lowers the post-layer SNR, so the optimal choice among the analyzed mechanisms is the graph convolution corresponding to $t=0$.
  • In the high-SNR regime, GCNs over-smooth after $O(n)$ layers while GATs with $t=\omega(\sqrt{\log n})$ preserve a constant fraction of node distinguishability for up to $\Theta(n)$ layers.
  • A multi-layer GAT that first reduces variance with convolution layers and then applies a strong attention layer achieves perfect classification for $\mu/\sigma=\omega(\sqrt{\log n}/n^{1/3})$ under $p=a\log^2 n/n$, $q=b\log^2 n/n$, which is strictly weaker than the single-layer threshold.
  • The design principle for practice is to keep attention intensity low while feature noise is high and raise it as the representation sharpens, which is exactly the schedule the paper's GAT* experiments validate.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's noise-regime theorem is proven for its hand-crafted sign rule, not for learned attention; its own appendix shows a learnable GATv2 learns near-uniform weights under feature noise and does not show the predicted degradation, so 'attention is not always beneficial' should be read as a property of fixed non-learnable attention rules on CSBMs, not of all attention mechanisms.
  • Because the multi-layer gain comes from convolution layers shrinking variance faster than the mean decays before the attention layer is switched on, the same 'denoise first, attend later' schedule might transfer to other classifiers and real graphs, though the CSBM proof does not establish that transfer.
  • The over-smoothing result suggests a depth-budget rule based on attention intensity: if $t$ is small, the exponential decay rate of the node-similarity measure is $(1-2q/(pe^{2t}+q))$, so the maximum useful depth grows with $t$, a quantitative statement readers could test on real networks.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper studies graph attention mechanisms on Contextual Stochastic Block Models (CSBMs). It introduces a non-learnable sign-based attention rule (Eq. 3) with attention intensity t, characterizes the post-attention expectation and variance (Theorem 2), and uses these to claim that attention helps when structure noise dominates feature noise but hurts when feature noise dominates. It also defines a notion of over-smoothing and claims GAT avoids it in the high-SNR regime (Theorem 3), and proposes a multi-layer GCN-then-GAT architecture that achieves perfect classification under SNR = ω(√log n / n^{1/3}) (Theorem 4), relaxing the prior ω(√log n) requirement. Experiments on synthetic and real datasets are provided.

Significance. If the results are correct, the paper gives a useful theoretical delineation of when attention helps versus hurts in a tractable random-graph model, extends previous single-layer GAT analysis by Fountoulakis et al., and provides a concrete multi-layer construction with a substantially weaker SNR threshold. The qualitative dichotomy between structure noise and feature noise is intuitive and supported by experiments, and the proposed multi-layer GAT* design is a natural practical takeaway. However, the quantitative formulas in the central noise-regime analysis contain a factor-of-two and a missing t-dependence in the variance, and one key auxiliary lemma has an invalid proof; these issues affect the derived SNR formulas and the proof of Theorem 3, although the qualitative conclusions appear repairable.

major comments (4)
  1. [Appendix G.1, Corollary 1, Eq. (83)] The variance computation is off by a factor of 2. Under Lemma 2, |N_i| = n(p+q)/2(1±...), so for t=0 the degree-normalized average has Var(X'_i)=σ²/|N_i| ≈ 2σ²/[n(p+q)], not σ²/[n(p+q)] as stated in Corollary 1 and Eq. (83). Consequently Eq. (4) should read μ'/σ' = √(n/2) δ(t) μ/σ, not √n δ(t) μ/σ. The qualitative monotonicity in t survives, but the displayed quantitative SNR-boost formula is incorrect by a factor √2.
  2. [Appendix G.2 and Corollary 1, second bullet] In the high-SNR regime the derivation in Eq. (97) gives (σ')² = (|N_p|e^{2t}+|N_q|e^{-2t}) / (|N_p|e^t+|N_q|e^{-t})² σ², which after substituting the Lemma 2 degree scalings becomes 2(p e^{2t}+q e^{-2t}) / [n(p e^t+q e^{-t})²] σ². This is t-dependent and carries a 1/n factor. It contradicts the second bullet of Corollary 1, which states a t-independent variance of 1/[n(p+q)]σ². Since Section 3.2.1 uses Corollary 1 to derive Eq. (4), this is a load-bearing error, not a typographical one.
  3. [Appendix K, Lemma 5] The proof of Lemma 5 is invalid as written. Monotonicity and boundedness show only that Γ(n,m) converges; since the lower bound in Eq. (132) tends to 0, the limit is 0. Monotone convergence of Γ to 0 does not imply Γ(n+c1,m+c2)/Γ(n,m)→1, which is the property needed in Eq. (50). A direct asymptotic argument is required. The lemma is used to replace S(. , |N_p|-1, |N_q|) with S(. , |N_p|, |N_q|-1) in the proof of Theorem 2, so this invalidates the simplified expectation and variance formulas as presently derived.
  4. [Appendix I, Eq. (111)] The step γ(X^(l)) = (1-2q/(p e^{2t}+q))^l μ(1+o(1)) = Θ(μ) is not correct for t=ω(√log n). For example, with t=(log n)^{2/3}, e^{-2t}=n^{-2/(log n)^{1/3}}, so (1-Θ(e^{-2t}))^n →0, not Θ(1). The theorem's conclusion that over-smoothing is avoided can still be reached because the per-layer decay factor tends to 1, so no positive constant C2 satisfies the definition's exponential bound at l=1; however, the displayed Θ(γ(X^(0))) identity is false and the proof needs to be corrected.
minor comments (4)
  1. [Theorem 1] The statement should explicitly require that t be sufficiently large (or that e^{2t} > q/p), since the proof uses 'for a sufficiently large t' at Eq. (25).
  2. [Section 4.1, Experiment 3] The text says 'we set a = 2, b = 3, and u = 10'; this should presumably read μ = 10.
  3. [Appendix F.3, Eq. (75)] The displayed equation labels the cross-term expectation as E[B], but the cross term is denoted C in Eq. (55); it should be E[C].
  4. [Title and abstract] The general claim that 'graph attention is not always beneficial' is demonstrated for the non-learnable sign-based mechanism of Eq. (3), not for learned attention generally. Appendix L.2 notes that GATv2 learns near-uniform attention under feature noise and does not show the predicted degradation; this scope limitation should be reflected in the abstract or contribution statements.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's theorems are derived from explicit CSBM assumptions and external prior results, not from their own conclusions.

full rationale

The load-bearing results are derived, not assumed. Theorem 1 uses the explicit sign-based attention rule (Eqn. 3) and Lemma 2 from Fountoulakis et al. (2023) as an external concentration tool; Theorem 2 computes the post-attention expectation and variance from Gaussian moment integrals, and Corollary 1 simplifications follow from the asymptotic events in that external lemma. The Snoise/Fnoise comparison in Section 3.2.1 is then a consequence of the formulas in Corollary 1, not a restatement of the definitions of Snoise and Fnoise. Theorem 4 amplifies the SNR through GCN layers using Lemma 7, which is imported from Wu et al. (2022b), and then invokes Theorem 1; no fitted parameter or target classification event is used as an input, so the relaxed SNR condition does not reduce to the conclusion by construction. The paper's own caveats, such as the omission of learnable attention parameters (Conclusion) and the Appendix L.2 observation that a learnable GATv2 does not show the predicted degradation under feature noise, are honest scope limitations rather than circular steps. A possible constant-factor error in the Corollary 1 variance formula (the degree concentration event Delta_2 gives |N_i| approximately n(p+q)/2, which would change the stated 1/(n(p+q)) factor) is a correctness concern, but it does not make any derived quantity equivalent to an input by definition or by self-citation. The few self-citations in the reference list (Ma et al. 2022; Zhang & Tan 2022) are contextual and not load-bearing.

Assumptions & free parameters 1 free parameters · 5 assumptions · 1 invented entities

The paper's central results depend on the CSBM generative model, on Assumption 1 restricting graph density and homophily, on a hand-crafted attention mechanism that may not represent learnable attention, on the unproven-as-written Lemma 5, and on a tailored over-smoothing definition. The attention intensity t is a hand-chosen hyperparameter, and no constants are fitted to data.

free parameters (1)
  • attention intensity t = asymptotic t = ω(√log n) for Theorems 3 and 4; manual schedule [0, 0.5, 0.5, 5] in Figure 1d experiments
    t is a tunable hyperparameter of the proposed attention mechanism, set by hand in experiments rather than fitted to data; the theorems only assume t is asymptotically large.
assumptions (5)
  • domain assumption CSBM with balanced classes, independent Bernoulli edges, and one-dimensional Gaussian features
    Section 2.1; the analysis is confined to this generative model, so findings may not transfer to other feature or graph distributions.
  • domain assumption Assumption 1: p, q = Ω(log² n/n) and p > q (homophilic, dense enough graphs)
    Required by Lemma 2 concentration events and used in every theorem; the paper admits it is dictated by proof technique, so sparse or heterophilic graphs are outside the proven scope.
  • ad hoc to paper Lemma 5 ratio limit: Γ(n+c1,m+c2)/Γ(n,m) → 1 as n,m → ∞
    Appendix K's proof is flawed as written because monotone convergence gives a zero limit for Γ, not a ratio limit; the claim is plausible but unproven and underpins the simplification in Theorem 2 and Corollary 1.
  • ad hoc to paper The sign-based attention mechanism Ψ(Xi,Xj)=±t (Eqn 3) serves as a proxy for graph attention
    All theorems analyze this hand-crafted non-learnable attention; general learnable GATs may adapt differently, as shown in Appendix L.2 for GATv2, so the abstract's general claims depend on this proxy premise.
  • ad hoc to paper Definition 2 counts only exponential decay in layer index as over-smoothing
    A sub-exponential decay of the similarity measure is classified as resolved over-smoothing even if γ tends to zero; this definitional choice makes Theorem 3's over-smoothing claim depend on the new definition rather than on the prior literature's notion.
invented entities (1)
  • Sign-based non-linear graph attention mechanism (Eqn 3) independent evidence
    purpose: Provide an analyzable proxy for graph attention that permits exact expectation and variance computations
    It is a new mechanism introduced for mathematical tractability, not a physically hypothesized entity; it is validated empirically in the paper, so it has a falsifiable handle.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Graph Attention is Not Always Beneficial: A Theoretical Analysis of Graph Attention Mechanisms via Contextual Stochastic Block Models." pith.science (2026). https://pith.science/paper/ZPIRCNOW

@misc{pith2026241215496,
  author       = {Pith},
  title        = {Pith review of: Graph Attention is Not Always Beneficial: A Theoretical Analysis of Graph Attention Mechanisms via Contextual Stochastic Block Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZPIRCNOW}},
  note         = {Machine review of arXiv:2412.15496}
}
abstract

Despite the growing popularity of graph attention mechanisms, their theoretical understanding remains limited. This paper aims to explore the conditions under which these mechanisms are effective in node classification tasks through the lens of Contextual Stochastic Block Models (CSBMs). Our theoretical analysis reveals that incorporating graph attention mechanisms is \emph{not universally beneficial}. Specifically, by appropriately defining \emph{structure noise} and \emph{feature noise} in graphs, we show that graph attention mechanisms can enhance classification performance when structure noise exceeds feature noise. Conversely, when feature noise predominates, simpler graph convolution operations are more effective. Furthermore, we examine the over-smoothing phenomenon and show that, in the high signal-to-noise ratio (SNR) regime, graph convolutional networks suffer from over-smoothing, whereas graph attention mechanisms can effectively resolve this issue. Building on these insights, we propose a novel multi-layer Graph Attention Network (GAT) architecture that significantly outperforms single-layer GATs in achieving \emph{perfect node classification} in CSBMs, relaxing the SNR requirement from $ \omega(\sqrt{\log n}) $ to $ \omega(\sqrt{\log n} / \sqrt[3]{n}) $. To our knowledge, this is the first study to delineate the conditions for perfect node classification using multi-layer GATs. Our theoretical contributions are corroborated by extensive experiments on both synthetic and real-world datasets, highlighting the practical implications of our findings.

Figures

Figures reproduced from arXiv: 2412.15496 by the authors.

Figure 1
Figure 1. c) support this theoretical result. 3.4. Perfect Node Classification in Multi-layer GATs Based on the preceding discussion, we have identified sce￾narios where the graph attention mechanism enhances node classification and mitigates the over-smoothing issue. Lever￾aging these insights, we can strategically design more ef￾fective multi-layer GATs for node classification tasks, i.e., using our proposed graph attention… view at source ↗
Figure 2
Figure 2. , we observe that when the feature noise is small, GAT outperforms GCN. However, as the feature noise in￾creases, GAT’s performance begins to fall behind that of GCN, which is consistent with our theoretical analysis in Section 3.2.1. Furthermore, GAT* exhibits greater robust￾ness to feature noise, maintaining high accuracy regardless of the noise strength, which also validates our theoretical results in Section 3.4… view at source ↗
Figure 3
Figure 3. Additional experimental results on real-world datasets. Figures 3a, 3b and 3c illustrate the results for the Citeseer, Cora, and Pubmed datasets, respectively [PITH_FULL_IMAGE:figures/full_fig_p039_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Accuracy heatmaps of models on ogbn-arxiv under varying structure and feature noise levels. From [PITH_FULL_IMAGE:figures/full_fig_p040_4.png]
Figure 5
Figure 5. Figure 5: Comparison of attention coefficients under high feature noise (fn = 0.8) and high structure noise (sn = 0.8) on the ogbn-arxiv dataset. The visualization is based on the last layer of the GATv2 model, focusing on the top 9 nodes with the highest degrees. 41 [PITH_FULL…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 35 canonical work pages

  1. [1]

    Community detection and stochastic block models: recent developments

    Abbe, E. Community detection and stochastic block models: recent developments. Journal of Machine Learning Research, 18 0 (177): 0 1--86, 2018

    2018

  2. [2]

    and Sandon, C

    Abbe, E. and Sandon, C. Community detection in general stochastic block models: Fundamental limits and efficient algorithms for recovery. In 2015 IEEE 56th Annual Symposium on Foundations of Computer Science, pp.\ 670--688. IEEE, 2015

    work page 2015

  3. [3]

    S., and Hall, G

    Abbe, E., Bandeira, A. S., and Hall, G. Exact recovery in the stochastic block model. IEEE Transactions on Information Theory, 62 0 (1): 0 471--487, 2015

    work page 2015

  4. [4]

    Almost Surely Asymptotically Constant Graph Neural Networks

    Adam-Day, S., Benedikt, M., Ceylan, \.I . \.I ., and Finkelshtein, B. Graph neural network outputs are almost surely asymptotically constant. arXiv preprint arXiv:2403.03880, 2024

    work page Pith review arXiv 2024

  5. [5]

    Graph convolution for semi-supervised classification: Improved linear separability and out-of-distribution generalization

    Baranwal, A., Fountoulakis, K., and Jagannath, A. Graph convolution for semi-supervised classification: Improved linear separability and out-of-distribution generalization. In International Conference on Machine Learning, pp.\ 684--693. PMLR, 2021

    work page 2021

  6. [6]

    Effects of graph convolutions in multi-layer networks

    Baranwal, A., Fountoulakis, K., and Jagannath, A. Effects of graph convolutions in multi-layer networks. In International Conference on Learning Representations, 2023

    work page 2023

  7. [7]

    An iterative clustering algorithm for the contextual stochastic block model with optimality guarantees

    Braun, G., Tyagi, H., and Biernacki, C. An iterative clustering algorithm for the contextual stochastic block model with optimality guarantees. In International Conference on Machine Learning, pp.\ 2257--2291. PMLR, 2022

    work page 2022

  8. [8]

    Supervised community detection with line graph neural networks

    Chen, Z., Li, L., and Bruna, J. Supervised community detection with line graph neural networks. In International Conference on Learning Representations, 2020

    work page 2020

Show all 48 references
  1. [9]

    Contextual stochastic block models

    Deshpande, Y., Sen, S., Montanari, A., and Mossel, E. Contextual stochastic block models. In Advances in Neural Information Processing Systems, volume 31, 2018

    2018

  2. [10]

    Exact recovery and bregman hard clustering of node-attributed stochastic block model

    Dreveton, M., Fernandes, F., and Figueiredo, D. Exact recovery and bregman hard clustering of node-attributed stochastic block model. In Advances in Neural Information Processing Systems, volume 36, 2024

    2024

  3. [11]

    and Zdeborova, L

    Duranthon, O. and Zdeborova, L. Optimal inference in contextual stochastic block models. Transactions on Machine Learning Research, 2024. ISSN 2835-8856

    2024

  4. [12]

    Edition, F., Papoulis, A., and Pillai, S. U. Probability, random variables, and stochastic processes. McGraw-Hill Europe: New York, NY, USA, 2002

    2002

  5. [13]

    Graph neural networks for social recommendation

    Fan, W., Ma, Y., Li, Q., He, Y., Zhao, E., Tang, J., and Yin, D. Graph neural networks for social recommendation. In The world wide web conference, pp.\ 417--426, 2019

    2019

  6. [14]

    and Lenssen, J

    Fey, M. and Lenssen, J. E. Fast graph representation learning with PyTorch Geometric . In ICLR 2019 Workshop on Representation Learning on Graphs and Manifolds, 2019

    2019

  7. [15]

    Graph attention retrospective

    Fountoulakis, K., Levi, A., Yang, S., Baranwal, A., and Jagannath, A. Graph attention retrospective. Journal of Machine Learning Research, 24 0 (246): 0 1--52, 2023

    2023

  8. [16]

    and Karo \'n ski, M

    Frieze, A. and Karo \'n ski, M. Introduction to random graphs. Cambridge University Press, 2015

    2015

  9. [17]

    D., Kosciolek, T., Leman, J

    Gligorijevi \'c , V., Renfrew, P. D., Kosciolek, T., Leman, J. K., Berenberg, D., Vatanen, T., Chandler, C., Taylor, B. C., Fisk, I. M., Vlamakis, H., et al. Structure-based protein function prediction using graph convolutional networks. Nature communications, 12 0 (1): 0 3168, 2021

    2021

  10. [18]

    W., Laskey, K

    Holland, P. W., Laskey, K. B., and Leinhardt, S. Stochastic blockmodels: First steps. Social networks, 5 0 (2): 0 109--137, 1983

    1983

  11. [19]

    Open graph benchmark: Datasets for machine learning on graphs

    Hu, W., Fey, M., Zitnik, M., Dong, Y., Ren, H., Liu, B., Catasta, M., and Leskovec, J. Open graph benchmark: Datasets for machine learning on graphs. In Advances in Neural Information Processing Systems, volume 33, pp.\ 22118--22133, 2020

    2020

  12. [20]

    S., Levi, A., and Valera, I

    Javaloy, A., Martin, P. S., Levi, A., and Valera, I. Learnable graph convolutional attention networks. In The Eleventh International Conference on Learning Representations, 2023

    2023

  13. [21]

    Not too little, not too much: a theoretical analysis of graph (over) smoothing

    Keriven, N. Not too little, not too much: a theoretical analysis of graph (over) smoothing. In Advances in Neural Information Processing Systems, 2022

    2022

  14. [22]

    Kipf, T. N. and Welling, M. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2022

    2022

  15. [23]

    B., Rossi, R

    Lee, J. B., Rossi, R. A., Kim, S., Ahmed, N. K., and Koh, E. Attention models in graphs: A survey. ACM Transactions on Knowledge Discovery from Data, 13 0 (6): 0 1--25, 2019

    2019

  16. [24]

    E., Rivest, R

    Leiserson, C. E., Rivest, R. L., Cormen, T. H., and Stein, C. Introduction to algorithms, volume 6. MIT press Cambridge, MA, USA, 2001

    2001

  17. [25]

    Towards deeper graph neural networks

    Liu, M., Gao, H., and Ji, S. Towards deeper graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp.\ 338--348, 2020

    2020

  18. [26]

    and Sen, S

    Lu, C. and Sen, S. Contextual stochastic block model: Sharp thresholds and contiguity. Journal of Machine Learning Research, 24 0 (54): 0 1--34, 2023

    2023

  19. [27]

    When do graph neural networks help with node classification? investigating the homophily principle on node distinguishability

    Luan, S., Hua, C., Xu, M., Lu, Q., Zhu, J., Chang, X.-W., Fu, J., Leskovec, J., and Precup, D. When do graph neural networks help with node classification? investigating the homophily principle on node distinguishability. In Advances in Neural Information Processing Systems, 2023

    2023

  20. [28]

    Hyperspectral image classification using feature fusion hypergraph convolution neural network

    Ma, Z., Jiang, Z., and Zhang, H. Hyperspectral image classification using feature fusion hypergraph convolution neural network. IEEE Transactions on Geoscience and Remote Sensing, 60: 0 1--14, 2022. doi:10.1109/TGRS.2021.3123423

    2022

  21. [29]

    Reynolds, D. A. et al. Gaussian mixture models. Encyclopedia of biometrics, 741 0 (659-663), 2009

    2009

  22. [30]

    K., Bronstein, M

    Rusch, T. K., Bronstein, M. M., and Mishra, S. A survey on oversmoothing in graph neural networks. arXiv preprint arXiv:2303.10993, 2023

    2023 arXiv

  23. [31]

    Graph attention networks

    Veli c kovi \'c , P., Cucurull, G., Casanova, A., Romero, A., Li \`o , P., and Bengio, Y. Graph attention networks. In International Conference on Learning Representations, 2018

    2018

  24. [32]

    Understanding heterophily for graph neural networks

    Wang, J., Guo, Y., Yang, L., and Wang, Y. Understanding heterophily for graph neural networks. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F. (eds.), Proceedings of the 41st International Conference on Machine Learning, v...

    2024

  25. [33]

    Graph attention convolution for point cloud semantic segmentation

    Wang, L., Huang, Y., Hou, Y., Zhang, S., and Shan, J. Graph attention convolution for point cloud semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 10296--10305, 2019 a

    2019

  26. [34]

    Deep graph library: A graph-centric, highly-performant package for graph neural networks

    Wang, M., Zheng, D., Ye, Z., Gan, Q., Li, M., Song, X., Zhou, J., Ma, C., Yu, L., Gai, Y., et al. Deep graph library: A graph-centric, highly-performant package for graph neural networks. arXiv preprint arXiv:1909.01315, 2019 b

    1909 arXiv

  27. [35]

    Kgat: Knowledge graph attention network for recommendation

    Wang, X., He, X., Cao, Y., Liu, M., and Chua, T.-S. Kgat: Knowledge graph attention network for recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp.\ 950--958, 2019 c

    2019

  28. [36]

    R., and Li, P

    Wei, R., Yin, H., Jia, J., Benson, A. R., and Li, P. Understanding non-linearity in graph neural networks from the bayesian-inference perspective. In Advances in Neural Information Processing Systems, volume 35, pp.\ 34024--34038, 2022

    2022

  29. [37]

    Graph neural networks in recommender systems: a survey

    Wu, S., Sun, F., Zhang, W., Xie, X., and Cui, B. Graph neural networks in recommender systems: a survey. ACM Computing Surveys, 55 0 (5): 0 1--37, 2022 a

    2022

  30. [38]

    A non-asymptotic analysis of oversmoothing in graph neural networks

    Wu, X., Chen, Z., Wang, W., and Jadbabaie, A. A non-asymptotic analysis of oversmoothing in graph neural networks. arXiv preprint arXiv:2212.10701, 2022 b

    2022 arXiv

  31. [39]

    Demystifying oversmoothing in attention-based graph neural networks

    Wu, X., Ajorlou, A., Wu, Z., and Jadbabaie, A. Demystifying oversmoothing in attention-based graph neural networks. 36, 2024

    2024

  32. [40]

    Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., and Philip, S. Y. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 32 0 (1): 0 4--24, 2020

    2020

  33. [41]

    How powerful are graph neural networks? In International Conference on Learning Representations, 2018

    Xu, K., Hu, W., Leskovec, J., and Jegelka, S. How powerful are graph neural networks? In International Conference on Learning Representations, 2018

    2018

  34. [42]

    Revisiting over-smoothing in deep gcns

    Yang, C., Wang, R., Yao, S., Liu, S., and Abdelzaher, T. Revisiting over-smoothing in deep gcns. arXiv preprint arXiv:2003.13663, 2020

    2003 arXiv

  35. [43]

    You can't ignore either: Unifying structure and feature denoising for robust graph learning

    Yang, T., Meng, J., Zhou, M., Yang, Y., Wang, Y., Li, X., and Tong, Y. You can't ignore either: Unifying structure and feature denoising for robust graph learning. arXiv preprint arXiv:2408.00700, 2024

    2024 arXiv

  36. [44]

    Yeh, J. J. Real analysis: theory of measure and integration. World Scientific Publishing Company, 2014

    2014

  37. [45]

    Zhang, A. Y. and Zhou, H. H. Minimax rates of community detection in stochastic block models. The Annals of Statistics, 44 0 (5): 0 2252--2280, 2016

    2016

  38. [46]

    and Tan, V

    Zhang, Q. and Tan, V. Y. Exact recovery in the general hypergraph stochastic block model. IEEE Transactions on Information Theory, 69 0 (1): 0 453--471, 2022

    2022

  39. [47]

    and Akoglu, L

    Zhao, L. and Akoglu, L. Pairnorm: Tackling oversmoothing in gnns. In International Conference on Learning Representations, 2020

    2020

  40. [48]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.