Pith. sign in

REVIEW 3 major objections 5 minor 3 references

Comment on "A Note on Over-Smoothing for Graph Neural Networks"

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A spectral proof establishes that, under a mild eigenvalue condition, the Dirichlet energy of GNN node embeddings decays exponentially with depth, covering Leaky-ReLU activations and polynomial filters.

desk verdict Leaky-ReLU proof is solid, but Lemma 3.1 is false without a spectral condition; the paper's headline rate result is not proven as stated. read the letter →

arxiv 2509.04178 v1 pith:XKAD4S5T submitted 2025-09-04 cs.LG

Razi Hasson , Reuven Guetta This is my paper
classification cs.LG MSC 05C5068T07
keywords over-smoothinggraphneuralnetworksDirichletenergyspectraltheoryLeaky-ReLUpolynomialfiltersaugmentednormalizedLaplacianexponentialconvergence
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This comment on a prior over-smoothing note tries to establish a sharper and more general formalization of over-smoothing in graph neural networks: if the smallest nonzero eigenvalue of the augmented normalized Laplacian is λ and the product of spectral norms of all weight matrices stays below 1/(1−λ)², then the Dirichlet energy of node embeddings decays exponentially with layer count. The authors prove this for GCN layers with ReLU and Leaky-ReLU activations, and they extend the bound to layers that apply a polynomial of the Laplacian before the nonlinearity. A sympathetic reader would care because the result makes precise why deeper GNNs lose discriminative power and because the proof isolates exactly which operations shrink energy: graph propagation, weight matrices, and activation functions. The paper also reports experiments suggesting edge deletion or heavy edge-weight amplification can raise energy and thus partially counteract over-smoothing.

What carries the argument

The central object is Dirichlet energy E(X)=tr(X^T Δ̃ X), measured against the augmented normalized Laplacian Δ̃=I−D̃^(−1/2)ÃD̃^(−1/2); it quantifies how different neighboring node embeddings are. The proof treats energy as a budget that each layer operation can only shrink or multiply by a known factor: the propagation matrix P=I−Δ̃ multiplies by at most (1−λ)² (λ = smallest nonzero eigenvalue), a weight matrix W multiplies by the squared spectral norm of W^T, and any positively homogeneous 1-Lipschitz activation (ReLU or Leaky-ReLU) multiplies by at most 1. For the polynomial extension, a symmetric polynomial P_l(Δ̃) of the Laplacian multiplies energy by at most P_l(λ)² when the polynomial

What would settle it

Take a graph whose augmented normalized Laplacian has λ_N + λ_2 > 2 (for example a star or a path), draw a random non-constant X, and compute the ratio E(PX)/E(X). If the ratio exceeds (1−λ)² for any such X, Lemma 3.1 fails; with such a graph, training a deep Leaky-ReLU GCN with s λ̄ < 1 and measuring Dirichlet energy per layer would reveal whether the exponential decay predicted by the corollary actually occurs.

Watch

Extended reading notes

Core claim

The central claim is that over-smoothing is not an artifact of a specific architecture: for any GNN layer of the form f_l(X)=MLP_l(PX), and for the generalized layer f_l(X)=MLP_l(P_l(Δ̃)X), the Dirichlet energy satisfies E(f_l(X)) ≤ s_l λ̄ E(X), where s_l is the product of the largest singular values of the layer's weight matrices and λ̄=(1−λ)² with λ the smallest nonzero eigenvalue of Δ̃. Iterating gives E(X^(L)) = O((s λ̄)^L). When s λ̄ < 1, the bound forces exponential convergence of the embeddings to the constant subspace, the precise content of over-smoothing. The proof is built from three lemmas: graph propagation contracts energy by at most (1−λ)², multiplying by a weight matrix infla

Load-bearing premise

The argument rests on assuming that one propagation step shrinks every non-constant embedding pattern by a factor set by the smallest nonzero Laplacian eigenvalue; for graphs whose largest Laplacian eigenvalue is close to 2, some patterns may shrink less than this factor, breaking the bound.

Editorial extensions

If this is right

  • Whenever s λ̄ < 1, every deep GNN in the covered family converges to the constant-error subspace at an exponential rate, so beyond a modest depth the embeddings no longer separate nodes.
  • Leaky-ReLU layers obey exactly the same energy bound as ReLU layers, so switching activations does not rescue a deep GCN from over-smoothing.
  • Spectral polynomial filters also suffer exponential over-smoothing under the stated monotonicity condition, meaning the phenomenon is tied to the propagation operator rather than to a single filter choice.
  • The experimental observation that edge deletion or extreme edge-weight increases raise Dirichlet energy suggests practical graph-editing strategies to retard the decay.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the energy decomposition is layerwise, a testable design rule follows: measure s_l λ̄ at each layer and normalize weights so the product stays below 1; this would give a practical early warning for over-smoothing.
  • The same three-lemma factorization suggests that any activation that is positively homogeneous and 1-Lipschitz would inherit the result, and on regular graphs possibly even Tanh or Sigmoid, making exponential decay a property of graph propagation rather than of ReLU specifically.
  • A sharper spectral constant may be obtainable: on graphs where λ_N + λ_2 > 2, Lemma 3.1's uniform factor could fail, so the true worst-case factor should be something like max(λ_2, 2−λ_N); testing this on star-like graphs would map the boundary of the theorem.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This manuscript comments on Cai and Wang (2020), aiming to provide a simpler proof that node embeddings in GCNs lose expressive power exponentially with depth. The authors define the augmented normalized Laplacian Δ̃ and the propagation matrix P = I − Δ̃, and use Dirichlet energy E(X) = tr(XᵀΔ̃X). The main theorem (Thm 3.4) claims that for a GCN layer with ReLU/Leaky-ReLU activations, E(f_l(X)) ≤ s_l (1−λ)² E(X), where λ is the smallest nonzero eigenvalue of Δ̃ and s_l is the product of the largest singular values of the weight matrices in the layer. Corollary 3.5 then gives exponential decay when s(1−λ)² < 1. Section 7 extends the analysis to spectral polynomial filters P_l(Δ̃), proving an analogous contraction under monotonicity assumptions, and includes a proof of Lemma 3.3 for Leaky-ReLU. The paper also reports experiments on edge deletion and weight amplification, suggesting practical ways to mitigate over-smoothing.

Significance. If the main claim were established, the paper would provide a clean, elementary alternative to the more technical results of Oono and Suzuki (2019), and would extend the over-smoothing analysis to Leaky-ReLU activations and to polynomial filters. The Leaky-ReLU proof in Section 7 is a genuine contribution and appears correct for scalar fields, with a plausible extension to vector fields. The experimental section is illustrative and could offer useful intuitions. However, the central contraction bound rests on a spectral inequality that is not valid in general, and the polynomial extension has a similar gap. The paper's value is therefore conditional on correcting these load-bearing steps.

major comments (3)
  1. [Lemma 3.1 (§3)] The statement E(PX) ≤ (1−λ)²E(X) is not true for all graphs. Since P = I − Δ̃, the exact bound is E(PX) = Σ_i c_i² λ_i (1−λ_i)² ≤ (max_i |1−λ_i|)² E(X), and the maximum is not necessarily attained at the smallest nonzero eigenvalue λ. The claimed inequality requires |1−λ_N| ≤ 1−λ, i.e. λ_2 + λ_N ≤ 2 after ordering eigenvalues. This is a nontrivial spectral condition that is not stated. For example, on K_{3,3} with self-loops the augmented normalized Laplacian has eigenvalues {0, 0.75, 1.5}; taking X in the 1.5 eigenspace gives E(PX) = 1.5·(1−1.5)² = 0.375, whereas (1−0.75)²E(X) = 0.0625·1.5 = 0.09375, directly falsifying Lemma 3.1. Consequently Theorem 3.4 and Corollary 3.5 are quantitatively wrong as stated: one can have s(1−λ)² < 1 while the high-frequency component is amplified. The theorem must be restated with the contraction factor sup_i |1−λ_i| or under an explicit spectral assump
  2. [Lemma 7.2 (§7)] The proof's last inequality, Σ c_i² λ_i P_l(λ_i)² ≤ P_l(λ)² E(f), is not justified. From the mere fact that P_l is monotonically decreasing on [0,2) it does not follow that λ_i P_l(λ_i)² ≤ λ P_l(λ)² for all eigenvalues λ_i. If P_l takes negative values or decays slowly at larger eigenvalues, the left-hand side can exceed the right-hand side. The assumption s_l P_l(λ_i)² < 1−ε for all i does not repair this pointwise comparison, because the inequality still needs a uniform bound on λ_i P_l(λ_i)². The correct contraction rate is max_i |P_l(λ_i)|², not P_l(λ)². This is a load-bearing gap in Proposition 7.1, which claims exponential convergence for polynomial filters.
  3. [Theorem 3.4 / Corollary 3.5 (notation and iteration)] The statement of Theorem 3.4 gives E(f_l(X)) ≤ s_l λ̄ E(X) for a single layer, with s_l = ∏_h s_{lh}. Corollary 3.5 then asserts E(X^{(L)}) ≤ O((s λ̄)^l). This requires that the bound holds uniformly over all layers with a common sup s, and that the contraction factors are multiplicative across layers. While this is implicitly intended, the notation 's' is overloaded (layer index vs. global sup) and the 'O' hides the initial energy E(X^{(0)}). More importantly, because Lemma 3.1 is false, this corollary inherits the error. The authors should either prove the corrected contraction with the full spectral sup or state the extra spectral condition as a hypothesis of the theorem.
minor comments (5)
  1. [Abstract / Introduction] The abstract reads as if it were describing the commented paper rather than this comment: phrases like 'this paper extends the analysis' and 'the paper provides a clean and intuitive way' are ambiguous. The authors should consistently distinguish their own contribution from that of Cai and Wang.
  2. [Section 5] There is a mismatch in wording: the text repeatedly says 'removing nodes' while the experiments appear to concern edge deletion. Please clarify which graph operation is applied, and define the exact randomization used.
  3. [Section 6] The conclusion correctly notes that the weight-norm assumption may be too strict and that the 'real over-smoothing effect' remains open. These caveats should appear earlier, near the statement of Theorem 3.4, so readers are not misled into accepting an unconditional exponential-decay claim.
  4. [Proof of Lemma 3.3] The proof for Leaky-ReLU is essentially correct, but the line '|c_1 a − c_2 b| ≥ |σ(c_1 a) − σ(c_2 b)|' should be justified explicitly, as it is the core of the contraction argument. The vector-field extension is only sketched; a short display of the sum over feature dimensions would make the proof self-contained.
  5. [Global] Typos and notational inconsistencies include 'relive over-smoothing' for 'relieve', 'Lipshitz' for 'Lipschitz', and inconsistent use of λ̄ in Corollary 3.5 (λ̄ is defined as (1−λ)², but then the exponent is written as 'l' while the layer index is L). A careful copy edit is needed.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the derivation is self-contained conditional on its stated spectral assumptions; the flagged issues are mathematical gaps, not circular reductions.

full rationale

The paper is a comment that re-derives and extends the over-smoothing bound of Cai & Wang using Dirichlet energy. Its central chain—Lemma 3.1 (spectral contraction of P), Lemma 3.2 (weight-matrix singular-value bound), Lemma 3.3 (ReLU/Leaky-ReLU contraction), Theorem 3.4 and Corollary 3.5—is a direct spectral decomposition argument. The bound E(PX) ≤ (1−λ)^2 E(X) is stated as a lemma and used as an input; it is not obtained by fitting or by renaming a prediction. The later polynomial-filter result (Section 7) is explicitly conditional on assuming per-eigenvalue contraction s_l P_l(λ_i)^2 < 1−ε and on monotonicity of P_l; under those assumptions the conclusion follows by spectral calculus. No parameter is fitted to data and then called a prediction; the experiments illustrate the effect of edge edits on energy and do not pretend to validate the theory by fitting. The self-referential label 'Proposition 7.1 (Razi-Reuven 2024)' is a naming convention rather than a load-bearing citation to prior work by the authors. Section 6 itself flags the strictness of the weight-norm assumption and states that the 'real over-smoothing effect, measured by the Rayleigh quotient for deep GNNs, remains an open problem'—these are acknowledged limitations, not circular moves. The questionable inequalities in Lemmas 3.1 and 7.2 (replacing a maximum over the spectrum by the smallest nonzero eigenvalue) are mathematical correctness risks, not circularity: they do not make the conclusion equivalent to an input by definition. The derivation is therefore self-contained conditional on its stated (if under-specified) spectral hypotheses.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new free parameters or invented entities. It relies on spectral properties of the augmented normalized Laplacian and on unstated inequalities in the proofs of Lemma 3.1 and Lemma 7.2.

assumptions (4)
  • standard math Eigenvalues of the augmented normalized Laplacian Delta_tilde lie in [0,2).
    Invoked in Section 2 to justify eigenvalue bounds for P. Standard for normalized Laplacians.
  • ad hoc to paper The contraction factor in Lemma 3.1 is (1 - lambda)^2, which requires |1 - lambda_i| <= 1 - lambda for all non-zero eigenvalues.
    This is not established. It implicitly assumes lambda_N + lambda_2 <= 2, which is not stated or proven.
  • ad hoc to paper For Lemma 7.2, lambda_i P_l(lambda_i)^2 <= lambda P_l(lambda)^2 for all eigenvalues lambda_i.
    This inequality does not follow from monotonicity of P_l alone; the proof lacks justification.
  • domain assumption There exists a uniform bound s = sup_l s_l on weight singular values, and s * lambda_bar < 1 for convergence.
    Standard normalization assumption for GNN weights, stated in Corollary 3.5.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Comment on "A Note on Over-Smoothing for Graph Neural Networks"." pith.science (2026). https://pith.science/paper/XKAD4S5T

@misc{pith2026250904178,
  author       = {Pith},
  title        = {Pith review of: Comment on "A Note on Over-Smoothing for Graph Neural Networks"},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XKAD4S5T}},
  note         = {Machine review of arXiv:2509.04178}
}
read the original abstract

We comment on Cai and Wang (2020, arXiv:2006.13318), who analyze over-smoothing in GNNs via Dirichlet energy. We show that under mild spectral conditions (including with Leaky-ReLU), the Dirichlet energy of node embeddings decreases exponentially with depth; we further extend the result to spectral polynomial filters and provide a short proof for the Leaky-ReLU case. Experiments on edge deletion and weight amplification illustrate when Dirichlet energy increases, hinting at practical ways to relieve over-smoothing.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

3 extracted references · 1 canonical work pages

  1. [1]

    Cai and Y

    [Cai & Wang, 20] C. Cai and Y. Wang. A note on Over-Smoothing for Graph Neural Networks. arXiv:2006.13318,

    arXiv 2006

  2. [2019]

    [Li et al., 18] Li, Q., Han, Z., and Wu, X. -M. (2018). Deeper insights into graph convolutional networks for semi -supervised learning. In Proceedings of the Thirty -Second AAAI Conference on Artificial Intelligence (AAAI-18). Contact Info Razihasson@campus.technion.ac.il

    work page 2018

  3. [2020]

    Oono and T

    [Oono & Suzuki, 19] K. Oono and T. Suzuki. Graph Neural networks Exponentially Lose Expressive Power for Node Classification. arXiv:1905.10947,

    arXiv 1905

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.