Pith. sign in

REVIEW 3 major objections 4 minor 44 references

PostDeg: Placement Beats Parameterization in LayerNorm GNNs

T0 review · 3 major / 4 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A per-node scalar placed after LayerNorm survives to the score head; placed before, it is absorbed (residual at most 2.44e-5). PostDeg, a parameter-free post-LN inverse-degree scale, turns that surviving slot into +2.5% to +5.6% gains on no

desk verdict Solid placement diagnostic, but the 'parameterization is irrelevant' claim rests on a spectral-norm confound that should be fixed before acceptance. read the letter →

arxiv 2606.14022 v2 pith:ETDKKHYB submitted 2026-06-12 cs.LG

Yash Tomar , Aryav Das This is my paper
classification cs.LG
keywords LayerNormgraphneuralnetworksnodeselectiondegreecentralityplacementrulenormalizationabsorptioninfluencemaximizationmaximumindependentset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish a placement rule for topology signals in LayerNorm-based graph neural networks: a positive per-node scalar (degree, centrality, or any such multiplier) inserted before LayerNorm is numerically erased, while the same scalar inserted after LayerNorm survives as representation magnitude that the score head can read. The rule is grounded in the LayerNorm absorption identity — LN(a*z) approx LN(z) up to a stabilizer term bounded by 2.44e-5 empirically — and is instantiated by PostDeg, a parameter-free inverse-degree scale placed in the surviving post-LayerNorm slot. PostDeg improves over the LayerNorm backbone by +3.5% on influence maximization, +2.5% on network dismantling, and +5.6% on maximum independent set, with 10/10 paired-seed wins per task, while learned same-slot variants are statistically equivalent. Four pre-registered falsifiers — a graphwise spectral scalar, extra normalization, expressive same-slot capacity, and a degree-injecting backbone — all fail to reject the rule, supporting the conclusion that placement, not parameterization, carries the gain.

What carries the argument

The central object is the LayerNorm absorption identity for positive scalars: LN(a_i z_i) ≈ LN(z_i) up to the stabilizer term ε_LN/(a_i²σ²), empirically at most 2.44e-5. The paper turns it into a placement diagnostic: pre-LN positive multipliers are absorbed; post-LN positive multipliers survive as representation magnitude. The companion degree-separation ratio R_ij = ((c_i+ε)/(c_j+ε))^γ > 1 predicts that a post-LN inverse-degree scale s_i = (c_i+ε)^(-1/2) gives low-degree nodes larger magnitude with a negative log-log slope. PostDeg, the parameter-free operator occupying the surviving slot, carries the argument: learned same-slot variants are TOST-equivalent to it, showing the mechanism is

What would settle it

Raise the LayerNorm stabilizer ε_LN from 10⁻⁵ to a value comparable to feature variance (σ² ≈ 0.5). The absorption identity predicts pre-LN degree multipliers should then survive and PostDeg should lose its edge; if pre-LN multipliers stay erased under a large stabilizer, the mechanism is wrong. Complementarily, on a reward provably aligned with degree magnitude (e.g., sum of selected degrees) versus an unaligned reward at matched degree skew, the rule predicts a gain only on the aligned side.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that LayerNorm erases topology signals at a specific, locatable place in the residual block. For positive inputs, LayerNorm is almost exactly scale-invariant, so a per-node multiplier placed before it survives only through the stabilizer term (residual at most 2.44e-5 across nodes, layers, seeds, and tasks), while the same multiplier placed after it multiplies the representation and reaches the score head as magnitude contrast. A parameter-free post-LN inverse-degree scale, PostDeg, exploits that surviving slot for +3.5%/+2.5%/+5.6% on three degree-sensitive node-selection tasks with 10/10 paired-seed wins; learned same-slot variants are TOST-equiva

Load-bearing premise

The load-bearing premise is that the absorption identity proven for a single LayerNorm block carries over to the 3-layer stack, and that the Epidemic task shows no gain because its reward is not aligned with degree-conditioned magnitude — an alignment judgment the paper makes post hoc, without a formal a priori definition.

Editorial extensions

If this is right

  • Pre-LN positive scalar multipliers — degree, PageRank, k-core, learned positive attention scores — are numerically absorbed by LayerNorm in GNN residual blocks, so a policy that must rank by topology needs the signal on the post-normalization side.
  • The post-LayerNorm slot is a general surviving channel: the same placement rule should hold for any positive per-node topology scalar in any normalized residual stack, not just inverse degree.
  • A zero-parameter operator in that slot suffices: learned same-slot variants are TOST-equivalent at ±1%, so practitioners gain without adding capacity.
  • The gains (+3.5% InfluMax, +2.5% Dismantle, +5.6% MIS, 10/10 paired-seed wins per task) persist under 2–3× size transfer, making the fix usable at larger evaluation graphs.
  • The boundary predictions are part of the rule: low degree heterogeneity (DD) or a reward not aligned with degree-conditioned magnitude (Epidemic) yields a null result, and a backbone that already injects degree inside aggregation (PNA) leaves no headroom.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: the placement diagnostic should bind for other positive per-node scalars — PageRank, k-core, eigenvector centrality, learned positive attention — and the paper's own prediction sheet for standard benchmarks is a direct, runnable test of that generalization.
  • My inference: because the proof covers one LayerNorm block while the experiments stack three, whether the absorption residual compounds across depth is open; a multi-layer mean-field analysis would either harden the rule for deep GNNs or reveal a depth correction.
  • My inference: the pre/post distinction is a general probe of normalizer erasure — the same move (multiply before versus after LayerNorm) in any normalized residual network, graph or not, should reproduce the absorption pattern, giving architects a cheap test of whether their normalizer hides per-item signals.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. PostDeg claims that in LayerNorm-based GNNs, a positive per-node scalar multiplier is absorbed when inserted before LayerNorm (Eq. 1, Prop. 16), while the same scalar inserted after LayerNorm survives to the score head as representation magnitude. The paper instantiates this post-LN slot with a parameter-free inverse-degree scale (PostDeg), reports gains over the LN backbone on three node-selection tasks (+3.5%/+2.5%/+5.6%, with 10/10 paired-seed wins per task), and tests four pre-registered falsifiers (graphwise scalar, extra LayerNorm, same-slot learned capacity, and PNA backbone-agnostic source). The central claim is that placement, not parameterization, drives the improvement.

Significance. If the result holds, the paper gives a simple, explainable mechanism for why degree signals are erased in normalized GNNs and a parameter-free fix. The absorption identity is proven and empirically bounded (stabilizer residual ≤2.44×10−5 across tasks, Table A10), the experimental grid is carefully controlled with paired seeds, TOST equivalence tests and Wilcoxon tests are reported, and the code/data layout is unusually detailed. The four falsifier design is a strong example of pre-registered reasoning, and the public prediction sheet for unrun benchmarks is a welcome addition. The main weakness is a confound in the same-slot capacity falsifier that currently prevents the headline 'placement, not parameterization' conclusion from being fully supported.

major comments (3)
  1. [§4.3 and Appendix C setup] The same-slot capacity falsifier is confounded by weight spectral normalization. Appendix C states: 'Learned-variant runs also use weight spectral normalization on the GAT projections through use_weight_sn; PostDeg and the alternative-normalization baselines do not.' Thus the TOST equivalence between PostDeg and PostDeg-L-FG/Adaptive (Table A13) shows only that the combined change (extra parameters + spectral norm) is neutral within ±1%, not that parameterization is irrelevant. The additional parameters could be beneficial and the spectral norm detrimental, with cancellation inside the equivalence margin. This gap directly affects the headline claim and should be closed by reporting at least PostDeg with spectral normalization and learned variants without it (or a 2×2 design).
  2. [§4.2, Table 4] The 'Epidemic = task-objective null' label is a post hoc explanation for a near-zero result. Epidemic has degree skew 0.45, nearly identical to Dismantle (0.46), yet Dismantle gains +2.5% while Epidemic gains +0.1%. The paper attributes this to reward misalignment without giving a formal, a priori definition of 'aligned with degree-conditioned magnitude'. To make the boundary case a genuine test of the placement rule, provide a measurable definition of reward alignment or run an auxiliary experiment that manipulates the reward to flip alignment while holding graph statistics fixed.
  3. [Appendix C.10 / §3.1] The theory is proven for a single LayerNorm block (Eq. 1, Prop. 16, Thm. 3), but the experiments use a 3-layer GAT stack. The paper measures per-layer variance at convergence (Table A10) but does not show that the absorption identity remains sharp after residual interactions across layers. The claim that a post-LN scalar 'reaches the score head as representation magnitude' in a 3-layer stack needs one of: (i) a multi-layer analysis, (ii) a control applying PostDeg only at the final layer versus all layers, or (iii) explicit per-layer absorption measurements after the full stack, to bridge this gap.
minor comments (4)
  1. [Table 4 / Table A17] The Epidemic column is described as a 'task-objective null' but the win rate against LN backbone is 9/10 (Table A17). The near-zero relative improvement with a high win rate should be commented on; the text currently says 'within seed noise' while the win rate suggests a small but consistent difference.
  2. [Figure 3(e) and §4.4] The BA-vs-SBM MIS comparison uses different evaluation sizes (n=200 vs n=300). The caption acknowledges this, but the text calls it a 'dose-response' and reports ratios 1.57×–1.88×. Please present a matched-size comparison or clearly label the result as a directional finding only, as is done in Table A30.
  3. [Equation (3) and Table A20] Corollary 5 predicts a log-log slope of −γ for the scalar factor, while Table A20 reports slopes around −0.83 (InfluMax) and −0.63 (MIS). The paper explains this via Remark 2, but it would help to tabulate the predicted slope alongside the empirical slope for each task, making the discrepancy and its explanation transparent.
  4. [Abstract] The abstract says the placement rule 'generalizes to any positive topology scalar' but only degree is tested; PageRank, k-core, and centrality are explicitly out of scope in the limitations. Suggest softening to 'is expected to generalize' or adding a proof-of-concept for one additional scalar.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PostDeg's empirical claims are tested, not derived from fitted constants or self-citations.

full rationale

The paper's central claim is that a positive per-node scalar placed after LayerNorm survives to the score head while the same scalar placed before LayerNorm is absorbed. The algebraic core (Eq. 1, Proposition 16) is a standard LayerNorm identity cited to independent prior work; the empirical absorption envelope (≤2.44e-5) is measured, not assumed. PostDeg is defined by Eq. 2 as a post-LN inverse-degree scale, so the scale ordering in Eq. 3 is a definitional restatement; however the paper's load-bearing claims—the +3.5/+2.5/+5.6% gains over a fixed LN backbone, the failure of GraphScalar/Extra LayerNorm, the TOST equivalence of learned variants, and the PNA redundancy—are empirical outcomes that could have falsified the placement rule and are not implied by the definition of the operator. Learned parameters (α, β, δ) are fitted but are only used as a capacity falsifier; their near-initialization values and TOST equivalence are observations, not fitted predictions. The 'task alignment' explanation for Epidemic is post hoc, but post hoc does not mean circular. The reviewer-identified spectral-normalization confound in Appendix C is a real threat to the 'placement, not parameterization' attribution, but it is a validity/confounding concern, not a circularity: the paper does not define the conclusion into the input. No load-bearing self-citation or imported uniqueness theorem is present. Therefore no circular step meets the evidentiary bar.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central PostDeg operator has no free parameters. The learned same-slot variants introduce (α,β,δ) but the paper shows they are paired-equivalent to the fixed operator. The main unstated premise is the binary 'reward alignment' used to classify Epidemic as a null.

free parameters (3)
  • α (learned amplitude in PostDeg-L variants) = ~1.56 per layer (Table A23)
    Learned in diagnostic same-slot variants; not part of PostDeg, and shown to be paired-equivalent to fixed α=1.
  • β (learned exponent in PostDeg-L variants) = 0.752–0.768
    Learned in diagnostic; converges near initialization 0.75; equivalent to fixed 0.5 within ±1%.
  • δ (spectral coefficient in PostDeg-L variants) = ~0.25
    Learned; contributes graph-dependent exponent modulation; not needed by PostDeg.
assumptions (5)
  • standard math LayerNorm absorption identity: LN(a x) ≈ LN(x) for a>0 when ε_LN ≪ a²σ²(x) (Eq. 1 / Prop. 16).
    Standard property of LayerNorm, proved in appendix; not ad hoc.
  • domain assumption Per-node feature variance σ²(z_i) is bounded below well above the stabilizer across all layers/epochs (measured: ε_LN/σ²_min ≤ 2.44e-5, Table A10).
    Empirically verified on the five tasks; the placement rule's tightness depends on it.
  • domain assumption The score head (trained MLP + greedy decoder) uses representation magnitude as a usable signal; no other channel recovers the scalar.
    The paper's mechanism section shows scale tracks degree and PNA redundancy, but this is not a proven guarantee.
  • ad hoc to paper Task rewards in InfluMax/Dismantle/MIS are aligned with degree-conditioned magnitude, while Epidemic is not ('task-objective null').
    No formal definition of alignment; used to explain why Epidemic (same heterogeneity as Dismantle) shows ~0 gain. This is the weakest, potentially post hoc assumption.
  • domain assumption Positive scalars only; sign-changing or complex multipliers out of scope (C.10).
    The identity requires a>0; results do not extend to signed attention weights.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PostDeg: Placement Beats Parameterization in LayerNorm GNNs." pith.science (2026). https://pith.science/paper/ETDKKHYB

@misc{pith2026260614022,
  author       = {Pith},
  title        = {Pith review of: PostDeg: Placement Beats Parameterization in LayerNorm GNNs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ETDKKHYB}},
  note         = {Machine review of arXiv:2606.14022}
}
abstract

LayerNorm-based GNNs routinely erase the topology signals (degree, centrality, $k$-core) that node-selection policies should depend on, but the literature has not located where in the residual block the erasure happens. We answer that question: a positive per-node scalar inserted before LayerNorm is divided out up to a stabilizer term, while the same scalar inserted after LayerNorm reaches the score head as representation magnitude. The surviving slot is the post-LayerNorm position. We instantiate it with PostDeg, a parameter-free post-LayerNorm inverse-degree scale, and pre-register four falsifiers (graphwise scalars, extra LayerNorm, expressive same-slot capacity, backbone-agnostic source) that would reject the rule. PostDeg gains $+3.5\%/+2.5\%/+5.6\%$ over the LN backbone on influence maximization, network dismantling, and maximum independent set, with $10/10$ paired-seed wins per task; none of the four falsifiers fires. The takeaway is that placement, not parameterization, carries the gain -- a small invariance check that generalizes to any positive topology scalar in any normalized residual stack.

Figures

Figures reproduced from arXiv: 2606.14022 by the authors.

Figure 1
Figure 1. Why placement matters. Left: a graph with mixed degree (low-degree cycle nodes labeled 2, cluster nodes labeled 3, one hub labeled 5); node colors index degree. Middle: after LayerNorm, every node’s representation magnitude is normalized to the same scale, so the per-node feature variation reaching the score head no longer encodes degree contrast. Right: PostDeg multiplies each post-LayerNorm representation by si = … view at source ↗
Figure 2
Figure 2. Method overview. (a) Inserting a positive per-node scalar ai before LayerNorm is absorbed up to a stabilizer term (residual ≤ 2.44×10−5 empirically). (b) Inserting the same scalar after LayerNorm reaches the score head as representation magnitude. The shaded backbone (residual GAT block, LayerNorm, score head, budgets, data generator, seeds) is held fixed across the main grid; only the dashed post-LN slot varies. Th… view at source ↗
Figure 3
Figure 3. Robustness of PostDeg across evaluation size and degree heterogeneity. (a-d) [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

44 extracted references · 4 linked inside Pith

  1. [1]

    J. L. Ba, J. R. Kiros, and G. E. Hinton. Layer normalization.arXiv:1607.06450, 2016

    arXiv 2016

  2. [2]

    Bouritsas, F

    G. Bouritsas, F. Frasca, S. P. Zafeiriou, and M. M. Bronstein. Improving graph neural network expressivity via subgraph isomorphism counting.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1):657–668, 2022

    2022

  3. [3]

    Braunstein, L

    A. Braunstein, L. Dall’Asta, G. Semerjian, and L. Zdeborová. Network dismantling.Proceed- ings of the National Academy of Sciences, 113(44):12368–12373, 2016

    2016

  4. [4]

    Brody, U

    S. Brody, U. Alon, and E. Yahav. How attentive are graph attention networks? InInternational Conference on Learning Representations, 2022

    2022

  5. [5]

    T. Cai, S. Luo, K. Xu, D. He, T.-Y . Liu, and L. Wang. GraphNorm: A principled approach to accelerating graph neural network training. InInternational Conference on Machine Learning, 2021

    2021

  6. [6]

    Cappart, D

    Q. Cappart, D. Chételat, E. Khalil, A. Lodi, C. Morris, and P. Veli ˇckovi´c. Combinatorial optimization and reasoning with graph neural networks.Journal of Machine Learning Research, 24(130):1–61, 2023

    2023

  7. [7]

    F. R. K. Chung.Spectral Graph Theory. American Mathematical Society, 1997

    1997

  8. [8]

    Corso, L

    G. Corso, L. Cavalleri, D. Beaini, P. Liò, and P. Veliˇckovi´c. Principal neighbourhood aggregation for graph nets. InAdvances in Neural Information Processing Systems, 2020

    2020

Show all 44 references
  1. [9]

    Dehmamy, A.-L

    N. Dehmamy, A.-L. Barabási, and R. Yu. Understanding the representation power of graph neural networks in learning graph topology. InAdvances in Neural Information Processing Systems, 2019

    2019

  2. [10]

    W. L. Hamilton, R. Ying, and J. Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems, 2017

    2017

  3. [11]

    Ioffe and C

    S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. InInternational Conference on Machine Learning, 2015

    2015

  4. [12]

    C. K. Joshi, Q. Cappart, L.-M. Rousseau, and T. Laurent. Learning the travelling salesperson problem requires rethinking generalization.Constraints, 27(1–2):70–98, 2022

    2022

  5. [13]

    Karalias and A

    N. Karalias and A. Loukas. Erd ˝os goes neural: An unsupervised learning framework for combinatorial optimization on graphs. InAdvances in Neural Information Processing Systems, 2020

    2020

  6. [14]

    Kempe, J

    D. Kempe, J. Kleinberg, and É. Tardos. Maximizing the spread of influence through a social network. InProceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2003

    2003

  7. [15]

    E. B. Khalil, H. Dai, Y . Zhang, B. Dilkina, and L. Song. Learning combinatorial optimization algorithms over graphs. InAdvances in Neural Information Processing Systems, 2017

    2017

  8. [16]

    T. N. Kipf and M. Welling. Semi-supervised classification with graph convolutional networks. InInternational Conference on Learning Representations, 2017

    2017

  9. [17]

    W. Kool, H. van Hoof, and M. Welling. Attention, learn to solve routing problems! In International Conference on Learning Representations, 2019

    2019

  10. [18]

    Morris, N

    C. Morris, N. M. Kriege, F. Bause, K. Kersting, P. Mutzel, and M. Neumann. TUDataset: A collection of benchmark datasets for learning with graphs.arXiv:2007.08663, 2020

    2007 arXiv

  11. [19]

    Press and L

    O. Press and L. Wolf. Using the output embedding to improve language models. InConference of the European Chapter of the Association for Computational Linguistics, 2017

    2017

  12. [20]

    Rampasek, M

    L. Rampasek, M. Galkin, V . P. Dwivedi, A. T. Luu, G. Wolf, and D. Beaini. Recipe for a general, powerful, scalable graph transformer. InAdvances in Neural Information Processing Systems, 2022. 10

    2022

  13. [21]

    Salimans and D

    T. Salimans and D. P. Kingma. Weight normalization: A simple reparameterization to accelerate training of deep neural networks. InAdvances in Neural Information Processing Systems, 2016

    2016

  14. [22]

    R. Sato, M. Yamada, and H. Kashima. Random features strengthen graph neural networks. In SIAM International Conference on Data Mining, 2020

    2020

  15. [23]

    Ulyanov, A

    D. Ulyanov, A. Vedaldi, and V . Lempitsky. Instance normalization: The missing ingredient for fast stylization.arXiv:1607.08022, 2016

    2016 arXiv

  16. [24]

    Veliˇckovi´c, G

    P. Veliˇckovi´c, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y . Bengio. Graph attention networks. InInternational Conference on Learning Representations, 2018

    2018

  17. [25]

    Xiong, Y

    R. Xiong, Y . Yang, D. He, K. Zheng, S. Zheng, C. Xing, H. Zhang, Y . Lan, L. Wang, and T.-Y . Liu. On layer normalization in the transformer architecture. InInternational Conference on Machine Learning, 2020

    2020

  18. [26]

    C. Ying, T. Cai, S. Luo, S. Zheng, G. Ke, D. He, Y . Shen, and T.-Y . Liu. Do transformers really perform badly for graph representation? InAdvances in Neural Information Processing Systems, 2021

    2021

  19. [27]

    Zhang and R

    B. Zhang and R. Sennrich. Root mean square layer normalization. InAdvances in Neural Information Processing Systems, 2019

    2019

  20. [28]

    Zhao and L

    L. Zhao and L. Akoglu. PairNorm: Tackling oversmoothing in GNNs. InInternational Conference on Learning Representations, 2020

    2020

  21. [29]

    K. Zhou, X. Huang, Y . Li, D. Zha, R. Chen, and X. Hu. Towards deeper graph neural networks with differentiable group normalization. InAdvances in Neural Information Processing Systems, 2020

    2020

  22. [30]

    K. Zhou, Y . Dong, K. Wang, W. S. Lee, B. Hooi, H. Xu, and J. Feng. Understanding and resolving performance degradation in deep graph convolutional networks. InProceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021

    2021

  23. [31]

    K. Xu, W. Hu, J. Leskovec, and S. Jegelka. How powerful are graph neural networks? In International Conference on Learning Representations, 2019

    2019

  24. [32]

    Morris, M

    C. Morris, M. Ritzert, M. Fey, W. L. Hamilton, J. E. Lenssen, G. Rattan, and M. Grohe. Weisfeiler and Leman go neural: higher-order graph neural networks. InAAAI, 2019

    2019

  25. [33]

    F. Wu, A. Souza, T. Zhang, C. Fifty, T. Yu, and K. Weinberger. Simplifying graph convolutional networks. InInternational Conference on Machine Learning, 2019

    2019

  26. [34]

    G. Li, M. Müller, A. Thabet, and B. Ghanem. DeepGCNs: can GCNs go as deep as CNNs? In IEEE International Conference on Computer Vision, 2019

    2019

  27. [35]

    M. Chen, Z. Wei, Z. Huang, B. Ding, and Y . Li. Simple and deep graph convolutional networks. InInternational Conference on Machine Learning, 2020

    2020

  28. [36]

    Oono and T

    K. Oono and T. Suzuki. Graph neural networks exponentially lose expressive power for node classification. InInternational Conference on Learning Representations, 2020

    2020

  29. [37]

    Topping, F

    J. Topping, F. Di Giovanni, B. P. Chamberlain, X. Dong, and M. M. Bronstein. Understanding over-squashing and bottlenecks on graphs via curvature. InInternational Conference on Learning Representations, 2022

    2022

  30. [38]

    Alon and E

    U. Alon and E. Yahav. On the bottleneck of graph neural networks and its practical implications. InInternational Conference on Learning Representations, 2021

    2021

  31. [39]

    K. Xu, C. Li, Y . Tian, T. Sonobe, K. Kawarabayashi, and S. Jegelka. Representation learning on graphs with jumping knowledge networks. InInternational Conference on Machine Learning, 2018. 11

    2018

  32. [40]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. InAdvances in Neural Information Processing Systems, 2017

    2017

  33. [41]

    Santurkar, D

    S. Santurkar, D. Tsipras, A. Ilyas, and A. Madry. How does batch normalization help optimiza- tion? InAdvances in Neural Information Processing Systems, 2018

    2018

  34. [42]

    effective heterogeneity

    T. K. Rusch, M. M. Bronstein, and S. Mishra. A survey on oversmoothing in graph neural networks.arXiv preprint arXiv:2303.10993, 2023. 12 Table of Contents in Appendix Contents A Reading guide and summary of evidence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ....

    2023 arXiv

  35. [43]

    Wins” is the number of paired seeds (out of 5) where PostDeg beats LN backbone. “∆%

    is a pre-aggregation degree scaler, so its degree dependence enters through messages. Graph- SAGE’s mean aggregator [10] averages over neighborhoods and is degree-aware through that mean. Structural encodings [2, 9, 20, 22, 26] concatenate degree- or centrality-based features ...

  36. [44]

    Institutional review board (IRB) approvals or equivalent for research with human subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals...

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.