Pith. sign in

REVIEW 3 major objections 4 minor 18 references

Personalized Layer Selection for Graph Neural Networks

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Every node has its own best GNN layer, and MetSelect finds it without retraining.

desk verdict A novel and plausible layer-selection heuristic, but the defining equations select the least-confident layer and the self-loop table contradicts the text; needs major corrections before the results can be trusted. read the letter →

arxiv 2501.14964 v2 pith:3XMUEARF submitted 2025-01-24 cs.LG

Kartik Sharma , Vineeth Rakesh , Yingtong Dou , Srijan Kumar , Mahashweta Das This is my paper · ORCID
classification cs.LG
keywords graphneuralnetworknodeclassificationpersonalizedlayerselectionmetriclearningoversmoothingheterophilyadversarialrobustnessprototypedistance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper challenges the standard practice of classifying every node from a GNN's final layer. Different nodes in a graph may need different amounts of neighborhood smoothing, and forcing one layer on all nodes can hurt accuracy. MetSelect measures, for each node, how close its representation in each layer is to the class prototype after variance normalization, and picks the layer with the smallest normalized distance. The authors show this per-node layer choice improves accuracy on heterophilic datasets, allows GNNs to go much deeper without the usual accuracy collapse, and makes them more robust to training-time link poisoning. If the paper is right, existing GNN representations already contain enough information for better prediction once the right layer is chosen for each node.

What carries the argument

The core object is the variance-normalized Mahalanobis distance from a node's representation in a layer to the class prototypes in that layer: $d^{(l)}_{i,c} = (\tilde{h}^{(l)}_i - \mu^{(l)}_c)^\top K_l^{-1} (\tilde{h}^{(l)}_i - \mu^{(l)}_c)$, where $\mu^{(l)}_c$ is the mean embedding of labeled nodes of class $c$ and $K_l$ is that layer's empirical covariance. MetSelect selects the layer $l^*(v)$ that minimizes the softmax of these distances over classes. This object carries the argument because it makes representations from different layers comparable, letting the method pick a layer without modifying the GNN's weights.

What would settle it

For a labeled graph, train a GNN, then separately record which layer, if used for prediction, actually labels each node correctly; if those oracle-best layers match MetSelect's choices no more often than chance, the claim that MetSelect finds the optimal layer is false.

Watch

Extended reading notes

Core claim

The paper claims that every node in a graph has its own best representation depth inside a fixed GNN, and that this layer can be found without retraining the GNN. MetSelect compares each node's representation in every layer with per-class prototype embeddings and picks the layer that minimizes the variance-normalized Mahalanobis distance to the class prototypes. Using the selected layer for prediction instead of the final layer improves node classification accuracy on heterophilic graphs, keeps accuracy from collapsing when the GNN is deepened to 128 layers, and reduces the damage from untargeted poisoning attacks. The method works in a plug-and-play way with GCN, GAT, and GIN encoders without altering their message-passing weights.

Load-bearing premise

The method relies on the assumption that the layer whose examples of a class are closest to the class average, after scaling by how spread out the layer is, will also be the best layer for predicting unlabeled nodes.

Editorial extensions

If this is right

  • GNNs can be trained to much greater depth (up to 128 layers) without the usual oversmoothing collapse in accuracy, because each node ignores layers that have already smoothed away its signal.
  • Node classification on heterophilic graphs improves by up to 20% over the final-layer baseline, since the method can automatically fall back to a node's own features (layer 0) when neighbors carry misleading label information.
  • Models trained with per-node layer selection are substantially more robust to untargeted structural poisoning attacks (Mettack): GCN+MetSelect preserved test accuracy that the standard GCN lost.
  • The improvement is plug-and-play: the same GCN, GAT, or GIN encoder is used unchanged, with only the prediction layer and the training loss replaced.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension would be to check whether the same prototype-distance rule can select not just depth but also other per-node choices, such as which aggregation operator or which feature subset to trust.
  • The robustness gain hints that adversarial structural perturbations often corrupt higher-order neighborhoods first, so a per-node layer choice acts as a cheap, unintended defense; this is an inference, not a claim of the paper.
  • If the method is right, a GNN's learned representations are already rich across layers and the bottleneck is the fixed decoding layer, which suggests that decoder-side personalization deserves more attention than architecture changes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The manuscript proposes MetSelect, a method that selects a personalized GNN layer for each node by comparing variance-normalized distances to class prototypes across the layers of a pretrained GNN encoder. The method is intended to be used as a plug-and-play replacement for final-layer prediction: for each node it chooses the layer whose representation is closest to the corresponding class prototype, trains per-layer decoders, and uses the selected layer at inference time. Experiments on 10 datasets with GCN, GAT, and GIN report consistent accuracy gains, especially on heterophilic graphs, and additional experiments report improved depth stability and robustness to structural poisoning attacks.

Significance. If the method works as described, the contribution is practically useful: it offers a simple, architecture-agnostic way to improve an existing GNN's predictions without changing message passing, and it is backed by a broad evaluation (10 datasets, 3 base GNNs, ablations, depth and robustness studies). The paper also provides explicit time and space complexity analyses. However, the central selection rule is defined in a way that contradicts the stated objective, and Algorithm 1 as written does not compute the moments before using them. These issues affect every reported result, so the empirical claims cannot currently be verified as stated.

major comments (3)
  1. [Section 5, Eqs. (4) and (5)] The selection rule in Eqs. (4) and (5) is inverted relative to the stated objective. The quantity being minimized is a softmax probability of the form exp(-d)/Z, which is monotonically decreasing in the Mahalanobis distance d. Thus Eq. (5), argmin_l of the true-class softmax probability, selects the layer in which the true class is least probable under the distance-based decoding, not the layer closest to the true class prototype. Eq. (4) is similarly problematic: min_y over the softmax probability picks the least probable class, and the outer argmin over layers then selects the layer with the lowest confidence in that least probable class. This is not a 'minimum distance to a class prototype' rule. Since Algorithm 1 and all downstream experiments use Eq. (5), the authors must either correct these equations to argmin_l d_{i,y_i} (or equivalently argmax_l of the true-class softmax probability) and re-examine the experiments, or clarify that the implemented method is different from the one described.
  2. [Algorithm 1, lines 3–13] The training procedure as written is internally inconsistent. In each epoch, the loop over training nodes accumulates \hat{\mu} and \hat{K}_{l,0} and also computes the loss using l* (lines 6–9), but the class means \mu^{(l)}_c and covariances K_l are only updated afterward at lines 12–13. Consequently, the l* used in the loss is not computed from the moments of the current forward pass; in the first epoch those moments are zero. This contradicts the surrounding text, which states that the moments are found first and then used to minimize the loss. The algorithm needs to be restructured so that the moments are computed before l* is evaluated, and the description of the actual implementation must be made consistent with this order.
  3. [Section 7.2 and Figures 3–4] The experimental reporting for the depth and robustness studies confuses the two loss functions. Section 7.2 says 'we use the distance-based loss function in Equation 6' and then refers to 'the linear loss (i.e., Equation 6)', but Eq. (6) is the personalized cross-entropy loss while Eq. (7) is the distance-based loss. The robustness section likewise states that the model was trained using Eq. (7). It is therefore unclear which loss was actually used for Figure 3 and Figure 4, and this ambiguity affects the reproducibility of the depth and robustness claims. Please specify the loss used in each experiment and correct the cross-references.
minor comments (4)
  1. [Section 5, Eq. (3)] The covariance formula drops the summation over nodes in the second term: as written, \sum_c 1[y_i=c] (\mu^{(l)}_c)(\mu^{(l)}_c)^\top is not summed over i. It should be \sum_i \sum_c 1[y_i=c](\mu^{(l)}_c)(\mu^{(l)}_c)^\top, or equivalently \sum_c N_c \mu^{(l)}_c (\mu^{(l)}_c)^\top as used in Algorithm 1.
  2. [Section 7.4, Table 2] The ablation baseline MetSelect-max should be revisited after Eqs. (4) and (5) are corrected; if the implemented selection is actually the argmin of a softmax probability, then what is called the 'polar opposite' baseline may coincide with the intended criterion. The interpretation of this ablation depends on resolving the definitional issue.
  3. [Section 6 and Table 1] There are several small naming and typographical inconsistencies, including 'Citseer' in Table 2, 'Ogba' in Table 3 versus 'ogbn-arxiv' elsewhere, and 'Mahlanabois' in Section 5. These should be cleaned up.
  4. [Section 8] The paper honestly notes that proving a particular layer is optimal is extremely hard; this limitation is acceptable for an empirical method paper, but it should be tied to the choice of prototypes and covariance estimates. The authors could acknowledge more explicitly that their selection rule is a heuristic that depends on the quality of the training-set prototype estimates.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the paper's claims are empirical and evaluated on held-out labels, with no load-bearing self-citation or definitional equivalence.

full rationale

The paper's central claim is an empirical method claim: MetSelect chooses a per-node layer using class prototypes and covariances estimated from training nodes, and its accuracy, depth, and robustness results are measured on held-out test labels. The layer selection is not fitted to test labels, so the reported gains are not forced by construction. The prototype/variance construction (Eqs. 2-3) is an externally motivated metric-learning design (Rippel et al.; Salakhutdinov and Hinton), not a result derived from the paper's own prior work, and no self-citation chain is load-bearing. The paper itself concedes that proving layer optimality is extremely hard, which undercuts any claim that the selection rule is tautologically optimal. The manuscript does contain a serious internal inconsistency: the prose says the selected layer minimizes normalized distance to a class prototype, while Eqs. (4)-(5) define l* as the argmin of exp(-d)/sum exp(-d), a decreasing function of distance, which appears to select the opposite of the stated objective. However, this is a correctness/verifiability problem (especially given the absence of released code), not a circular reduction of a prediction to its inputs. No equation in the paper is equivalent to its own inputs by construction, and no fitted parameter is renamed as a prediction. Therefore the circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the equivalence of optimal layer and minimum variance-normalized prototype distance. That equivalence is assumed, not proven. No new physical or conceptual entities are introduced; MetSelect is an algorithmic decision rule built from existing GNN representations.

free parameters (3)
  • Trainable projection W0,b0 and per-layer transforms W,b
    Features are projected to a d-dimensional space and each layer's representation is transformed before distance computation; these matrices and biases are learned on training data and their final values are not reported.
  • Per-layer decoders g_Phi^(l)
    L+1 decoder networks, one per layer, are trained with the personalized cross-entropy loss (Eq. 6); their parameters are fit to data.
  • Alpha in the distance loss (Eq. 7) = 1
    The margin in L_distance is set to 1 without tuning; main experiments use L_CE, so this hyperparameter does not drive the headline results.
assumptions (4)
  • domain assumption The layer with the smallest variance-normalized distance from the class prototype is the optimal prediction layer for that node.
    This is the selection rule in Eq. (4); the paper calls the result optimal but Section 8 admits that proving optimality is extremely hard.
  • domain assumption Class prototypes and covariances estimated from labeled training nodes are representative for unlabeled nodes in each layer.
    Inference selects l*(v) for test nodes using prototypes from V_tr only; no distribution-shift analysis is provided.
  • standard math The empirical covariance K_l is invertible and Mahalanobis distances are well-defined for every layer.
    Eq. (4) requires K_l^{-1}; the paper does not describe regularization for singular or poorly conditioned covariances, which can occur on small datasets.
  • domain assumption Layer representations h^(l) can be compared across depths as independent candidate prediction spaces.
    The method treats each layer as a separate candidate space, though all layers come from one backbone and are statistically dependent; the effect of this dependence is not analyzed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Personalized Layer Selection for Graph Neural Networks." pith.science (2026). https://pith.science/paper/3XMUEARF

@misc{pith2026250114964,
  author       = {Pith},
  title        = {Pith review of: Personalized Layer Selection for Graph Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3XMUEARF}},
  note         = {Machine review of arXiv:2501.14964}
}
read the original abstract

Graph Neural Networks (GNNs) combine node attributes over a fixed granularity of the local graph structure around a node to predict its label. However, different nodes may relate to a node-level property with a different granularity of its local neighborhood, and using the same level of smoothing for all nodes can be detrimental to their classification. In this work, we challenge the common fact that a single GNN layer can classify all nodes of a graph by training GNNs with a distinct personalized layer for each node. Inspired by metric learning, we propose a novel algorithm, MetSelect1, to select the optimal representation layer to classify each node. In particular, we identify a prototype representation of each class in a transformed GNN layer and then, classify using the layer where the distance is smallest to a class prototype after normalizing with that layer's variance. Results on 10 datasets and 3 different GNNs show that we significantly improve the node classification accuracy of GNNs in a plug-and-play manner. We also find that using variable layers for prediction enables GNNs to be deeper and more robust to poisoning attacks. We hope this work can inspire future works to learn more adaptive and personalized graph representations.

Figures

Figures reproduced from arXiv: 2501.14964 by the authors.

Figure 1
Figure 1. GNNs with personalized prediction layer can empower existing GNNs for more [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. GNN with node-optimal prediction layer trained using MetSelect. Labeled nodes are denoted in [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Test Accuracy of GCN and GCN+MetSelect with varying depths for different datasets. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Accuracy of GCN and GCN+MetSelect after different strengths of Mettack perturbations. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Proportion of test nodes with a particular personalized layer [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 4 canonical work pages

  1. [3]

    Sub-center arcface: Boosting face recognition by large-scale noisy web faces

    Jiankang Deng, Jia Guo, Tongliang Liu, Mingming Gong, and Stefanos Zafeiriou. Sub-center arcface: Boosting face recognition by large-scale noisy web faces. InComputer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI 16, pp. 741–757. Springer,

    work page 2020

  2. [8]

    Geom-gcn: Geometric graph convolutional networks.arXiv preprint arXiv:2002.05287,

    Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. Geom-gcn: Geometric graph convolutional networks.arXiv preprint arXiv:2002.05287,

    arXiv 2002

  3. [9]

    Metric Learning with Adaptive Density Discrimination

    Oren Rippel, Manohar Paluri, Piotr Dollar, and Lubomir Bourdev. Metric learning with adaptive density discrimination. arXiv preprint arXiv:1511.05939,

    work page Pith review arXiv

  4. [10]

    Dropedge: Towards deep graph convolutional networks on node classification.arXiv preprint arXiv:1907.10903,

    Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. Dropedge: Towards deep graph convolutional networks on node classification.arXiv preprint arXiv:1907.10903,

    arXiv 1907

  5. [13]

    Masked label prediction: Unified message passing model for semi-supervised classification.arXiv preprint arXiv:2009.03509,

    Yunsheng Shi, Zhengjie Huang, Shikun Feng, Hui Zhong, Wenjin Wang, and Yu Sun. Masked label prediction: Unified message passing model for semi-supervised classification.arXiv preprint arXiv:2009.03509,

    arXiv 2009

  6. [14]

    Non-target-specific node injection attacks on graph neural networks: A hierarchical reinforcement learning approach

    13 Published in Transactions on Machine Learning Research (04/2025) Yiwei Sun, Suhang Wang, Xianfeng Tang, Tsung-Yu Hsieh, and Vasant Honavar. Non-target-specific node injection attacks on graph neural networks: A hierarchical reinforcement learning approach. InProc. WWW, volume 3,

    work page 2025

  7. [15]

    Graph attention networks.arXiv preprint arXiv:1710.10903,

    Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks.arXiv preprint arXiv:1710.10903,

    arXiv

  8. [16]

    Topology attack and defense for graph neural networks: An optimization perspective.arXiv preprint arXiv:1906.04214,

    Kaidi Xu, Hongge Chen, Sijia Liu, Pin-Yu Chen, Tsui-Wei Weng, Mingyi Hong, and Xue Lin. Topology attack and defense for graph neural networks: An optimization perspective.arXiv preprint arXiv:1906.04214,

    arXiv 1906

Show all 18 references
  1. [17]

    How powerful are graph neural networks? arXiv preprint arXiv:1810.00826, 2018a

    Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826, 2018a. Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. Representation learning on graphs wi...

    arXiv

  2. [18]

    Pairnorm: Tackling oversmoothing in gnns

    Lingxiao Zhao and Leman Akoglu. Pairnorm: Tackling oversmoothing in gnns. arXiv preprint arXiv:1909.12223,

    1909 arXiv

  3. [2007]

    Outrageously large neural networks: The sparsely-gated mixture-of-experts layer.arXiv preprint arXiv:1701.06538,

    Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer.arXiv preprint arXiv:1701.06538,

    arXiv

  4. [2015]

    Semi-supervised classification with graph convolutional networks.arXiv preprint arXiv:1609.02907,

    Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks.arXiv preprint arXiv:1609.02907,

    arXiv

  5. [2017]

    How attentive are graph attention networks?arXiv preprint arXiv:2105.14491,

    Shaked Brody, Uri Alon, and Eran Yahav. How attentive are graph attention networks?arXiv preprint arXiv:2105.14491,

    arXiv

  6. [2019]

    Adaptive universal generalized pagerank graph neural network.arXiv preprint arXiv:2006.07988,

    Eli Chien, Jianhao Peng, Pan Li, and Olgica Milenkovic. Adaptive universal generalized pagerank graph neural network.arXiv preprint arXiv:2006.07988,

    2006 arXiv

  7. [2020]

    Graph neural networks as gradient flows.arXiv preprint arXiv:2206.10991,

    Francesco Di Giovanni, James Rowbottom, Benjamin P Chamberlain, Thomas Markovich, and Michael M Bronstein. Graph neural networks as gradient flows.arXiv preprint arXiv:2206.10991,

    arXiv

  8. [2022]

    Revisiting graph neural networks: All we have is low-pass filters.arXiv preprint arXiv:1905.09550,

    Hoang Nt and Takanori Maehara. Revisiting graph neural networks: All we have is low-pass filters.arXiv preprint arXiv:1905.09550,

    1905 arXiv

  9. [2023]

    Drew: Dynamically rewired message passing with delay

    12 Published in Transactions on Machine Learning Research (04/2025) Benjamin Gutteridge, Xiaowen Dong, Michael M Bronstein, and Francesco Di Giovanni. Drew: Dynamically rewired message passing with delay. InInternational Conference on Machine Learning, pp. 12252–12267. PMLR,

    2025

  10. [2024]

    Gradient gating for deep multi-rate learning on graphs.arXiv preprint arXiv:2210.00513,

    T Konstantin Rusch, Benjamin P Chamberlain, Michael W Mahoney, Michael M Bronstein, and Siddhartha Mishra. Gradient gating for deep multi-rate learning on graphs.arXiv preprint arXiv:2210.00513,

    arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.