{"id":"da844f5d-b976-402a-88d2-b7db786c3cdf","arxiv_id":"2412.04064","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A Cluster-Normalize-Activate module replaces plain activations in GNNs, limiting oversmoothing and improving accuracy with fewer parameters.","lead":"The paper introduces CNA, a plug-and-play module for graph neural networks that clusters node features, normalizes each cluster, and applies learnable activation functions per cluster. It claims this reduces oversmoothing and improves accuracy on node classification, graph classification, and regression benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SOTA claim is not yet supported: Table 4 compares best CNA results per dataset to PwC leaderboard numbers without stating the train/val/test splits, so the headline 94.18/95.75 wins may be artifacts of evaluation protocol rather than the method.","rationale":"The paper does provide genuine controlled evidence that CNA improves its own backbones: Table 1 shows consistent gains on Cora across four architectures, Table 2 shows gains on graph classification, and Figure 4 shows deeper CNA networks retaining higher accuracy than vanilla or linearized GNNs. These results support the more modest claim that CNA is a useful plug-and-play module. However, the abstract's headline claim is about beating state-of-the-art methods, and that claim is anchored in Table 4. The absence of any stated split protocol is a decisive gap: the exceptionally high numbers on Cora and CiteSeer, combined with baselines that match the standard 20-label-per-class split, make it likely that the CNA results were produced under a different evaluation protocol. If so, the 8/11 win rate and the specific 94.18/95.75 figures in the abstract are not valid comparisons. I considered the reader's weakest assumption about non-differentiable and potentially unstable k-means cluster assignments. That is a real concern, but the controlled ablations and robustness experiments partially mitigate it; moreover, even a perfectly stable clustering would not save the central claim if the benchmark comparison is protocol-mismatched. Thus the most load-bearing issue is the uncontrolled leaderboard comparison, not internal training stability. The reader's rationale already mentions unstated splits and best-architecture selection, so the verdict of CONDITIONAL remains appropriate; the paper should be accepted only if the split protocol is clarified and the key Table 4 results are reproduced under the standard splits used by the cited leaderboards.","tokens_in":17040,"tokens_out":7263,"duration_ms":71325,"concrete_test":"Reproduce the Cora, CiteSeer, and PubMed rows of Table 4 using exactly the public/semi-supervised split used by the cited PwC leaderboard entries (for Cora and CiteSeer, 20 labels per class for training, with the fixed 500/1000 val/test masks), with the same architecture and hyperparameters from Appendix A.2. If reported accuracies fall below the PwC baselines (e.g., Cora below 90.16 or CiteSeer below 82.07), the SOTA claim fails; if they match, adding explicit split specifications to the paper resolves the concern.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that CNA 'significantly improves accuracy over the state-of-the-art' rests on Table 4. That table selects the best of five CNA architectures per dataset and compares it against a single PwC leaderboard entry. The paper nowhere states the train/val/test split used for the node-classification results; Appendix A.2 lists only epochs, layers, clusters, hidden units, learning rates, and weight decay. The reported baselines in Table 1 (GCN 81.59 on Cora) match the standard 20-label/class split, yet CNA is claimed to reach 93.66-94.18 on that same dataset and 95.75 on CiteSeer. A 12-14 point gain over the backbone on a 20-label-per-class transductive benchmark is far outside the range of published methods on the same split and strongly suggests the CNA numbers were obtained under a different split or label-per-class setting. Since no split is specified, the comparison is not apples-to-apples and the headline '8/11 SOTA wins' is not established. This is load-bearing because if Table 4's CNA entries were evaluated on a more favorable split, the 94.18 and 95.75 numbers would not be comparable to the PwC baselines, and the abstract's strongest claim would be unsupported. The controlled experiments in Tables 1, 2, and 6 still suggest CNA helps over its own backbones, so this is a protocol/evidence concern rather than an internal contradiction.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Cluster-Normalize-Activate (CNA), a plug-and-play module for message-passing GNNs that replaces the usual activation function with three steps: hard k-means clustering of node features, per-cluster normalization, and per-cluster learnable rational activations. The authors claim that CNA limits oversmoothing, improves accuracy over the state of the art on node classification and property prediction, reduces parameter counts, and benefits graph-level classification and node regression. The experimental sections report gains over backbone architectures (Tables 1, 2, 3), a leaderboard comparison (Table 4), an ogbn-arxiv parameter-efficiency study (Table 5), and an ablation (Table 6). A short theoretical section (Section 3.4) argues that existing oversmoothing proofs do not apply to CNA.","tokens_in":17379,"tokens_out":6147,"duration_ms":61597,"significance":"If the empirical claims are confirmed, CNA is a simple, architecture-agnostic component with broad potential: it gives consistent gains over its own backbones in controlled experiments, appears to permit much deeper GNNs without the usual performance collapse (Figure 4), and may yield parameter savings (Table 5). The paper ships code and provides a fairly wide evaluation across node classification, node regression, graph classification, and a large OGB dataset. However, the strongest claim in the abstract—state-of-the-art accuracy on Cora and CiteSeer—is not yet established because the evaluation protocol for the leaderboard comparison is incompletely specified, and the theoretical discussion contains a concrete error in its extremal-case analysis. The core ablation in Table 6 is a real strength: it shows all three components are needed for the effect on both Cora and ogbn-arxiv.","major_comments":[{"comment":"The node-classification experiments do not state the train/val/test split. Appendix A.2 (Table 9) lists epochs, layers, clusters, hidden units, learning rates, and weight decay, but no split and no label-rate protocol. Table 1 reports a GCN baseline of 81.59, which matches the standard 20-labels-per-class Cora split, while Table 4 reports 94.18 for the best CNA architecture on Cora. If the CNA numbers were obtained on a different split, the comparison to the PwC leaderboard entries is not apples-to-apples, and the abstract's headline 94.18% and 95.75% figures are unsupported. The authors should report the exact split and label rates for every dataset and confirm that the leaderboard baselines use the same protocol.","section":"Section 4, Table 4"},{"comment":"The 'Best CNA Result' column selects the best of five CNA-equipped architectures per dataset, while the PwC leaderboard entry is a single method. This is a best-of-five selection and introduces winner's bias in the claimed 8/11 win count. The paper should either report results for all CNA-equipped architectures, or fix a single architecture for the leaderboard comparison and report the selection rule explicitly.","section":"Section 4, Table 4"},{"comment":"The extremal-case argument contains a technical error. With K=N clusters, Eq. (1) normalizes each node over a singleton cluster: the mean equals the feature value and the variance is zero, so every normalized feature becomes zero (up to the numerical epsilon in the denominator). This does not 'render the normalization step ineffective' and does not recover the standard MPNN; it collapses all inputs to zero before the activation. The statement that K=1 corresponds to standard MPNN is also inaccurate, since K=1 is global normalization. Consequently, the section does not provide a sound argument that CNA escapes oversmoothing; the practical claim must rest on the empirical results, or the theoretical discussion should be revised.","section":"Section 3.4"}],"minor_comments":[{"comment":"'Oversquasching' is a typo for 'oversquashing'.","section":"Section 2"},{"comment":"GraphCON is cited to McCallum et al. [2000]; this reference is for the Cora dataset, not for GraphCON. The GraphCON citation should be corrected.","section":"Section 2"},{"comment":"Eq. (2) defines the rational activation with numerator degree m and denominator degree n, but Appendix A.2 states 'n=5 for the numerator and m=4 for the denominator'. The notation is inconsistent and should be fixed.","section":"Section 3.3 and Appendix A.2"},{"comment":"The NMSE values for GCN and GAT on Chameleon are identical (0.207) and nearly identical on Squirrel (0.143). This looks suspicious; please double-check the baseline results or clarify whether these are shared or rounded values.","section":"Section 4, Table 3"},{"comment":"The cluster assignment from k-means is non-differentiable and changes across layers. The paper does not state how gradients flow through the clustering step (e.g., whether the assignment is detached and treated as a constant in backpropagation). Please clarify this in the method description.","section":"Section 3.1"}],"recommendation":"major_revision","confidential_remarks":"The main issue is evaluation protocol: the missing split specification makes the leaderboard comparison in Table 4 unverifiable. This is fixable by reporting the exact protocol and, ideally, re-running the comparison under a common split. The error in Section 3.4 is also fixable by rewriting the extremal-case discussion. The controlled ablations (Tables 1, 2, 3, 6) are convincing enough that the method deserves a chance to be resubmitted with these corrections. I would not reject on the current evidence, because the central empirical mechanism is plausible and the paper ships code."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is simple and worth knowing: replace the activation in any MPNN layer with a cluster-then-normalize-then-activate module, using hard k-means clusters and per-cluster rational activations. The controlled experiments show it helps across architectures and tasks, and the ablation shows all three pieces matter. That part is credible.\n\nWhat is not credible yet is the SOTA claim. Table 4 compares the best CNA variant per dataset to a single PwC leaderboard entry, and the paper never states the train/val/test split. The Table 1 baseline GCN at 81.59 on Cora matches the standard 20-label-per-class split, but CNA reaching 93.66 and 94.18 is far outside the range of published methods on that split. Either the split is different (likely a larger labeled fraction) or the method is doing something remarkable that needs a much more careful controlled comparison. The ogbn-arxiv result (+4.5 points over GCN) is more plausible but still rests on a reproduced baseline with unspecified details.\n\nOther soft spots: the theory section does not actually prove oversmoothing is mitigated; it only shows the proof assumptions break and gives a toy extremal case. The non-differentiable cluster assignment is not analyzed, though in practice it seems stable enough. These are minor relative to the split problem.\n\nCredit where due: the paper ships code, uses standard benchmarks, reports seeds, and is honest in the limitations section about not scaling to huge graphs and not establishing a formal link. The depth experiment (Fig 4) and the ablation are useful.\n\nBottom line: this is a genuinely useful empirical module that deserves a serious referee, but the paper needs major revision of the evaluation protocol before the headline can be believed. If I were the editor, I would send it to review and ask the authors to specify splits, rerun Table 4 with matched splits, and tone down the SOTA language.","headline":"Useful plug-and-play module with credible controlled gains, but the SOTA claims are undermined by missing split details and leaderboard cherry-picking.","tokens_in":17906,"tokens_out":2163,"would_cite":true,"duration_ms":23311,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Cluster-Normalize-Activate modules stop node features from collapsing in deep graph networks, and the paper reports accuracy gains over state-of-the-art baselines on most benchmark tasks.","keywords":["graph neural networks","oversmoothing","cluster-normalize-activate","learnable activations","rational activations","node classification","message passing"],"falsifier":"Train a GCN with CNA on Cora while fixing the cluster assignment once at initialization instead of re-clustering at every layer. If fixed clusters match the full CNA accuracy, the per-layer clustering step is not what prevents oversmoothing; if accuracy collapses, the claim that fresh cluster assignments carry the benefit is confirmed.","tokens_in":16838,"feed_emoji":"🧩","tokens_out":7251,"duration_ms":60289,"temperature":0.7,"pith_summary":"Graph neural networks lose accuracy when made deep because node features converge to a common fixed point, a failure known as oversmoothing. This paper proposes Cluster-Normalize-Activate (CNA), a plug-and-play module that replaces the activation function in the update step of any message-passing GNN. CNA clusters node features with k-means, normalizes each cluster separately, then applies a distinct learned rational activation to each cluster, effectively creating per-cluster super nodes. The authors report that CNA improves node-classification accuracy on 8 of 11 benchmark datasets, reaching 94.18% on Cora and 95.75% on CiteSeer, lowers regression error, improves graph classification, and uses substantially fewer parameters than competing architectures. If the claims hold, CNA is a general and parameter-efficient remedy to oversmoothing that makes deeper GNNs practical.","feed_headline":"Cluster-normalize-activate beats state of the art on most graph tasks","feed_subtitle":"A plug-and-play module clusters nodes, normalizes each group, and learns separate activations to stop feature collapse.","key_machinery":"The central object is the CNA module itself, inserted after the aggregate-and-update computation of any message-passing GNN. Step 1, Cluster, applies hard k-means to the node feature vectors, yielding K clusters that the paper calls super nodes. Step 2, Normalize, standardizes features within each cluster using per-cluster, per-dimension means and variances, without an affine transform. Step 3, Activate, applies a separate learnable rational activation $R(x) = P(x)/Q(x)$ to each cluster, where rationals are smoothly differentiable universal approximators whose unbounded, non-Lipschitz form breaks the assumptions of existing oversmoothing proofs. The module's work is to give different groups of nodes different learned transformations, preserving discriminative information as layers deepen.","core_discovery":"The paper argues that oversmoothing can be countered by making the node-feature update adaptive per group of nodes rather than applying one fixed nonlinearity to all nodes. At each layer, node features are partitioned into K clusters by hard k-means; the features in each cluster are then normalized separately per dimension, and each cluster is passed through its own learnable rational activation function. The paper claims this Cluster-Normalize-Activate sequence keeps representations from collapsing to a single point, allows training networks with up to 96 layers without the usual accuracy drop, and outperforms state-of-the-art methods on a majority of node classification, graph classification, and node regression benchmarks while requiring fewer learnable parameters.","pith_inferences":["The paper does not analyze the stability of the k-means assignments across epochs; if clusters are noisy, part of the gain could come from normalization and rational activations alone, not from semantically meaningful super nodes.","A direct testable extension is to replace hard k-means with differentiable soft clustering; comparable accuracy would suggest the hard assignment is not load-bearing, while a drop would confirm that it is.","Because attention layers behave like message passing on complete graphs, CNA-style per-cluster normalization and activation may transfer to Transformers, where token representations similarly converge with depth.","Cluster quality itself, for example the silhouette score on learned features, could be measured and correlated with accuracy gains; the paper does not report whether better clusters yield better generalization."],"forward_implications":["CNA preserves accuracy in node classification at depths up to 96 layers, where ReLU networks collapse and even linearized GNNs degrade.","CNA is a drop-in activation replacement for GCN, GAT, GraphSAGE, TransformerConv, and Dir-GNN, so its benefit is not tied to one architecture.","Against the 11-dataset Papers with Code node-classification leaderboard, CNA achieves the best reported accuracy on 8 of 11 datasets, including 94.18% on Cora and 95.75% on CiteSeer.","CNA also reduces normalized mean squared error on Chameleon and Squirrel node regression and improves graph-level classification on Mutag, Enzymes, and Proteins.","Models with CNA reach higher or comparable accuracy with far fewer learnable parameters than competing deep GNNs, such as 74.64% on ogbn-arxiv with 389.2k parameters."],"supporting_citations":[{"why":"Provides Gradient Gating, the oversmoothing mitigation baseline that motivates CNA and that CNA outperforms in comparisons.","marker":"[Rusch et al., 2023b]"},{"why":"Supplies the rational activation functions used in the Activate step and their universal-approximation property.","marker":"[Molina et al., 2019]"},{"why":"Differentiable Group Normalization is the soft-clustering normalization baseline that CNA contrasts with and outperforms in Table 11 and the ablation.","marker":"[Zhou et al., 2020b]"},{"why":"Introduces the k-means algorithm used for the hard clustering step.","marker":"[MacQueen, 1967]"},{"why":"Establishes the oversmoothing and Dirichlet energy results whose assumptions CNA breaks in Section 3.4.","marker":"[Rusch et al., 2022]"},{"why":"Defines the GCN architecture used across the depth, ablation, and ogbn-arxiv experiments.","marker":"[Kipf and Welling, 2016]"}],"fun_headline_variants":["CNA module stops GNN oversmoothing, enabling deep nets with fewer params","Cluster-Normalize-Activate: a plug-in that stops oversmoothing in GNNs","CNA lets GNNs go deep: 96 layers without oversmoothing","Adaptive activations per cluster stop feature collapse in GNNs","Deep GNNs saved by per-cluster normalization and activation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's gains rest on the assumption that hard k-means cluster assignments at each layer are stable enough that normalizing and activating each cluster separately produces useful, trainable gradients; the paper does not analyze how the non-differentiable, changing assignment affects training.","fun_headline_variants_meta":{"raw":{"variants":["CNA module stops GNN oversmoothing, enabling deep nets with fewer params","Cluster-Normalize-Activate: a plug-in that stops oversmoothing in GNNs","CNA lets GNNs go deep: 96 layers without oversmoothing","Adaptive activations per cluster stop feature collapse in GNNs","Deep GNNs saved by per-cluster normalization and activation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000825,"raw_usage":{"total_tokens":3562,"prompt_tokens":856,"completion_tokens":2706,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":472,"completion_tokens_details":{"reasoning_tokens":2603}},"tokens_in":472,"tokens_out":2706,"duration_ms":17144,"temperature":1.0,"reasoning_tokens":2603,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T21:47:52.807797+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a GCN with CNA on Cora while fixing the cluster assignment once at initialization instead of re-clustering at every layer. If fixed clusters match the full CNA accuracy, the per-layer clustering step is not what prevents oversmoothing; if accuracy collapses, the claim that fresh cluster assignments carry the benefit is confirmed.","supporting_citations":[{"cited_title":"Padé Activation Units : End -to-end Learning of Flexible Activation Functions in Deep Networks","cited_arxiv_id":null,"evidence_quote":"Supplies the rational activation functions used in the Activate step and their universal-approximation property."},{"cited_title":"Graph- Coupled Oscillator Networks","cited_arxiv_id":null,"evidence_quote":"Establishes the oversmoothing and Dirichlet energy results whose assumptions CNA breaks in Section 3.4."}],"review_version":1}