{"id":"07505740-4b93-437c-bbe7-8d52ecb4a52d","arxiv_id":"2506.17576","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A training strategy that grows GCN depth gradually, using LoRA fine-tuning and identity initialization, is claimed to reduce over-smoothing and improve accuracy of deep vanilla GCNs.","lead":"This paper claims that the trainable linear transformations inside standard graph neural networks are a major cause of 'over-smoothing', where deep networks lose the ability to tell nodes apart. It proposes a staged training recipe, adding layers one at a time with identity initialization and low-rank fine-tuning, and reports that it lets a plain 32-layer GCN outperform specialized anti-over-smoothing models on small citation datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Fig. 1 GCN-vs-SGC comparison does not isolate trainable linear transformations from nonlinearities or stacked propagation, so the causal claim that linear transformations drive over-smoothing is not established.","rationale":"The reader's weakest assumption correctly identifies the SGC comparison as insufficient to isolate linear transformations. My independent read agrees and sharpens it: SGC also removes nonlinearities and collapses the propagation, so Figure 1 cannot support the causal statement in the abstract. The proposed LGT may still be a useful training recipe; the ablation shows gains over vanilla GCN, and identity initialization is a sensible way to stack layers. However, the paper's central scientific contribution—the overlooked role of trainable linear transformations—rests on this confounded comparison. The controlled experiment above would settle it. Missing strong baselines (GCNII, JK-Net, DropEdge) weaken the SOTA label but are secondary to the mechanism claim; the anomalous training-time entry in Table 3 (AmazonPhoto GCN+LGT 16-layer 1452s vs 32-layer 1194s) affects only the efficiency claim. Because no new objection beyond the reader's weakest assumption changes the assessment, the verdict stays conditional pending the proposed check.","tokens_in":11475,"tokens_out":7957,"duration_ms":84960,"concrete_test":"Run a controlled ablation on Cora with four variants: (a) standard GCN with ReLU and trainable W per layer; (b) linear GCN, H^{k+1}=L_hat H^k W^k, no nonlinearity, trainable W; (c) SGC, H=softmax(L_hat^K X W); (d) GCN with identity W and ReLU. Track accuracy and Dirichlet energy at depths 4, 8, 16, and 32. If variant (b) collapses while (c) stays stable, the linear-transformation hypothesis is supported; if (b) is stable, nonlinearities or propagation depth are the driver instead.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central causal claim—that trainable linear transformations, not just the Laplacian, are a primary driver of over-smoothing—rests entirely on Figure 1, which compares standard GCN with SGC. SGC differs from GCN in three simultaneous ways: it removes per-layer nonlinearities, collapses the stacked propagation into a single matrix power, and reduces trainable weights to a single final classifier. Therefore the contrast cannot attribute GCN's collapse to linear transformations; any of the three differences could be responsible. The statement in Section 4 that 'SGC removes these transformations' is imprecise: SGC retains a linear classifier and a fixed propagation matrix, and it removes nonlinearities together with per-layer weights. Consequently, the identity-initialization and layer-wise-freezing recipe may be helping by avoiding end-to-end optimization difficulty or gradient interference rather than by counteracting learned weights. The paper's own conclusion admits the lack of theoretical analysis, which makes the confounded empirical diagnosis the only support for the mechanism. If the diagnosis is wrong, the method might still improve accuracy, but the paper's central scientific contribution would not hold.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that over-smoothing in deep GCNs is driven not only by repeated Laplacian propagation but also by trainable linear transformations, and it proposes Layer-wise Gradual Training (LGT), which combines incremental layer-wise training, LoRA-based fine-tuning of frozen layers, and identity initialization of newly added layers. LGT is evaluated on Cora, Citeseer, Pubmed, and AmazonPhoto for depths 4, 8, 16, and 32, and is also combined with PairNorm and ContraNorm. The authors report large accuracy gains over plain GCNs, improved training efficiency, and state-of-the-art performance for deep vanilla GCNs.","tokens_in":11532,"tokens_out":5120,"duration_ms":54605,"significance":"If the causal claim were established, the paper would make a useful contribution by reframing over-smoothing as partly an optimization problem and by offering a training-side remedy that is compatible with normalization-based methods. The LGT recipe is simple, the code is public, and the ablation in Figure 4 gives some support for the contribution of each component. However, the central diagnostic claim rests on a confounded comparison, the state-of-the-art claim is not supported by the included baselines, and one reported training-time result contradicts the efficiency claim. The empirical improvement of deep GCNs with LGT is plausible and interesting, but the paper's scientific and comparative claims need substantial revision.","major_comments":[{"comment":"The central diagnostic claim—that trainable linear transformations, rather than the graph Laplacian, are a primary driver of over-smoothing—is not established by the evidence presented. Figure 1 compares GCN with SGC, but SGC differs from GCN in three simultaneous ways: it removes per-layer nonlinearities, collapses the stacked propagation into a single matrix power, and reduces trainable weights to a final classifier. Any of these differences could explain SGC's stability at 32 layers, so the comparison cannot isolate the effect of trainable linear transformations. Section 4 repeats this attribution when motivating LGT, and the paper's own conclusion concedes that no theoretical analysis is provided. To support the claim, the authors should add controlled experiments that vary only the trainability of the linear weights at fixed depth and nonlinearity (e.g., a deep GCN with frozen or identity-initialized weights versus trained weights), and report a quantitative feature-collapse measure such as Dirichlet energy rather than relying on accuracy and t-SNE plots alone.","section":"Section 1, Figure 1, Section 4"},{"comment":"The claim that LGT achieves 'state-of-the-art performance on vanilla GCN' is substantially overstated. Table 2 compares LGT only against SGC, IresGCN, PSNR, PairNorm, ContraNorm, and vanilla GCN; standard deep-GCN baselines such as GCNII, JK-Net, and DropEdge—which are discussed in Section 2.1—are missing. Moreover, GCN+LGT does not consistently outperform the included baselines: on AmazonPhoto at 32 layers, IresGCN reaches 91.68% versus 91.34% for GCN+LGT, and on several settings PairNorm+LGT is worse than PairNorm itself (e.g., Citeseer at 16 layers: 65.12% versus 66.66%). The 'state-of-the-art' wording should be replaced by a more precise claim, and the missing baselines should be added or the comparison scope stated explicitly.","section":"Section 5.1, Table 2"},{"comment":"Table 3 contradicts the efficiency claim that LGT 'significantly reduces training time.' On AmazonPhoto at 16 layers, GCN+LGT requires 1452 seconds versus 228 seconds for vanilla GCN, an increase of more than sixfold, and at 8 layers GCN+LGT (41s) is only marginally faster than GCN (72s) despite the qualitative claim in Figure 5. The authors should either correct the table, explain the anomaly (e.g., different hardware or LoRA rank settings), or restrict the efficiency claim to the settings where it actually holds; as written, Section 5.4's central assertion is not supported by its own data.","section":"Section 5.4, Table 3"}],"minor_comments":[{"comment":"In the rank analysis paragraph, 'shadow networks' should read 'shallow networks.'","section":"Section 5.5"},{"comment":"Reference [10] contains a typo: 'International onference' should be 'International Conference.'","section":"References"},{"comment":"The experimental setup does not report key hyperparameters such as the number of stages, epochs per stage, early stopping patience, learning rate, or LoRA rank per dataset; the code link alone is insufficient for reproducibility.","section":"Section 5.1"},{"comment":"The caption and text describe SGC as maintaining 'stable performance,' but SGC accuracy on Cora drops from 79.88% at 4 layers to 69.62% at 32 layers, so 'stable' should be qualified.","section":"Figure 1 and Section 5.2"},{"comment":"The abstract's statement that LGT 'further enhances' PairNorm and ContraNorm is too broad; Table 2 shows several settings where PairNorm+LGT or ContraNorm+LGT is worse than the base method, so the compatibility claim should be stated as improving performance in most but not all settings.","section":"Abstract, Table 2"}],"recommendation":"major_revision","confidential_remarks":"The main risk is the confounded diagnosis in Figure 1: the GCN-versus-SGC comparison cannot support the causal claim about linear transformations. I would ask the authors to either provide a controlled ablation or explicitly demote the mechanism claim to a hypothesis. The missing GCNII/JK-Net/DropEdge baselines and the Table 3 anomaly should also be resolved before the paper is suitable for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague—\n\nThe thing to know about this paper: the LGT training recipe is real and worth trying. Incremental layer-wise training plus LoRA fine-tuning plus identity initialization makes 16–32 layer vanilla GCNs get accuracy in the high 70s to low 80s on these benchmarks, which is a big jump over standard GCN. The ablation shows all three pieces contribute, and the method is architecture-agnostic and plays well with PairNorm/ContraNorm. That is a genuine engineering contribution.\n\nWhat is not solid is the paper's explanation of why it works. The central claim that trainable linear transformations are a primary driver of over-smoothing rests on Figure 1, which compares GCN with SGC. SGC differs from GCN in three ways at once: no nonlinearities, no per-layer weights, and the propagation collapsed into a single matrix power. The comparison cannot attribute the observed collapse to linear transformations. The stress-test note gets this exactly right. The paper's own conclusion admits the lack of theory, so the confounded figure is the only support for the mechanism.\n\nOther soft spots: the 'state-of-the-art' claim is overstated. There are no GCNII, JK-Net, or DropEdge comparisons, and only four small datasets with five splits. Experimental details like stage schedules and LoRA rank per stage are missing. And Table 3 contradicts the 'reduces training time' claim: on AmazonPhoto at 16 layers, GCN+LGT takes 1452 seconds versus 228 for GCN. That is an anomaly the authors should explain.\n\nTo be fair, the method is clearly described, the code link is provided, and the empirical gains over vanilla GCN are consistent across all four datasets. The training recipe is plausible even if the mechanism is not established. The paper would benefit from a proper diagnostic experiment—e.g., ablating nonlinearities separately from per-layer weights—and a stronger baseline set.\n\nI would not cite this in my own work right now, but I would bring it to a reading group as an example of a training-centric approach to over-smoothing. If this crossed my desk, I would send it to peer review: the recipe deserves scrutiny and the baseline comparison can be fixed in revision. I would not desk-reject it, but I would expect major revision before acceptance.","headline":"LGT is a genuinely useful training recipe for deep GCNs, but the paper's causal story about linear transformations driving over-smoothing is not supported by the SGC comparison.","tokens_in":12220,"tokens_out":2695,"would_cite":false,"duration_ms":27090,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Trainable linear transformations in GCNs drive over-smoothing, and a gradual layer-wise training schedule with identity initialization and low-rank fine-tuning keeps plain GCNs accurate at 32 layers.","keywords":["Graph Convolutional Networks","over-smoothing","feature collapse","layer-wise training","low-rank adaptation","identity initialization","deep graph neural networks","simplified graph convolution"],"falsifier":"Train an SGC-style network that keeps per-layer nonlinear activations but fixes all linear weights at the identity (SGC plus ReLU). If its accuracy and feature diversity collapse by 8 layers, the collapse is not caused by trainable linear transformations.","tokens_in":11107,"feed_emoji":"🕸️","tokens_out":7096,"duration_ms":66040,"temperature":0.7,"pith_summary":"The paper tries to establish that the main engine of over-smoothing in deep GCNs is not the graph propagation itself but the trainable linear transformations inside each layer, and that a training schedule rather than a new architecture can solve it. It shows that SGC, which removes learned weights, keeps separable features up to 32 layers, while vanilla GCN collapses by 8 layers. The proposed Layer-wise Gradual Training (LGT) adds layers one at a time, initializes each new layer as the identity, and fine-tunes earlier layers with low-rank updates, allowing a vanilla GCN to reach 32 layers with accuracy near that of specialized anti-over-smoothing methods. If correct, this shifts the over-smoothing discussion from architectural design to optimization and offers a general recipe that can be combined with existing methods.","feed_headline":"Training trick lets plain GCNs stay accurate at 32 layers","feed_subtitle":"One layer at a time, with identity starts and low-rank fine-tuning, keeps deep GCN features distinct.","key_machinery":"The central object is Layer-wise Gradual Training (LGT), a training schedule with three components: incremental layer-wise training adds one layer at a time and freezes earlier ones; low-rank adaptation writes each frozen weight as $W_0 + AB$ with a small rank $r$ so shallow layers stay lightly trainable; identity initialization starts every new layer at the identity matrix so it passes representations through unchanged at first. Together these isolate newly added parameters from the already stabilized network, so depth grows without a sudden random perturbation that collapses features.","core_discovery":"The paper's central claim is that trainable linear transformations in GCN layers, rather than repeated Laplacian propagation alone, are a primary accelerant of over-smoothing: on Cora, standard GCN accuracy drops to about 40% at 32 layers while SGC, which removes learned weights, stays near 70%. To keep expressiveness while avoiding collapse, the paper proposes LGT and reports that GCN+LGT reaches around 81% on Cora at 32 layers and similarly strong results on Citeseer, Pubmed, and AmazonPhoto, outperforming vanilla GCN, SGC, and several anti-over-smoothing baselines. LGT also improves PairNorm and ContraNorm in most tested settings and cuts training time by avoiding full-model retraining.","pith_inferences":["Inference: if the diagnosis is right, the same staged schedule should transfer to attention-based or message-passing architectures with learned per-layer maps, such as GAT, without architectural changes.","Inference: the paper's account predicts a new test: monitoring feature diversity (for example Dirichlet energy) across LGT stages would show diversity preserved during the schedule, not just at the final layer; this is not reported.","Inference: a cheaper variant might drop LoRA and rely on identity initialization alone for some datasets; the paper's ablations show LoRA helps at depth, but the interaction is data-dependent.","Inference: the role of the learned weight matrices could be probed by freezing all layers at identity after their initial stage, which would separate the benefit of the schedule from the benefit of continued light adaptation."],"forward_implications":["A vanilla GCN can be trained to 32 layers with accuracy comparable to specialized anti-over-smoothing architectures, so depth itself need not force an architecture change.","Normalization-based methods such as PairNorm and ContraNorm inherit the training-efficiency and accuracy gains, so LGT can be stacked on top of existing fixes.","Because LGT only changes the training schedule, it applies to any GCN-like model without modifying the forward pass or the graph structure.","Deep GCN training becomes cheaper: since only the newest layer is fully trained and older layers are lightly adapted, reported training time drops substantially at 8, 16, and 32 layers."],"supporting_citations":[{"why":"Supplies the SGC comparison showing stable features up to 32 layers, and the idea of identity-like fixed transformations that motivates identity initialization.","marker":"[28]"},{"why":"Defines the vanilla GCN architecture whose deep training is the target, and the semi-supervised node classification setup.","marker":"[11]"},{"why":"Provides the low-rank adaptation (LoRA) technique used to fine-tune frozen shallow layers.","marker":"[10]"},{"why":"Represents the prior Laplacian-based attribution of over-smoothing that the paper's diagnosis challenges.","marker":"[14]"},{"why":"Supplies the theoretical result that deep GNNs exponentially lose expressive power, framing the over-smoothing problem.","marker":"[19]"},{"why":"Baseline normalization method that LGT is shown to complement; used for compatibility experiments.","marker":"[30]"},{"why":"Baseline normalization method that LGT is shown to complement; used for compatibility experiments.","marker":"[9]"}],"fun_headline_variants":["Over-smoothing culprit: GCN's learnable weights, not just Laplacian","LGT: train GCNs layer by layer to beat 32-layer collapse","Fine-tuning with identity starts keeps deep GCN features alive","New training strategy lets GCNs go deep without over-smoothing","Layer-wise training and low-rank tweaks solve GCN depth limit"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The diagnosis rests on the SGC comparison, which removes linear transformations together with nonlinearities and layer collapse, so the stability of SGC at 32 layers does not isolate the effect of the learned weights.","fun_headline_variants_meta":{"raw":{"variants":["Over-smoothing culprit: GCN's learnable weights, not just Laplacian","LGT: train GCNs layer by layer to beat 32-layer collapse","Fine-tuning with identity starts keeps deep GCN features alive","New training strategy lets GCNs go deep without over-smoothing","Layer-wise training and low-rank tweaks solve GCN depth limit"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00021,"raw_usage":{"total_tokens":1441,"prompt_tokens":1003,"completion_tokens":438,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":619,"completion_tokens_details":{"reasoning_tokens":340}},"tokens_in":619,"tokens_out":438,"duration_ms":4350,"temperature":1.0,"reasoning_tokens":340,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:30:24.232658+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train an SGC-style network that keeps per-layer nonlinear activations but fixes all linear weights at the identity (SGC plus ReLU). If its accuracy and feature diversity collapse by 8 layers, the collapse is not caused by trainable linear transformations.","supporting_citations":[{"cited_title":"The Snowflake Hypothesis: Training Deep GNN with One Node One Receptive field","cited_arxiv_id":"2308.10051","evidence_quote":"Supplies the SGC comparison showing stable features up to 32 layers, and the idea of identity-like fixed transformations that motivates identity initialization."},{"cited_title":"In: International onference on Learning Representations (2022)","cited_arxiv_id":null,"evidence_quote":"Provides the low-rank adaptation (LoRA) technique used to fine-tune frozen shallow layers."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Represents the prior Laplacian-based attribution of over-smoothing that the paper's diagnosis challenges."},{"cited_title":"In: International Conference on Learning Representations (2020)","cited_arxiv_id":null,"evidence_quote":"Supplies the theoretical result that deep GNNs exponentially lose expressive power, framing the over-smoothing problem."},{"cited_title":"In: Dy, J., Krause, A","cited_arxiv_id":null,"evidence_quote":"Baseline normalization method that LGT is shown to complement; used for compatibility experiments."},{"cited_title":"In: The Eleventh International Conference on Learning Representations (2023)","cited_arxiv_id":null,"evidence_quote":"Baseline normalization method that LGT is shown to complement; used for compatibility experiments."}],"review_version":1}