{"id":"b3cf1eef-39ad-435b-829c-03d6582ec101","arxiv_id":"2506.16790","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"SPoGInit stabilizes forward, backward, and embedding-variation signal propagation in deep graph convolutional networks, mitigating the performance degradation that normally comes with depth.","lead":"The paper defines three signal-propagation metrics for deep graph networks and searches for initialization weight scales that keep all three stable, producing an initialization method called SPoGInit. In experiments, SPoGInit reduces or reverses accuracy drops as GCNs get deeper, especially on tasks that need long-range information.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SPoGInit's label-dependent backward-signal term, absent from all baselines, leaves the causal claim that stabilized signal propagation drives the accuracy gains untested.","rationale":"The reader's verdict is already CONDITIONAL and explicitly flags both the infinite-width faithfulness assumption and the label-dependence asymmetry. I focus on the label-dependence issue because it is the one most directly load-bearing for the paper's central causal claim: that stabilizing the three metrics at initialization is what enables deep-GCN improvements. The infinite-width concern is real but less decisive, since SPoGInit's search is actually run on the finite-width network and the empirical metric-stabilization plots are direct, not NNGP-derived. By contrast, the BSP term's use of true training labels creates an uncontrolled advantage over every baseline and an untested alternative explanation for the accuracy gains. The proposed random-label experiment is a clean, low-cost control that would settle whether the concern lands. Because the reader already conditioned the verdict on this family of issues, and because the concern does not by itself show the method is unsound, I keep the verdict at CONDITIONAL rather than moving it.","tokens_in":57213,"tokens_out":2977,"duration_ms":38821,"concrete_test":"Rerun the SPoGInit search on the 32-layer vanilla GCN/Cora setup from Table 20 with the BSP term computed from randomly permuted training labels (or from a fixed label-agnostic surrogate such as a constant one-hot target), while keeping FSP and GEV terms, the search procedure, and all hyperparameters identical. Then train with the true labels as in the paper. If the random-label BSP version matches the true-label SPoGInit accuracy (about 75.8) and still beats Xavier, label dependence is not the driver; if it falls back toward the single-metric or two-metric ablation results, the reported gains are at least partly attributable to label information rather than to backward signal-propagation stability.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that stabilizing the three SP metrics is what improves deep GCN performance. The BSP component of the SPoGInit objective, however, is computed from the gradient of the training loss on the true training labels (Algorithm 1, line 5; Appendix E, Eq. 49), making the searched initialization label-dependent. Every baseline initialization is label-agnostic. This asymmetry is never disclosed or controlled, and it threatens the causal interpretation: observed gains could come from variance scales that exploit label information at initialization rather than from the SP stabilization the paper hypothesizes. The FSP and GEV terms are label-agnostic and the ablation in Appendix G.2 shows all three terms matter, but no experiment isolates whether the BSP term's contribution is due to gradient-magnitude stabilization or due to label information. The manuscript's stated limitations (Section 7) do not mention this asymmetry. This is not an accusation of leakage in the usual sense, since training labels are legitimately available at initialization; it is a comparison-integrity and interpretability problem. If a label-agnostic version of BSP performs equally well, the concern is resolved; if it does not, the paper's main conclusion about the SP framework is not supported by the current experiments.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper argues that depth-related performance degradation in graph convolutional networks can be understood as a signal-propagation failure involving three metrics: forward signal propagation (FSP), backward signal propagation (BSP), and graph embedding variation (GEV), the last being a graph-specific normalized Dirichlet energy. Under an NNGP infinite-width approximation, it proves that for ReLU vanilla GCNs and linear ResGCNs standard variance choices drive FSP and/or GEV to degenerate limits, and it proposes SPoGInit, an initialization search that minimizes a weighted combination of the three metrics by rescaling per-layer weight variances. Experiments on Cora, PubMed, OGBN-Arxiv, Arxiv-year, additional homophily/heterophily datasets, missing-feature settings, and MILP show that SPoGInit stabilizes the SP metrics and reduces accuracy degradation in deep GCN, ResGCN, MixHop, and GAT models compared with Xavier, Conventional, VirgoFor, VirgoBack, and G-Init initializations.","tokens_in":57402,"tokens_out":8558,"duration_ms":87745,"significance":"If the causal claim were established, the paper would make a useful contribution: it offers a compact diagnostic (FSP/BSP/GEV), gives careful NNGP derivations for the forward and GEV recurrences, and shows that depth degradation can be attacked at initialization, a route that is orthogonal to normalization, dropout, and architectural rewiring. The empirical scope is broad, the SP plots in Figures 3-5 directly illustrate the proposed diagnostic, and the integration experiments with DGN, CO-GNN, and G-Init strengthen the practical claims. The main weakness is that the paper does not yet isolate the mechanism: SPoGInit's backward term is label-dependent while all baselines are label-agnostic, and the theoretical results cover only a subset of the stated claims. These gaps are fixable with targeted controls and rewording.","major_comments":[{"comment":"The BSP term in SPoGInit's objective is computed from the gradient of the training loss on the true training labels (Algorithm 1, line 5; Eq. (49), with g^(l)=∂ℓ/∂W^(l)), so the searched initialization is label-dependent. Every baseline initialization (Conventional, Xavier, VirgoFor, VirgoBack, G-Init) is label-agnostic, and Appendix G.2's ablation does not control for this asymmetry because removing the BSP term removes both gradient stabilization and label information. The experiments therefore do not isolate whether SPoGInit's gains come from stabilizing backward signal propagation or from exploiting label information at initialization; this is a comparison-integrity and interpretability problem rather than leakage, since labels are legitimately available at initialization. A decisive control is to run the same search with random labels (or a label-agnostic BSP proxy) and compare accuracy; if the gains persist, the causal interpretation is supported, and if not, the central claim needs substantial qualification.","section":"Section 4, Algorithm 1 (Appendix E), Eq. (49)"},{"comment":"The abstract and Section 3 state that standard initializations are theoretically proven to fail to control all three SP metrics simultaneously. However, Theorem 3.1 (vanilla GCN) and Theorem 3.3 (ResGCN) establish results only for FSP and GEV; no theorem anywhere characterizes BSP, and the sentence following Theorem 3.3 asserts a shrinking BSP that is not a stated or proved consequence of the theorem. In addition, Theorem 3.1 is proved for ReLU while the main vanilla GCN experiments use tanh (Section 5.1), and Theorem 3.3 is proved for linear ResGCN while the ResGCN experiments use ReLU. The theoretical support for the headline claim is therefore narrower than presented; the authors should either supply the missing BSP and nonlinearity results or reword the abstract and Section 3 to match what is proved.","section":"Section 3 and Abstract"},{"comment":"All experiments use width 64, while Propositions C.1 and D.1 are sequential infinite-width limits. The only direct validation is the Kolmogorov-Smirnov test in the Infinite-width-limit regime paragraph of Section 6, applied to three node embeddings from the final layer of a 4-layer network; this does not establish that the NNGP recursions are faithful at L=64 and width 64 for all three metrics. Since the negative theorems are stated in the NNGP limit, the transfer of those conclusions to the finite-width models in Figures 3-5 and Tables 2-5 needs additional support, for example a width-ablation showing that the SP metric curves or the searched variances stabilize as width grows. I treat this as a correctness-risk concern rather than an observed contradiction.","section":"Section 6, Infinite-width-limit regime, and Section 5.1"}],"minor_comments":[{"comment":"The text introducing the w2 analysis says Table 7 reports the values, but the displayed table is numbered Table 6; fix this cross-reference.","section":"Appendix E.2"},{"comment":"On OGBN-Arxiv, SPoGInit's vanilla GCN degradation from 4 to 64 layers is 21.4 points, substantially worse than VirgoFor (8.5) and VirgoBack (8.8); the text in Section 5.2 says SPoGInit significantly reduces performance degradation and should be qualified to most settings, with a discussion of this exception.","section":"Table 2"},{"comment":"The paper reports that all results are averaged over at least three runs, but Tables 4, 5, and 9-11 report single numbers without standard deviations; for differences of one to two accuracy points this is insufficient evidence, and the authors should report error bars or significance tests.","section":"Section 5.1"},{"comment":"Please state whether the code for SPoGInit will be released; the search algorithm is intricate, and reproducibility would benefit from code or detailed pseudo-code in addition to the current description.","section":"Appendix G.1"}],"recommendation":"major_revision","confidential_remarks":"Dear Editor, this is a solid empirical paper with careful NNGP analysis, but the label-dependent BSP term is a genuine comparison-integrity issue for the causal claim. I recommend major revision even though the method itself is promising; the authors should add the label-agnostic BSP control and/or reframe the claims. The theoretical overstatements about which metrics are proven can be handled by rewording."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. First, the GEV metric and the three-metric (FSP/BSP/GEV) decomposition are a genuine step forward: over-smoothing is treated as a signal-propagation problem with its own metric, and Theorem 3.2 delivers a clean negative result (ReLU GCN GEV is independent of variance). The NNGP derivations are careful, and the SP plots in Figures 3–5 convincingly show SPoGInit stabilizing all three metrics where VirgoFor/VirgoBack do not. Second, the empirical gains on deep GCNs look real across architectures and tasks, but the paper's central causal story—stabilizing SP drives the accuracy gains—is not actually tested. The BSP term in the SPoGInit objective is computed from the gradient of the training loss on the true training labels (Algorithm 1, Appendix E), making the searched initialization label-dependent. Every baseline is label-agnostic. This asymmetry is never flagged in Section 7, and it is a comparison-integrity and interpretability problem. A label-agnostic BSP variant (e.g., random labels or a squared-activation proxy) is needed to resolve it; without that, the SP framework's causal role is unproven.\n\nWhat the paper does well: it ships formal NNGP theorems (FSP decay, GEV independence), a clean ablation showing all three metrics matter, and a broad empirical study including missing-feature and MILP tasks. The method is simple and architecture-agnostic. That is real content.\n\nSoft spots, in order of seriousness. (1) The label-dependent BSP term, as above. (2) The abstract overclaims: 'theoretically demonstrate ... fail to control these three metrics simultaneously' is only proven for FSP and GEV; BSP is empirical. Also 'enables performance improvements as GNNs deepen' is too strong—Table 2 shows SPoGInit still degrades on some datasets (e.g., GCN on Arxiv, 21-point drop), so 'reduces depth degradation' would be more accurate. (3) Main tables lack error bars, though the appendix has some. (4) No code release, limiting verification of the finite-width NNGP claims.\n\nNone of this is fatal. The GEV concept and the negative result are novel, and the method is practical and likely to be used. But the causal claim needs the label-agnostic control, and the claims need tightening. This paper deserves a serious referee—I would send it out, with a request for that control experiment and full disclosure of the label dependence.","headline":"A useful three-metric signal-propagation framework and a practical initialization search, but the central causal claim is confounded by a label-dependent backward term that the paper never discloses.","tokens_in":58018,"tokens_out":3207,"would_cite":true,"duration_ms":35388,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Searching for layer-wise weight variances that keep forward, backward, and graph-embedding signals stable yields an initialization, SPoGInit, that turns depth in GCNs from a liability into an asset.","keywords":["graph neural networks","initialization","signal propagation","over-smoothing","graph embedding variation","deep GCN","NNGP correspondence","variance search"],"falsifier":"Train a 64-layer GCN with SPoGInit's searched variance profile but at width 16 and width 512 on the same datasets: if the accuracy gains over baselines shrink or vanish at either width, or if the three evaluated metrics no longer track training stability, the infinite-width premise is not faithful. A second decisive check is to run SPoGInit without the label-dependent backward term: if the accuracy gains disappear, the source of the improvement is gradient geometry informed by training labels, not generic signal stability.","tokens_in":56963,"feed_emoji":"📈","tokens_out":5989,"duration_ms":60595,"temperature":0.7,"pith_summary":"This paper claims that performance degradation in deep graph convolutional networks is best understood as a joint failure of three signal-propagation mechanisms, and that fixing all three at initialization is enough to let depth help rather than hurt. It defines three metrics: forward signal propagation, backward signal propagation, and graph embedding variation, a normalized Dirichlet-energy quantity tied to over-smoothing. It proves that standard initializations, including Conventional, Kaiming, LeCun, and Xavier, cannot keep all three metrics stable simultaneously in vanilla GCNs or residual GCNs. The proposed remedy, SPoGInit, searches per-layer weight variances to stabilize the three metrics, and the paper reports that deep GCNs across several architectures improve with depth or degrade far less than baselines. A sympathetic reader would care because this makes initialization alone, without changing the architecture or message-passing mechanism, a viable and transferable fix for the depth problem.","feed_headline":"SPoGInit turns GCN depth into accuracy gains","feed_subtitle":"Layer-wise variance search keeps forward, backward, and graph-embedding signals stable, so deep ResGCNs beat their shallow versions.","key_machinery":"The machinery is a three-metric signal-propagation analysis paired with a variance-search algorithm. The forward metric $M^{(L)}_{\\mathrm{FSP}}$ is the expected output-input Frobenius norm ratio, the backward metric $M^{(L)}_{\\mathrm{BSP}}$ is the expected squared gradient norm of the first-layer weights at initialization, and the graph embedding variation metric $M^{(L)}_{\\mathrm{GEV}}$ is the expected normalized Dirichlet energy $\\mathbb{E}[\\mathrm{Dir}(H^{(L)})/\\|H^{(L)}\\|_F^2]$. SPoGInit solves $\\min_{\\{\\sigma_{w,l}\\}} w_1 V_{\\mathrm{FSP}} + w_2 V_{\\mathrm{BSP}} - w_3 M^{(L)}_{\\mathrm{GEV}}$ using projected gradient descent on layer-wise variance scaling factors, with the NNGP covariance recursion $\\Sigma^{(l+1)} = \\sigma_w^2 \\hat{A} G(\\Sigma^{(l)}) \\hat{A}$ as the theoretical lens. The graph embedding variation term is what connects initialization directly to over-smoothing, and the search is what lets the same framework adapt to vanilla GCNs, residual GCNs, MixHop, GAT, and bipartite GCNs.","core_discovery":"The central claim is that over-smoothing and gradient pathology in deep GCNs are two faces of one signal-propagation problem, and that a variance search over three metrics can resolve both. Under the infinite-width neural-network Gaussian process approximation, the paper shows that ReLU-activated vanilla GCNs driven by standard initializations either lose forward signal or lose graph embedding variation, and that the variation metric is actually independent of weight variance for ReLU, so no scalar rescaling can fix over-smoothing there. For residual GCNs, it shows that any nonzero initialization variance that satisfies the natural residual condition makes forward signal explode while graph embedding variation vanishes. The paper then proposes SPoGInit, which minimizes a weighted objective combining forward stability, backward stability, and graph embedding variation by adjusting layer-wise initialization variances, and demonstrates that the resulting initialization stabilizes all three metrics and yields accuracy gains as depth grows, including on long-range tasks and combinatorial optimization problems.","pith_inferences":["A label-agnostic variant of SPoGInit that drops the backward signal term, or computes it from random labels, would isolate whether the gains come from label-dependent gradient geometry or from generic signal stability; the paper's BSP term uses training labels, which makes the searched initialization label-dependent while all baselines are label-agnostic.","If the mechanism is right, the same three-metric objective could serve as a training-free diagnostic for whether a proposed GCN architecture will tolerate depth, predicting the depth at which degradation begins before any training is run.","The variance-search idea may transfer to other message-passing layers and graph transformers, since the metrics only require forward and backward passes plus a normalized Dirichlet energy computation over the final embeddings.","Because SPoGInit uses training labels at initialization, it is effectively a hybrid between an initialization and a warm start; future comparisons should state this asymmetry explicitly when interpreting accuracy differences."],"forward_implications":["Widely used initializations provably fail to control all three signal-propagation metrics at once, so the depth problem in GCNs is not only an architectural or normalization issue but also an initialization issue.","For ReLU-activated vanilla GCNs, graph embedding variation is independent of weight variance, meaning over-smoothing cannot be cured by scalar rescaling; a per-layer search or a different activation is required.","SPoGInit stabilizes forward, backward, and graph embedding variation across GCN, ResGCN, gatResGCN, MixHop, and GAT, with 64-layer ResGCN and gatResGCN models gaining accuracy over their 4-layer counterparts.","On long-range tasks such as missing-feature node classification and MILP-based combinatorial optimization, SPoGInit shifts the optimal depth toward larger networks and improves the accuracy of 16-layer bipartite GCNs.","SPoGInit is transferable across starting initializations: starting from Conventional, Xavier, VirgoFor, or VirgoBack, the variance search improves deep-network accuracy relative to the starting point."],"supporting_citations":[{"why":"Supplies the vanilla GCN and residual GCN architectures whose depth degradation is the problem under study.","marker":"Kipf & Welling, 2017"},{"why":"Supplies the mean-field signal-propagation theory that motivates the forward and backward propagation metrics.","marker":"Poole et al., 2016"},{"why":"Supplies the deep information-propagation analysis and the output-input norm-ratio viewpoint behind the forward metric.","marker":"Schoenholz et al., 2017"},{"why":"Defines Kaiming initialization, a baseline that the paper proves fails to control all three metrics simultaneously.","marker":"He et al., 2015"},{"why":"Defines Xavier initialization, a baseline and also the default starting point for the SPoGInit search.","marker":"Glorot & Bengio, 2010"},{"why":"Provides the spectral eigendecomposition of the normalized adjacency matrix used in the proofs of the signal-propagation theorems.","marker":"Oono & Suzuki, 2019"},{"why":"Connects Dirichlet energy to over-smoothing, which is the basis for the graph embedding variation metric.","marker":"Cai & Wang, 2020"},{"why":"Provides VirgoFor and VirgoBack, GCN-specific initialization baselines that SPoGInit is compared against.","marker":"Li et al., 2023"},{"why":"Source of the Conventional initialization variance and the missing-feature evaluation protocol for long-range tasks.","marker":"Zhao & Akoglu, 2020"},{"why":"Supplies the GNN-guided predict-and-search MILP framework used for the maximal independent set experiments.","marker":"Han et al., 2022"}],"fun_headline_variants":["SPoGInit turns GCN depth into accuracy gains","Signal-propagation variance search deepens GCN performance","SPoGInit optimizes three signals to rescue deep GCNs","Deep GCNs get a signal boost from SPoGInit"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The theoretical results live in the infinite-width Gaussian-process limit, and the experiments use width-64 networks, so the entire argument stands on the assumption that this limit faithfully predicts signal propagation and training behavior at finite width.","fun_headline_variants_meta":{"raw":{"variants":["SPoGInit turns GCN depth into accuracy gains","Signal-propagation variance search deepens GCN performance","SPoGInit optimizes three signals to rescue deep GCNs","Deep GCNs get a signal boost from SPoGInit"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00045,"raw_usage":{"total_tokens":2269,"prompt_tokens":945,"completion_tokens":1324,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":561,"completion_tokens_details":{"reasoning_tokens":1253}},"tokens_in":561,"tokens_out":1324,"duration_ms":11520,"temperature":1.0,"reasoning_tokens":1253,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:19:42.749246+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a 64-layer GCN with SPoGInit's searched variance profile but at width 16 and width 512 on the same datasets: if the accuracy gains over baselines shrink or vanish at either width, or if the three evaluated metrics no longer track training stability, the infinite-width premise is not faithful. A second decisive check is to run SPoGInit without the label-dependent backward term: if the accuracy gains disappear, the source of the improvement is gradient geometry informed by training labels, not generic signal stability.","supporting_citations":[{"cited_title":"Exponential expressivity in deep neural networks through transient chaos","cited_arxiv_id":null,"evidence_quote":"Supplies the mean-field signal-propagation theory that motivates the forward and backward propagation metrics."},{"cited_title":"Graph neural networks exponentially lose expressive power for node classification","cited_arxiv_id":null,"evidence_quote":"Provides the spectral eigendecomposition of the normalized adjacency matrix used in the proofs of the signal-propagation theorems."}],"review_version":2}