{"id":"fca8432c-fa0e-447d-b482-4b8f0c301436","arxiv_id":"2412.07243","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"DYNAMO-GAT prunes GNN attention edges between highly correlated nodes to prevent oversmoothing, but its main theoretical lemma contradicts its goal and its accuracy claims exceed its own table.","lead":"This paper proposes DYNAMO-GAT, a pruning method for graph neural networks that cuts attention edges between similar nodes to keep deep networks from collapsing into oversmoothing. The dynamical systems theory in the paper is internally inconsistent, and the experiments overstate the gains over the G2GAT baseline.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Lemma 4 contradicts Lemma 3: decreasing the spectral radius at the oversmoothing fixed point makes it more stable, not less, so the stated theorem predicts the opposite of the paper's central claim.","rationale":"The reader's weakest_assumption was that Lemmas 1 and 2 conflate convergence to a fixed point with convergence to the uniform oversmoothing state. That is a real looseness, but the decisive problem is the contradiction between Lemma 3 and Lemma 4: the paper's own stability criterion says smaller rho means more stable, while Lemma 4 claims smaller rho means less stable. The reader did identify this contradiction in their strongest_claim, so the rejection is well-founded, but the weakest_assumption I was asked to compare against is not the same load-bearing concern. The Lemma 4 contradiction is internal and does not depend on any outside consensus; it invalidates the formal justification for DYNAMO-GAT as a method that disrupts convergence to oversmoothed states. Even if the pruning heuristic works in practice, the stated theory cannot support the advertised mechanism, so the paper is not acceptable in its current form.","tokens_in":19143,"tokens_out":3317,"duration_ms":35690,"concrete_test":"Implement the linearized dynamics around the uniform oversmoothing fixed point X* for a small graph (e.g., two nodes, one feature) before and after DYNAMO-GAT pruning, and compute rho(J_F(X*)) and rho(J_FP(X*)) explicitly. If rho(J_FP) < rho(J_F), then Lemma 3 says the pruned fixed point is more stable, contradicting the paper's conclusion. Alternatively, on Cora with 128 layers, evaluate J_F and J_FP at the converged uniform state and report the full eigenvalue spectra; the stated claim requires rho(J_FP) > 1 while rho(J_F) <= 1 to show destabilization, but Lemma 4 asserts the reverse.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing concern is an internal contradiction in the theoretical core. Lemma 3 defines a fixed point X* of the GAT update as stable iff rho(J_f(X*)) <= 1 and explicitly states that rho > 1 makes it unstable. Lemma 4 then claims that DYNAMO-GAT pruning gives rho(J_FP(X*)) < rho(J_F(X*)) and concludes \"reduced stability of the oversmoothing fixed point.\" That conclusion is backwards: by the paper's own Lemma 3, decreasing rho makes the fixed point more stable, not less. The proof of Lemma 4 asserts the conclusion by saying that pruning \"reduces the entries in the Jacobian\" and therefore lowers rho; it never explains how lowering rho could destabilize anything. Consequently, the central theorem, taken literally, predicts that pruning stabilizes the very oversmoothing state the method is supposed to disrupt. To destabilize the oversmoothing fixed point, one would need rho(J_FP(X*)) > 1, the opposite inequality. This is not a disagreement with consensus; it is an internal inconsistency between Lemma 3 and Lemma 4. A secondary issue is Lemma 5's rank claim: rank(C(t)) = d is impossible when the number of nodes n is less than the feature dimension d (e.g., Cornell has 183 nodes and d = 1703 input features), since rank(C(t)) <= n-1 < d; but the Lemma 4 contradiction alone is decisive.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a dynamical-systems analysis of oversmoothing in graph attention networks and introduces DYNAMO-GAT, a pruning strategy based on noise-driven covariance analysis and Anti-Hebbian principles. The theoretical part states three lemmas on fixed points, convergence, and stability of GAT updates, followed by two lemmas claiming that the pruning rule reduces the spectral radius of the Jacobian at the oversmoothing fixed point and preserves the full rank of the feature covariance matrix. Experiments on Cora, Citeseer, Cornell, and synthetic datasets compare accuracy, oversmoothing coefficient, and computational efficiency against GCN, GAT, and G2GAT.","tokens_in":19509,"tokens_out":3266,"duration_ms":33332,"significance":"If the theoretical claims were correct, the paper would offer a principled connection between attention pruning and oversmoothing, which is a relevant and timely topic. The empirical study is broad: it covers multiple real and synthetic datasets, reports the oversmoothing coefficient across depths, and evaluates the accuracy/GFLOPS trade-off, and it should be credited for comparing against a strong recent baseline (G2GAT). However, the central theoretical result contains a sign error that inverts the paper's main claim, and a second lemma makes an impossible rank assertion; as presented, the theoretical contribution is not valid. The empirical results may still be of interest, but they do not by themselves establish the claimed mechanism.","major_comments":[{"comment":"Lemma 4 states that rho(J_FP(X*)) < rho(J_F(X*)) and concludes that DYNAMO-GAT pruning reduces the stability of the oversmoothing fixed point. This is logically inverted relative to Lemma 3, which defines a fixed point as stable precisely when rho(J_f(X*)) <= 1 and unstable when rho > 1. Decreasing the spectral radius therefore makes the oversmoothing fixed point more stable, not less, so the stated lemma predicts the opposite of the abstract's claim that DYNAMO-GAT disrupts convergence to oversmoothed states. To support the claimed destabilization, the paper would need rho(J_FP(X*)) > rho(J_F(X*)) (or rho > 1), not the inequality proved.","section":"Theoretical Results, Lemma 4"},{"comment":"The proof asserts that pruning reduces the magnitudes of Jacobian entries and therefore reduces the spectral radius, but this is not derived from the pruning rule of Eqs. (4)-(5); it is essentially the lemma's conclusion restated as the proof's premise. In addition, the lemma assumes that the oversmoothing fixed point X* of F remains a point at which the pruned map FP is evaluated, but no argument shows that F(X*) = X* implies FP(X*) = X*. Since pruning changes the transformation, stability analysis of X* under FP requires FP(X*) = X*, and this is not established.","section":"Lemma 4, Short Proof"},{"comment":"The claim that rank(C(t)) = d for all t is impossible whenever the number of nodes n is smaller than the feature dimension d. For example, Cornell has n = 183 nodes and input feature dimension d = 1703, so rank(C(t)) <= n - 1 < d for any centered covariance matrix. The base case of the induction, rank(C(0)) = d, is therefore false for such datasets, and the proof cannot establish the claimed full-rank preservation.","section":"Lemma 5"},{"comment":"These lemmas conflate convergence to a fixed point (or attractor) of the GAT update with oversmoothing, i.e., indistinguishability of all node representations. A contraction mapping on feature space can converge to a non-uniform fixed point; uniqueness of a fixed point does not imply that all nodes share the same feature vector. This conflation is load-bearing because it motivates the objective of destabilizing the uniform fixed point; if the fixed point reached under contraction is not necessarily the oversmoothed state, then the dynamical-systems rationale for the pruning criterion collapses.","section":"Lemmas 1 and 2"}],"minor_comments":[{"comment":"The number of edges for Cornell is reported as 280 in Table 1 but as 295 in the supplementary experimental setup; these numbers should be reconciled.","section":"Table 1 and Supplementary Experimental Section"},{"comment":"The input list includes the initial pruning constant K0, but K0 is never used in the algorithm body; either use it or remove it from the requirements.","section":"Algorithm 1"},{"comment":"The name 'DYNAMO-GAT' is misspelled as 'DYANMO-GAT' in the last two paragraphs of the Related Works section.","section":"Related Works"},{"comment":"The GAT reference appears twice (Veličković et al., 2018), once with a full citation and once as Veličković et al. in the ICLR format; duplicate entries should be merged.","section":"References"}],"recommendation":"reject","confidential_remarks":"The internal contradiction between Lemma 3 and Lemma 4 is not a matter of presentation: the central theoretical claim that pruning reduces the stability of the oversmoothing fixed point is the opposite of what the paper's own stability criterion implies. Lemma 5's rank claim is untenable for the very datasets used in the experiments. These are load-bearing errors that cannot be fixed by local edits; a substantially new theoretical argument would be needed. The empirical comparison may be salvageable as a standalone study, but the paper in its current form is not ready for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the pruning algorithm is real and new, but the theory is backwards. Lemma 4 says decreasing the spectral radius at the oversmoothing fixed point \"reduces stability.\" Lemma 3 already defined stability as rho <= 1. So decreasing rho makes the fixed point more stable, not less. The theorem, read literally, predicts the opposite of the paper's claim. The proof of Lemma 4 just says pruning reduces matrix entries and therefore rho; it never explains why that would destabilize anything. Lemma 5 is also false as stated: rank(C(t)) = d is impossible when n < d, which is the case for Cornell (183 nodes, d = 1703). The covariance matrix of n centered vectors has rank at most n-1.\n\nWhat's actually new: DYNAMO-GAT's pruning rule — Gaussian noise injection, covariance-based anti-Hebbian scoring, gradual weight decay with recalibration — is a concrete algorithm I haven't seen in the GNN oversmoothing literature. That's worth acknowledging. The paper also runs experiments on three real and two synthetic datasets and reports an efficiency metric, which is more than many oversmoothing papers do.\n\nThe soft spots are not minor. Lemmas 1 and 2 conflate convergence to any fixed point with convergence to the uniform state; a contraction can have a non-uniform fixed point, so the dynamical-systems motivation doesn't establish oversmoothing. Table 1 itself shows G2GAT beating DYNAMO-GAT on Cora and Citeseer, contradicting the text's \"consistently achieves the highest accuracy.\" There are no error bars, no code, and the key hyperparameters (sigma, r0, gamma, beta, epsilon) are not specified. The supplementary proofs are sketches that assert their conclusions.\n\nWho is this for? Someone working on deep GNNs might find the pruning heuristic interesting, but the paper as written would mislead more than help. The theoretical core is internally inconsistent, and the experiments don't back the headline claims. I'd desk reject. If the authors fix the sign error in Lemma 4, rework Lemma 5, and release code with proper ablations, the algorithm could be worth a fresh look.","headline":"The pruning rule is genuinely new, but the theory has the sign backwards: Lemma 4 contradicts Lemma 3, so the central claim predicts the opposite of what the paper intends.","tokens_in":19995,"tokens_out":2882,"would_cite":false,"duration_ms":29101,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DYNAMO-GAT prunes attention edges between correlated nodes so that deep graph networks keep node features distinct instead of collapsing toward a uniform state.","keywords":["DYNAMO-GAT","oversmoothing","graph attention networks","spectral radius","covariance-based pruning","anti-Hebbian learning","fixed-point stability","deep GNNs"],"falsifier":"Compute the Jacobian of a trained GAT at a state where all nodes have identical features, apply DYNAMO-GAT pruning, and evaluate both spectral radii; if the pruned spectral radius falls below 1 while the network still converges to, or stays near, the uniform state, then the claimed destabilization is not what preserves diversity.","tokens_in":18932,"feed_emoji":"🕸️","tokens_out":6934,"duration_ms":68091,"temperature":0.7,"pith_summary":"This paper argues that oversmoothing in graph neural networks is a dynamical consequence: repeated attention-based message passing drives node representations toward a low-dimensional attractor where all nodes become alike. To stop that convergence, it introduces DYNAMO-GAT, which injects Gaussian noise, estimates the covariance of node features, and prunes attention edges between highly correlated nodes using an anti-Hebbian rule, then recalibrates the surviving weights. The paper claims that this pruning shrinks the spectral radius of the Jacobian of the pruned layer transformation at the oversmoothing fixed point, destabilizing that fixed point and preventing feature collapse. It further claims that the covariance matrix of node features keeps full rank at every layer, and reports that a 128-layer DYNAMO-GAT maintains near-constant oversmoothing coefficients and better accuracy than GCN, GAT, and G2GAT on Cora, Citeseer, and Cornell while using fewer floating-point operations.","feed_headline":"Pruning rule keeps 128-layer graph networks from going flat","feed_subtitle":"Noise-driven covariance pruning holds node features apart, beating GCN, GAT, and G2GAT on the tested graphs.","key_machinery":"The load-bearing object is the Jacobian matrix of the GNN layer transformation at the uniform fixed point, together with the noise-injected covariance matrix of node features. The Jacobian supplies the paper's stability criterion: a fixed point is stable when the spectral radius of the Jacobian at that point is at most 1, and the paper claims pruning reduces that spectral radius. The covariance matrix identifies pairs of nodes whose features are drifting together, and the anti-Hebbian pruning probability pij = r(t)·|αij|/(τ(t)·(Cii + Cjj ∓ 2Cij)) weakens exactly those edges. Recalibration of the surviving attention weights keeps information flow balanced after pruning.","core_discovery":"The paper's central claim is that oversmoothing can be treated as convergence to an unstable-to-be-disrupted fixed point of the GNN layer map, and that pruning one kind of attention edge alters the stability of that point. Formally, Lemma 4 states that for the oversmoothing fixed point X*, the pruned map FP satisfies ρ(JFP(X*)) < ρ(JF(X*)), where J denotes the Jacobian; the paper interprets this smaller spectral radius as reduced stability of the oversmoothing fixed point. Lemma 5 states that DYNAMO-GAT preserves rank(C(t)) = d at every layer, so node features never collapse into a subspace of dimension less than d. The paper presents these as theoretical support for its experimental finding that deep DYNAMO-GAT networks keep node representations diverse and accurate at 128 layers.","pith_inferences":["Read literally, Lemma 4 cuts against the paper's verbal claim: Lemma 3 defines the oversmoothing fixed point as stable exactly when the spectral radius is at most 1, so a smaller spectral radius after pruning would make that fixed point more stable, not less; the paper never reconciles this sign reversal.","Because pruning changes the layer map itself, X* need not remain a fixed point of the pruned map; if it is not, comparing Jacobians at X* describes a state the dynamics no longer sit at, so the theorem's object may not be the right one.","A natural testable extension is to check whether the covariance criterion actually removes the edges whose deletion changes the dominant eigenvector of the Jacobian, rather than only its spectral radius."],"forward_implications":["A GNN built on this rule can be run to 128 layers without the accuracy drop that normally follows from oversmoothing, on the datasets tested.","The pruning rule doubles as an efficiency mechanism: reported accuracy per floating-point operation is higher than GCN, GAT, and G2GAT because most attention edges are removed while accuracy stays near the best baselines.","If Lemma 5 holds, feature covariance never loses rank, so no layer can project all node features into a subspace of dimension below the feature dimension d.","Strong performance on dense, low-homophily synthetic graphs suggests the rule does not rely on homophily; it helps where neighbors are dissimilar."],"supporting_citations":[{"why":"Introduces oversmoothing as homogenization of node features, the problem the paper targets.","marker":"Li et al. [2018]"},{"why":"Provides the theoretical result that repeated message passing exponentially loses expressive power, motivating the dynamical-systems framing.","marker":"Oono and Suzuki [2020]"},{"why":"Defines the Graph Attention Network layer whose attention weights DYNAMO-GAT prunes.","marker":"Velickovic et al. [2018]"},{"why":"Supplies the oversmoothing coefficient metric and evidence that attention GNNs still oversmooth without regulation.","marker":"Wu et al. [2023]"},{"why":"Provides G2GAT, the main deep-network baseline that DYNAMO-GAT is compared against.","marker":"Rusch et al. [2023a]"},{"why":"Supplies the noise-driven covariance probing idea from which the anti-Hebbian pruning criterion is adapted.","marker":"Moore and Chaudhuri [2020]"},{"why":"Provides the synthetic datasets with controlled density and homophily used in the evaluation.","marker":"Zhu et al. [2020]"},{"why":"Frames oversmoothing in dynamical-systems terms, the perspective the paper extends.","marker":"Roth and Liebig [2024]"}],"fun_headline_variants":["Anti-Hebbian pruning disrupts oversmoothing in deep graph nets","Covariance-based pruning stops node features from collapsing","Dynamical systems lens yields pruning rule that beats oversmoothing"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that whenever the GAT update converges to a fixed point, the node features become identical; if a GAT can converge to a non-uniform equilibrium, then destabilizing the uniform fixed point is not the right objective and the pruning rule loses its theoretical justification.","fun_headline_variants_meta":{"raw":{"variants":["Anti-Hebbian pruning disrupts oversmoothing in deep graph nets","Covariance-based pruning stops node features from collapsing","Dynamical systems lens yields pruning rule that beats oversmoothing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000227,"raw_usage":{"total_tokens":1453,"prompt_tokens":909,"completion_tokens":544,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":525,"completion_tokens_details":{"reasoning_tokens":487}},"tokens_in":525,"tokens_out":544,"duration_ms":46250,"temperature":1.0,"reasoning_tokens":487,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:56:30.739997+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the Jacobian of a trained GAT at a state where all nodes have identical features, apply DYNAMO-GAT pruning, and evaluate both spectral radii; if the pruned spectral radius falls below 1 while the network still converges to, or stays near, the uniform state, then the claimed destabilization is not what preserves diversity.","supporting_citations":[{"cited_title":"Graph neural networks exponentially lose expressive power for node classification","cited_arxiv_id":null,"evidence_quote":"Provides the theoretical result that repeated message passing exponentially loses expressive power, motivating the dynamical-systems framing."},{"cited_title":"Graph attention networks","cited_arxiv_id":null,"evidence_quote":"Defines the Graph Attention Network layer whose attention weights DYNAMO-GAT prunes."},{"cited_title":"Using noise to probe recurrent neural network structure and prune synapses","cited_arxiv_id":null,"evidence_quote":"Supplies the noise-driven covariance probing idea from which the anti-Hebbian pruning criterion is adapted."},{"cited_title":"Beyond homophily in graph neural networks: Current limitations and effective designs","cited_arxiv_id":null,"evidence_quote":"Provides the synthetic datasets with controlled density and homophily used in the evaluation."},{"cited_title":"Simplifying the Theory on Over-Smoothing","cited_arxiv_id":"2407.11876","evidence_quote":"Frames oversmoothing in dynamical-systems terms, the perspective the paper extends."}],"review_version":1}