{"id":"b3f9b02e-3950-4d04-b1d2-1db72b7f5c59","arxiv_id":"2506.19482","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"FastEGNN and DistEGNN use ordered, learnable virtual nodes with an MMD alignment loss to make equivariant GNNs accurate on sparse and distributed large geometric graphs.","lead":"Adding a small set of learnable virtual nodes lets equivariant graph networks skip most particle-to-particle connections and still predict positions accurately. The approach also splits huge graphs across multiple GPUs, keeping virtual nodes synchronized, and is demonstrated on fluid simulations with over 100,000 particles.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The MMD regularizer in Eq. (11) aligns virtual nodes to ground-truth positions X^GT, not to the model's own real coordinates X^(L); the alignment assumed by Prop. IV.3 is therefore untested at inference, where ground truth is absent.","rationale":"The reader's weakest assumption identifies the virtual-node approximation premise and notes the p=1.00 degradation. I agree that this is the central risk, but I sharpen it to a specific, testable flaw: Eq. (11) aligns virtual nodes to X^GT rather than to the model's own real-node coordinates X^(L). This means the condition assumed by Prop. IV.3—that virtual coordinates match the real coordinates used in message passing—is not enforced by the training loss and is unlikely to hold at test time, especially under rollout or distribution shift. The small MMD sample size (3 nodes) makes the alignment even weaker. This is not an ad hominem or a disagreement with consensus; it is an internal inconsistency between the theoretical premise and the actual objective. The paper has real strengths: the equivariance proof is plausible, the benchmarks are extensive, and the code is promised, but this concern directly affects the interpretation of the headline results. I recommend keeping the reader's CONDITIONAL verdict, adding the concrete test above as a condition for acceptance. I set agreement_with_reader to 'partial' because the reader pointed to the general area but did not identify the specific ground-truth-target mismatch or the small MMD sample size.","tokens_in":33935,"tokens_out":6953,"duration_ms":76213,"concrete_test":"Retrain FastEGNN-⟨3,0.75⟩ on Protein Dynamics (and, if feasible, Water-3D) with Eq. (11) modified so that L_MMD is computed against X^(L), the model's own final real-node coordinate predictions, instead of X^GT, keeping all other hyperparameters from Table IX identical. Report test MSE and the test-time MMD between Z^(L) and X^(L) over the test set for both variants. If the modified model achieves test MSE within 5% of the original and maintains low test-time MMD, the ground-truth-alignment concern is mitigated. If test MSE degrades substantially, the original gains depend on information from ground-truth positions leaking through the auxiliary loss, and the central approximation claim needs to be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that a small ordered set of virtual nodes can approximate the full real-node message-passing graph. The theoretical support, Prop. IV.3, is conditional on the virtual coordinates 'well approximating' the real coordinates. But the only mechanism enforcing this is the MMD loss in Eq. (11), whose target is X^GT, the ground-truth final positions, not the model's own predicted real coordinates X^(L). This creates a train/test mismatch: during training, gradients push Z^(L) toward the future/target positions, and the real nodes learn to exploit virtual nodes that are 'aware' of the target distribution; at inference, virtual nodes are initialized at the CoM and evolve without any such signal, so the alignment condition assumed by Prop. IV.3 is not guaranteed to hold. The problem is compounded by Table IX, which shows the MMD term is estimated from only 3 sampled real nodes per graph on the three benchmark datasets, making the alignment signal extremely weak and noisy. The paper's own all-edges-dropped results are consistent with this concern: on N-body, FastEGNN-⟨3,1.00⟩ gives MSE 9.52e-2 versus 1.06e-2 with full edges, close to the Linear baseline (12.66e-2), indicating that the virtual nodes alone do not actually recover the real-node message passing. Thus the load-bearing premise of the paper—that virtual nodes approximate the real graph—is not validated by the training objective as written, and the theoretical argument in Section IV-D does not bridge this gap.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces FastEGNN and DistEGNN, two extensions of EGNN that add a small ordered set of learnable virtual nodes. FastEGNN keeps a sparse real-real edge set but connects every real node to every virtual node, using distinct message passing for each virtual channel and an MMD regularizer to encourage spatial alignment between virtual and real coordinates. DistEGNN partitions large graphs across devices and shares the virtual nodes across partitions to propagate global information. The authors report accuracy improvements over EGNN and other baselines on N-body, protein dynamics, Water-3D, and a new Fluid113K benchmark, along with substantial inference speedups when edges are dropped and per-device memory reductions in distributed training. The E(3)-equivariance proof in Appendix A is straightforward and appears correct. The central theoretical claim is that a small fixed set of virtual nodes can approximate the information carried by the full real-node graph, supported by Proposition IV.3, which is conditional on the virtual coordinates 'well approximating' the real coordinates. The main weaknesses are an inconsistency between the MMD formulation in Eq. (10) and the loss in Eq. (11), the lack of any estimate of variance across seeds, and an overstatement of the all-edges-dropped results on N-body.","tokens_in":34323,"tokens_out":6015,"duration_ms":66260,"significance":"If the claims hold, the paper advances the scalability of equivariant GNNs in a useful direction: it shows that a modest number of virtual nodes can preserve much of the expressivity of dense message passing while allowing aggressive edge sparsification, and it provides a concrete distributed scheme that processes graphs with over 100K nodes. The modular extension to RF, SchNet, and TFN broadens the potential impact. The new Fluid113K benchmark and the accompanied code release are concrete community assets. The E(3)-equivariance argument in Appendix A is clean and machine-verifiable in structure. However, the load-bearing premise that virtual nodes approximate the full real-node graph is not yet convincingly validated: the theoretical result is conditional, and the empirical evidence at the extreme edge-dropping rate is mixed, especially on N-body. The MMD target inconsistency and the absence of seed variance prevent the current empirical claims from being fully assessed.","major_comments":[{"comment":"There is a direct inconsistency between the MMD definition and its use in the loss. Eq. (10) defines LMMD using the predicted real node coordinates x_i^(L), but Eq. (11) writes LMMD(Z^(L), X^GT), i.e., against ground-truth final positions. If the implementation follows Eq. (11), then the regularizer leaks future target information during training while being absent at inference, so the alignment condition assumed by Proposition IV.3 is not guaranteed at test time. If the implementation follows Eq. (10), the paper should correct Eq. (11). The authors should state which target is used and provide an evaluation (e.g., measuring MMD between virtual coordinates and predicted real coordinates at inference) to support the approximation premise.","section":"Section IV-C, Eq. (10)-(11)"},{"comment":"The all-edges-dropped claims are overstated. The text says FastEGNN 'can still perform promisingly even when all edges are dropped' and that virtual nodes approximate the real graph, but Table I shows N-body MSE of 9.25–9.72e-2 at p=1.00, which is about 9x worse than the same model with full edges (1.04–1.10e-2) and only modestly better than the Linear baseline (12.66e-2). This contradicts the claim that a small ordered set of virtual nodes effectively approximates the large unordered graph of real nodes. The Water-3D and Protein results at p=1.00 are more supportive, but the N-body result should be discussed honestly and the claim should be restricted to settings where it actually holds.","section":"Section IV-D and Table I"},{"comment":"No error bars, confidence intervals, or seed variance are reported anywhere in the empirical evaluation. Several accuracy differences used to support design choices are small (e.g., FastEGNN-⟨10,0.75⟩ at 0.99e-2 versus FastEGNN-⟨1,0.75⟩ at 1.03e-2 on N-body; the MMD ablation differences in Table II). Without multiple seeds or a variance estimate, the claimed superiority over baselines and the conclusions of the ablations cannot be reliably assessed. The authors should add seed variance or, at minimum, report the number of seeds and the standard deviation for the main tables.","section":"Section VII-B, Tables I-III"},{"comment":"Proposition IV.3 establishes a universal form for an update that depends only on virtual nodes, but it does not by itself prove that C virtual nodes can approximate the information from N real neighbors. The sentence 'f(x_i,Z) is able to universally approximate the messages from all the real nodes, if the virtual coordinates Z can well approximate the real ones X' is a heuristic bridge, not a theorem. The paper needs either a quantitative statement of approximation error as a function of C and the MMD, or a direct empirical test of the assumption (e.g., measuring the MMD between virtual and real node coordinates at inference on the test set). As written, the theoretical support for the central claim is not load-bearing.","section":"Section IV-D, Proposition IV.3"},{"comment":"The distributed-memory claims should be qualified. While per-device memory is reduced, the total memory across devices grows with the number of devices, and DistEGNN often uses more total memory than EGNN at the same device count (e.g., Fluid113K at 8 devices: DistEGNN-5 total memory 12.62 GB versus EGNN 3.77 GB; relative training time 0.83 versus 0.15). The abstract's phrase 'dramatically reducing memory and computational overhead' is therefore only true per device and relative to the single-device setting. The paper should distinguish per-device and total memory, and should compare against the best feasible single-device baseline (which for Fluid113K is not available) rather than implying a universal overhead reduction.","section":"Section VII-D, Tables IV-V"}],"minor_comments":[{"comment":"Typo: 'We have the flowing theoretical assurance' should read 'following'.","section":"Section IV-B, before Proposition IV.1"},{"comment":"The notation 'CM' in Eq. (5) is undefined; it appears to denote concatenation over c, but it should be written explicitly (e.g., concatenation symbol) and introduced in the text.","section":"Section IV-B, Eq. (5)"},{"comment":"The phrase 'surjection from dao to' contains a typo; it should be 'surjection from ... to ...' with the intended sets spelled out.","section":"Appendix A, Theorem A.5"},{"comment":"The dataset statistics for Water-3D are inconsistent: the text says 1000/100/100 trajectories, while Table VIII lists 15,000/1,500/1,500 samples. The authors should clarify the units and the relationship between trajectories and training samples.","section":"Section VII-A and Table VIII"},{"comment":"Several places contain duplicated or overlapping text blocks (e.g., around Eqs. (14)-(15) and in the captions of Figs. 3-4), likely artifacts of the LaTeX compilation. These should be cleaned before final submission.","section":"General presentation"},{"comment":"The claimed '29% improvement' over EGNN on N-body does not match Table I: 1.04e-2 versus 1.41e-2 is a 26% reduction. Please recompute or correct the percentage.","section":"Section VII-B1, observation 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a candidate for publication after substantial revision. The core empirical results at moderate sparsification rates (p=0.75) and the distributed results are interesting, and the benchmark contribution is real. The main risk is the gap between the theoretical narrative and the actual training objective: the MMD target inconsistency in Eq. (11) and the lack of a direct test of the virtual-node approximation at inference. These issues can be fixed within the manuscript's scope, so I do not recommend rejection. I would also ask the editor to encourage the authors to add seed variance, as the current tables do not allow significance assessment."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know about arXiv:2506.19482. First, the genuinely new part is DistEGNN, the distributed extension with a gradient-corrected all_reduce for cross-device virtual nodes, plus the Fluid113K dataset with 113K nodes. FastEGNN itself appeared in the authors' ICML 2024 paper and is cited as such. Second, the central theoretical claim—that a few virtual nodes can replace the real message-passing graph—is only partially supported, and the paper overstates what happens when all real-real edges are dropped.\n\nWhat the paper does well: the distributed framework is a real step forward. The experiments show DistEGNN maintains accuracy across 1-8 devices on Water-3D, and it is the first equivariant GNN that can handle over 100K nodes. The plugin results on RF, SchNet, and TFN show the virtual-node idea generalizes beyond EGNN. The E(3)-equivariance proof in Appendix A is clean, and the ablations (ordered vs. global virtual nodes, with/without MMD) support the design choices. The Fluid113K dataset is a useful benchmark for the community.\n\nWhere the soft spots are: the biggest one is the MMD objective. Eq. (11) aligns virtual nodes with ground-truth coordinates, not with the model's own predicted real coordinates. The theoretical guarantee in Prop. IV.3 assumes virtual coordinates approximate real coordinates, but that assumption is never enforced at inference, where ground truth is absent. The paper's own N-body results at p=1.00 (MSE 9.52e-2 vs 1.06e-2 with full edges) show the virtual nodes alone do not recover the real message passing. The claim that FastEGNN 'still performs well even when all edges are dropped' is overstated on N-body. On Water-3D the all-edges-dropped result is better than EGNN, but still 1.4x worse than FastEGNN with full edges. The MMD is computed from only 3 sampled real nodes per graph on the three benchmark datasets (Table IX), which is a very weak alignment signal. The paper needs error bars or seed variance; none are reported anywhere. Minor points: the 'plugin' claim is slightly generous for FastTFN, which reduces TFN to a single channel; the 'first distributed equivariant GNN' claim is plausible but the related work on distributed geometric GNNs is thin.\n\nThe stress-test note rings true, but with a caveat. Optimizing MMD against X^GT is a surrogate for the target distribution, and one can imagine the virtual nodes tracking the model's own coordinates indirectly. The paper does not test or argue this, and the p=1.00 results suggest it doesn't hold in the extreme. This needs a clearer discussion and probably a fix: use the model's own X^(L) in the MMD, or at least report the mismatch.\n\nBottom line: the paper is worth a serious referee. The distributed contribution and the dataset are significant. The theoretical support for the virtual-node approximation is weaker than claimed, and the all-edges-dropped statement needs to be toned down. I'd recommend major revision: add error bars, clarify the MMD target, and soften the universality claim. This is the kind of paper I'd take to reading group and cite in my own work.","headline":"DistEGNN and the Fluid113K dataset are the real contributions here; the theoretical claim that virtual nodes approximate the real graph is shakier than the paper admits.","tokens_in":34863,"tokens_out":3529,"would_cite":true,"duration_ms":31378,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a small ordered set of virtual nodes can replace most real-real message passing in equivariant graph neural networks, preserving accuracy while scaling to graphs with more than 100,000 nodes and cutting memory and…","keywords":["equivariant graph neural networks","virtual nodes","E(3)-equivariance","graph sparsification","distributed training","maximum mean discrepancy","physical dynamics simulation","large-scale geometric graphs"],"falsifier":"Train the p=1.00 variant on the N-body system while sweeping C from 1 to N; if the MSE plateaus well above the full-edge EGNN baseline even when C=N and the virtual nodes are initialized at the real coordinates, the central approximation claim is falsified.","tokens_in":33755,"feed_emoji":"💧","tokens_out":7249,"duration_ms":66432,"temperature":0.7,"pith_summary":"The paper is trying to establish that equivariant graph neural networks can be sparsified and distributed without losing accuracy by adding a small ordered set of virtual nodes. It argues that these virtual nodes, kept mutually distinct and spatially aligned with real nodes, can approximate the information carried by dense real-real message passing, so real-real edges can be dropped for speed. If true, it would let large physical simulation graphs, up to 113,000 nodes in the paper's Fluid113K benchmark, be processed on a single device or across multiple low-memory devices. The paper reports that FastEGNN beats full-edge EGNN across all three smaller benchmarks, and that DistEGNN cuts per-device memory sharply while limiting accuracy loss.","feed_headline":"Virtual nodes let equivariant GNNs scale to 113K particles","feed_subtitle":"A few virtual nodes replace dense edges; DistEGNN cuts per-device memory to 1.58 GB across 8 devices.","key_machinery":"The central object is the ordered set of $C$ virtual nodes with coordinates $\\vec{Z}\\in\\mathbb{R}^{3\\times C}$ and features $S\\in\\mathbb{R}^{H\\times C}$, initialized at the graph center of mass so the model starts E(3)-equivariant. Three message types carry the computation: EGNN-style real-real messages $m^r_{ij}$ (Eq. 3), an E(3)-invariant virtual-global matrix $m^v = (\\vec{Z} - \\bar{x}\\mathbf{1}^\\top)^\\top(\\vec{Z} - \\bar{x}\\mathbf{1}^\\top)$ (Eq. 4), and per-pair real-virtual messages $m^v_{ic}$ (Eq. 5). Real coordinates update by pulling toward neighbor differences and virtual-node differences, while virtual coordinates update by aggregating over all real nodes. The MMD loss in Eq. (10), with an RBF kernel, is the mechanism that keeps the virtual nodes spread out and aligned with the real distribution, and it is what makes Proposition IV.3's premise, that virtual coordinates approximate real ones, plausible.","core_discovery":"On the paper's own terms, the discovery is that a small ordered set of virtual nodes, each with a distinct semantic role, gives an E(3)-equivariant update $f(\\vec{x}_i, \\vec{Z}) = \\vec{x}_i + \\sum_{c=1}^C (\\vec{z}_c - \\vec{x}_i)\\psi_c\\big(\\odot_{c=1}^C \\|\\vec{z}_c - \\vec{x}_i\\|^2, m^v\\big)$ that is capable of approximating the real-node aggregation, provided the virtual coordinates $\\vec{Z}$ approximate the real coordinates $\\vec{X}$. The paper enforces that alignment with an E(3)-invariant Maximum Mean Discrepancy loss (Eq. 10), and enforces distinct roles by treating the virtual nodes as an ordered set with separate message-passing functions. FastEGNN realizes this as EGNN-style real-real message passing plus real-virtual message passing, and DistEGNN extends it by sharing the virtual nodes across partitioned devices and synchronizing them during training. The reported consequence is that with 75% of edges dropped, FastEGNN matches or beats full-edge EGNN, and with all edges dropped it degrades gracefully, whereas EGNN collapses to near-linear performance.","pith_inferences":["A direct extension of the paper's reasoning is that the virtual-node aggregator's capacity should scale with the complexity of the target interaction: for systems with strong many-body or long-range couplings, $C$ may need to grow with $N$, so a useful test is to measure the p=1.00 error as a function of $C/N$ on the N-body benchmark.","The same MMD-alignment mechanism could be reused for equivariant output heads beyond coordinates, such as forces or energies, where the virtual coordinates would serve as an invariant summary scaffold rather than as predicted positions; the paper does not test this.","Because DistEGNN's virtual nodes are synchronized across devices, communication cost is roughly linear in $C$ per layer regardless of partition count, suggesting the speedup over EGNN should keep growing as devices are added; this could be checked by profiling communication time separately from compute time."],"forward_implications":["Equivariant GNNs can be run on sparse graphs: dropping 75% of the longest edges costs little accuracy, with FastEGNN-⟨10,0.75⟩ reaching MSE 0.99×10⁻² on the N-body system, better than full-edge EGNN's 1.41×10⁻², at 53% of EGNN's inference time.","Even with all real-real edges removed, FastEGNN keeps physically meaningful predictions on Water-3D, with MSE 3.40×10⁻⁴ versus 12.38×10⁻⁴ for EGNN with all edges stripped, so virtual nodes alone can carry a substantial share of global information.","The virtual-node module transfers across architectures: FastRF, FastSchNet, and FastTFN improve over their baselines in accuracy and efficiency, with FastTFN-3-0.75 using 2% of TFN's memory and 20% of its time while lowering MSE by 56%.","DistEGNN makes 100K-node graphs tractable on modest hardware; with 8 devices, Fluid113K training is 7.34× faster and per-device memory drops from about 48 GB to 1.58 GB, while error only rises 15% across devices compared to 69% for EGNN."],"supporting_citations":[{"why":"Supplies the EGNN baseline whose message passing FastEGNN extends, including the real-real message form and the accuracy baseline the method must beat.","marker":"[9]"},{"why":"Supplies the Maximum Mean Discrepancy objective used to align virtual and real coordinates in Eq. (10).","marker":"[12]"},{"why":"Provides Proposition 1, the basis for the ordered-virtual-node update form in Proposition IV.3.","marker":"[24]"},{"why":"Provides Lemma 2, used to justify the virtual-global matrix $m^v$ as a universal E(3)-invariant function of virtual coordinates.","marker":"[29]"},{"why":"Provides Proposition 10 characterizing the general equivariant update, which the paper contrasts with its ordered-virtual-node universal form.","marker":"[30]"},{"why":"Supplies the METIS graph partitioning algorithm used in the partitioning ablation for DistEGNN.","marker":"[31]"},{"why":"Used to generate the Fluid113K dataset that demonstrates distributed scaling.","marker":"[37]"}],"fun_headline_variants":["Virtual nodes make equivariant GNNs scale to 113K particles","Equivariant GNNs scale to 113K particles via virtual nodes","Virtual nodes replace dense edges for fast equivariant GNNs","DistEGNN: Virtual nodes as global bridges for extreme-scale graphs","75% fewer edges, same accuracy with virtual-node GNNs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a small, fixed set of virtual nodes whose coordinates match the real nodes' distribution can carry the information that full real-real message passing would have carried.","fun_headline_variants_meta":{"raw":{"variants":["Virtual nodes make equivariant GNNs scale to 113K particles","Equivariant GNNs scale to 113K particles via virtual nodes","Virtual nodes replace dense edges for fast equivariant GNNs","DistEGNN: Virtual nodes as global bridges for extreme-scale graphs","75% fewer edges, same accuracy with virtual-node GNNs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000743,"raw_usage":{"total_tokens":3379,"prompt_tokens":1072,"completion_tokens":2307,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":688,"completion_tokens_details":{"reasoning_tokens":2214}},"tokens_in":688,"tokens_out":2307,"duration_ms":15937,"temperature":1.0,"reasoning_tokens":2214,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:06:48.706989+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the p=1.00 variant on the N-body system while sweeping C from 1 to N; if the MSE plateaus well above the full-edge EGNN baseline even when C=N and the virtual nodes are initialized at the real coordinates, the central approximation claim is falsified.","supporting_citations":[{"cited_title":"E (n) equiv- ariant graph neural networks,","cited_arxiv_id":null,"evidence_quote":"Supplies the EGNN baseline whose message passing FastEGNN extends, including the real-real message form and the accuracy baseline the method must beat."},{"cited_title":"Integrating structured biological data by kernel maximum mean discrepancy,","cited_arxiv_id":null,"evidence_quote":"Supplies the Maximum Mean Discrepancy objective used to align virtual and real coordinates in Eq. (10)."},{"cited_title":"Learning physical dynamics with subequivariant graph neural networks,","cited_arxiv_id":null,"evidence_quote":"Provides Proposition 1, the basis for the ordered-virtual-node update form in Proposition IV.3."},{"cited_title":"Equivariant graph mechanics networks with constraints,","cited_arxiv_id":null,"evidence_quote":"Provides Lemma 2, used to justify the virtual-global matrix $m^v$ as a universal E(3)-invariant function of virtual coordinates."},{"cited_title":"Scalars are universal: Equivariant machine learning, structured like classical physics,","cited_arxiv_id":null,"evidence_quote":"Provides Proposition 10 characterizing the general equivariant update, which the paper contrasts with its ordered-virtual-node universal form."},{"cited_title":"Metis—a software package for parti- tioning unstructured graphs, partitioning meshes and computing fill-reducing ordering of sparse matrices,","cited_arxiv_id":null,"evidence_quote":"Supplies the METIS graph partitioning algorithm used in the partitioning ablation for DistEGNN."},{"cited_title":"La- grangian fluid simulation with continuous convolutions,","cited_arxiv_id":null,"evidence_quote":"Used to generate the Fluid113K dataset that demonstrates distributed scaling."}],"review_version":1}