REVIEW 2 major objections 6 minor 56 references
Distributed Equivariant Graph Neural Networks for Large-Scale Electronic Structure Prediction
T0 review · 2 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read This paper claims that a distributed equivariant graph neural network using an 'incoming edge' data layout and a low-neighbor graph partitioner can scale electronic structure prediction to graphs of up to 190,000 atoms: it reports strong…
desk verdict Real systems contribution in the partitioner and distribution scheme; the scaling headline needs an end-to-end training measurement before it reads as training scaling. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The 'incoming edge' data layout: every rank owns a subset of nodes and every edge directed into one of its nodes, so that the attention mechanism and the aggregation of incoming messages require no communication; the only exchange is fetching the source-node embeddings belonging to other ranks during message creation. The Low-NN partitioner: a recursive bisection of the atom graph that balances total node degree, respects 3-D periodic boundaries, and at each cut chooses the dimension adding the fewest new neighbouring partitions, producing ring-like communication topologies instead of the dense topologies of minimum-cut partitioners.
What would settle it
Measure end-to-end training step time (forward and backward passes including gradient communication) for the same 3,000-atom HfO2 graph at 32, 128, and 512 GPUs and compare the parallel efficiency against the paper's forward-pass-only weak-scaling curve; if per-step efficiency falls below about 80% at 512 GPUs while the forward pass alone shows 87%, the headline scaling overstates training performance.
Extended reading notes
Core claim
The paper's central claim is that the memory and communication bottlenecks that limited rotationally equivariant ESP networks to graphs of under 150 atoms can be broken by distributing the atom graph so that each GPU owns its nodes and all incoming edges, making attention and aggregation communication-free, and by partitioning with a recursive bisection that minimizes the number of neighbouring ranks rather than the number of cut edges. With this scheme the paper reports strong scaling to 128 GPUs and weak scaling to 512 GPUs at 87% parallel efficiency on graphs of 3,000 to 190,000 atoms, and constructs a dense Hamiltonian of size 104,833 orbitals for an 8,064-atom GeSbTe cell with a mean absolute error of 2.64 meV, comparable to prior ESP models trained on much smaller structures.
Load-bearing premise
The scaling results assume that the single-GPU message-throughput saturation curve and the measured forward-pass runtimes represent real training, so that backward-pass gradient communication and compute-communication interaction do not erode the reported efficiency.
Editorial extensions
If this is right
- Training graphs with more than an order of magnitude more atoms than prior eGNN ESP models (8,000+ versus about 150) become feasible.
- The 'incoming edge' scheme is not specific to Hamiltonian prediction: any node-and-edge GNN with dense graphs avoids communication in attention and aggregation, so the distribution strategy transfers to other tensor-valued atomistic targets.
- Device-scale quantum transport simulations can, in principle, be fed DFT-level Hamiltonians for disordered structures such as phase-change memory cells instead of periodically tiled ideal crystals.
- With Low-NN partitioning, weak scaling efficiency stays near 87% even when the problem grows in 3-D, not just in one dimension.
Reading between the lines
- The accuracy claim (2.64 meV) is measured on a near-crystalline validation structure of 1,008 atoms, while the 8,064-atom Hamiltonian is shown as a reconstruction without a DFT reference; the scaling result is therefore better supported than the accuracy-at-scale result.
- Because the backward pass is handled separately by the deep-learning framework, a natural test is whether gradient communication across ranks preserves the forward-pass efficiency; the paper's own split suggests the 87% number may be an upper bound for training.
- The low-neighbor objective could be evaluated as a generic graph-partitioning heuristic for other dense-message-passing GNN workloads, for example by comparing end-to-end step times to METIS on graphs of heterogeneous density.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents a distributed implementation of an equivariant graph neural network (eGNN) for predicting DFT-level Hamiltonian matrices. The authors propose an 'incoming edge' data distribution scheme, in which each rank owns all edges directed into its local nodes, and a 'Low-NN' graph partitioning algorithm that minimizes the number of neighboring ranks while balancing node degree. They report strong-scaling measurements of the node-update block up to 128 GPUs and weak-scaling measurements of the forward pass up to 512 GPUs (87% efficiency) on the Alps supercomputer, for HfO2 systems with up to tens of thousands of atoms. As an application, they train a model on a 1,008-atom amorphous GST structure and perform Hamiltonian inference on a 8,064-atom tiled structure, obtaining a mean absolute error of 2.64 meV.
Significance. If the reported scaling extends to training, this is a significant advance for machine-learned electronic structure: it would allow full-batch training and inference on densely connected atomic graphs at device-relevant sizes, going well beyond the ~150-atom limit of prior ESP models. The benchmarking methodology is careful (medians of 120 runs, first 20 discarded, per-rank variance shown), the comparison against METIS provides a legitimate external baseline, and the code availability statement is positive. The main unresolved point is whether the scaling holds for end-to-end training rather than only for the forward pass; this limits the strength of the claims as currently presented.
major comments (2)
- [Abstract, §1, §4.1, §5-6] The abstract and Section 1 claim that the distributed implementation enables training on structures with over an order of magnitude more atoms than previously treated and that it shows strong scaling to 128 GPUs and weak scaling to 512 GPUs with 87% efficiency. However, the scaling measurements in Figures 6 and 7 are for the node-update block and the forward pass only. Section 4.1 states that the backward pass is performed on each rank and that gradient communication is handled by torch, but no timing or scaling data for the backward pass or for end-to-end training are reported. The GST experiment in Section 6 trains for 7,700 steps on 40 GPUs but gives no total training time or scaling efficiency for that run. Since full-batch training requires gradient synchronization at every optimizer step, and since that communication is not measured, the headline scaling numbers do not substantiate the training claim. The authors should either measure end-to-end training time (including backward pass and gradient allreduce) as a function of rank count, or explicitly rephrase the scaling claims as applying to the forward pass and inference.
- [Figure 6 and Section 4.4] The strong-scaling speedup in Figure 6 is computed relative to the best-performing distribution at 32 ranks, not relative to a single rank, and no absolute parallel-efficiency values are reported. Because the single-GPU saturation curve (Fig. 3) motivates using 32 ranks as the baseline, the reader cannot assess how much of the available single-GPU performance is retained at 128 ranks. Reporting the actual throughput or parallel efficiency values would strengthen the claim 'strong scaling up to 128 GPUs' and make the result more reproducible.
minor comments (6)
- [Section 4.4 / Figure 6] The text states that strong-scaling runtimes are measured for two values of r_cut = 6.0 to 7.0 Å, while the caption of Figure 6 states r_cut = 12.0 and 14.0 Å; please correct the inconsistency.
- [Figure 5] The caption says Fig. 5(a) 'contains 24k atoms and 14.5 million edges', but Section 4.4 describes Fig. 5(a) as the 3,000-atom unit cell before tiling to 24k atoms; please clarify which structure is shown.
- [Section 5] The sentence 'we therefore measure the weak scaling efficiency of the forward pass when the problem size is increased by tiling in 1-D (Fig. 7(a)...) or 3-D (Fig. 7(a))' contains a typo: the second reference should be Fig. 7(b).
- [Section 4.4] The text refers to the 'node update block (Fig. 2(c))', but Fig. 2(c) shows the edge update block; the node update block is depicted in Fig. 2(b).
- [Appendix A] The pseudocode for the Low-NN algorithm has inconsistencies: Get-Cut is called with arguments (pos_xyz, N_D, dim, O) but its signature is Get-Cut(pos_xyz, N_D, dim, O); the return values are assigned inconsistently (e.g., 'O_xyz = Get-Cut(...)' in one branch versus 'O, pos_l, pos_r = Get-Cut(...)' in another), and the variable L is used in Cut-Domain but is not in the parameter list.
- [References] References [8] and [16] are the same paper (Wang et al., Universal materials model); they should be merged or cited once.
Circularity Check
No significant circularity: the scaling and accuracy claims are measured outcomes, and the cited architecture dependency is not load-bearing for the paper's distributed-contribution claims.
full rationale
The paper's central claims are empirical results obtained by direct measurement on the Alps supercomputer: strong scaling of the node-update block up to 128 GPUs, weak scaling of the forward pass up to 512 GPUs with 87% parallel efficiency, a 10.8% median speedup from Low-NN over METIS at 128 GPUs, and a 2.64 meV mean absolute error for the GST Hamiltonian. None of these numbers is forced by a fitted parameter or by the definition of an input quantity. The Low-NN partitioning algorithm is specified independently in Appendix A (balanced recursive bisection minimizing neighbor count) and is then evaluated against the independent METIS baseline, so its benefit is an observed outcome rather than a construction. The architecture is stated to be 'based on the architectures of Refs. [26, 31]', where Ref. [26] is prior work by overlapping authors, but this is a standard component dependency: the present contribution is the distributed implementation, the incoming-edge distribution scheme, and the Low-NN partitioner, not the SO(2)-equivariant message-passing core. No equation in the paper reduces a target prediction to a quantity that was fit in advance, and no load-bearing conclusion depends solely on a self-citation. The skeptic concern that backward-pass gradient communication is not directly measured in the scaling plots is a measurement-completeness or correctness-risk issue, not a circularity issue, because the forward-pass and node-update timings are presented as measurements of those specific components rather than as predictions derived from the scaling claims.
Assumptions & free parameters
free parameters (4)
- Embedding size E =
16
- Maximum angular momentum lmax =
4
- Cutoff radius r_cut =
12.0 Å for GST; 6.0, 7.0, 12.0, 14.0 Å in scaling tests
- Number of message passing layers M =
3
assumptions (5)
- standard math Rotational covariance of Hamiltonian subblocks under Wigner-D and Clebsch-Gordan coupling.
- domain assumption Localized Gaussian-type orbital basis and DFT (CP2K) Hamiltonians are valid ground truth.
- domain assumption Near-sightedness justifies a finite cutoff radius that captures all relevant non-zero H entries.
- ad hoc to paper The single-GPU message-throughput saturation curve bounds the useful distributed regime.
- ad hoc to paper Low-NN objective: minimizing neighbor partitions and balancing node degree reduces communication overhead enough to beat minimum-cut partitioning.
Cite this review
Pith. "Pith review of Distributed Equivariant Graph Neural Networks for Large-Scale Electronic Structure Prediction." pith.science (2026). https://pith.science/paper/CWOG6Q4Z
@misc{pith2026250703840,
author = {Pith},
title = {Pith review of: Distributed Equivariant Graph Neural Networks for Large-Scale Electronic Structure Prediction},
year = {2026},
howpublished = {\url{https://pith.science/paper/CWOG6Q4Z}},
note = {Machine review of arXiv:2507.03840}
}
read the original abstract
Equivariant Graph Neural Networks (eGNNs) trained on density-functional theory (DFT) data can potentially perform electronic structure prediction at unprecedented scales, enabling investigation of the electronic properties of materials with extended defects, interfaces, or exhibiting disordered phases. However, as interactions between atomic orbitals typically extend over 10+ angstroms, the graph representations required for this task tend to be densely connected, and the memory requirements to perform training and inference on these large structures can exceed the limits of modern GPUs. Here we present a distributed eGNN implementation which leverages direct GPU communication and introduce a partitioning strategy of the input graph to reduce the number of embedding exchanges between GPUs. Our implementation shows strong scaling up to 128 GPUs, and weak scaling up to 512 GPUs with 87% parallel efficiency for structures with 3,000 to 190,000 atoms on the Alps supercomputer.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
Mads Brandbyge, José-Luis Mozos, Pablo Ordejón, Jeremy Taylor, and Kurt Stokbro. 2002. Density-functional method for nonequilibrium electron trans- port. Physical Review B, 65, 16, (Mar. 2002). doi:10.1103/physrevb.65.165401
-
[2]
Leonard Deuschle, Alexander Maeder, Vincent Maillou, Nicolas Vetsch, Anders Winka, Jiang Cao, Alexandros Nikolaos Ziogas, and Mathieu Luisier. 2024. To- wards exascale simulations of nanoelectronic devices in the GW approximation. In SC24: International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, (Nov. 2024), 1–16. d...
arXiv 2024
-
[3]
Geoffrey Yeap et al. 2024. 2nm platform technology featuring energy-efficient nanosheet transistors and interconnects co-optimized with 3DIC for AI, HPC and mobile SoC applications. In2024 IEEE International Electron Devices Meeting (IEDM). IEEE, (Dec. 2024), 1–4. doi:10.1109/iedm50854.2024.10873475
arXiv 2024
-
[4]
W. Kohn and L. J. Sham. 1965. Self-consistent equations including exchange and correlation effects. Physical Review, 140, 4A, (Nov. 1965), A1133–A1138. doi:10.1103/physrev.140.a1133
-
[5]
Alexandros Nikolaos Ziogas, Tal Ben-Nun, Guillermo Indalecio Fernández, Timo Schneider, Mathieu Luisier, and Torsten Hoefler. 2019. A data-centric approach to extreme-scale ab initio dissipative quantum transport simulations. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC ’19). ACM, (Nov...
-
[6]
Anker, Lena Simine, and Volker L
Yuanbin Liu, Ata Madanchi, Andy S. Anker, Lena Simine, and Volker L. De- ringer. 2024. The amorphous state as a frontier in computational materials design. Nature Reviews Materials, 10, 3, (Dec. 2024), 228–241. doi:10.1038/s4157 8-024-00754-2
-
[7]
Haiyang Yu, Zhao Xu, Xiaofeng Qian, Xiaoning Qian, and Shuiwang Ji. 2023. Efficient and equivariant graph networks for predicting quantum hamiltonian. (2023). doi:10.48550/ARXIV.2306.04922
work page Pith review arXiv doi:10.48550/arxiv.2306.04922 2023
-
[9]
Xiaoxun Gong, He Li, Nianlong Zou, Runzhang Xu, Wenhui Duan, and Yong Xu
Show all 56 references
-
[10]
Kühne et al
Thomas D. Kühne et al. 2020. CP2K: an electronic structure and molecular dy- namics software package - quickstep: efficient and accurate electronic structure calculations. J. Chem. Phys., 152, 19, (May 2020), 194103
2020
-
[11]
Frank Neese. 2011. The orca program system. WIREs Computational Molecular Science, 2, 1, (June 2011), 73–78. doi:10.1002/wcms.81
2011 doi
-
[12]
Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess E
Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P. Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess E. Smidt, and Boris Kozinsky
- [13]
-
[14]
Jörg Behler. 2016. Perspective: machine learning potentials for atomistic simu- lations. The Journal of Chemical Physics , 145, 17, (Nov. 2016). doi:10.1063/1.496 6192
2016 doi
- [15]
-
[16]
Yuxiang Wang et al. 2024. Universal materials model of deep-learning density functional theory hamiltonian. Science Bulletin, 69, 16, 2514–2521. doi:https://d oi.org/10.1016/j.scib.2024.06.011
2024 doi
-
[17]
Deringer
Yuxing Zhou, Wei Zhang, En Ma, and Volker L. Deringer. 2023. Device-scale atomistic modelling of phase-change memory materials. Nature Electronics, 6, 10, (Sept. 2023), 746–754. doi:10.1038/s41928-023-01030-x
2023 doi
-
[18]
Nicolas Onofrio, David Guzman, and Alejandro Strachan. 2015. Atomic origin of ultrafast resistance switching in nanoscale electrometallization cells. Nature Materials, 14, 4, (Mar. 2015), 440–446. doi:10.1038/nmat4221
2015 doi
-
[19]
Manasa Kaniselvan, Mathieu Luisier, and Marko Mladenović. 2023. An atom- istic model of field-induced resistive switching in valence change memory.ACS Nano, 17, 9, (Mar. 2023), 8281–8292. doi:10.1021/acsnano.2c12575
2023 doi
- [20]
-
[21]
Weile Jia, Han Wang, Mohan Chen, Denghui Lu, Lin Lin, Roberto Car, Weinan E, and Linfeng Zhang. 2020. Pushing the limit of molecular dynamics with ab initio accuracy to 100 million atoms with machine learning. In Proceedings of the International Conference for High Performance...
2020
-
[22]
Divya Suman, Jigyasa Nigam, Sandra Saade, Paolo Pegolo, Hanna Türk, Xing Zhang, Garnet Kin-Lic Chan, and Michele Ceriotti. 2025. Exploring the design space of machine learning models for quantum chemistry with a fully differen- tiable framework. Journal of Chemical Theory and ...
2025 doi
- [23]
-
[24]
Johann Brehmer, Sönke Behrends, Pim de Haan, and Taco Cohen. 2024. Does equivariance matter at scale? (2024). https://arxiv.org/abs/2410.23179 arXiv: 2410.23179 [cs.LG]
2024 arXiv
- [25]
-
[26]
Chen Hao Xia, Manasa Kaniselvan, Alexandros Nikolaos Ziogas, Marko Mlade- nović, Rayen Mahjoub, Alexander Maeder, and Mathieu Luisier. 2025. Learning the hamiltonian matrix of large atomic systems. (2025). doi:10.48550/ARXIV.25 01.19110
2025 doi
-
[27]
Hohenberg and W
P. Hohenberg and W. Kohn. 1964. Inhomogeneous electron gas.Physical Review, 136, 3B, (Nov. 1964), B864–B871. doi:10.1103/physrev.136.b864
1964 doi
-
[28]
Boris Kozinsky, Albert Musaelian, Anders Johansson, and Simon Batzner. 2023. Scaling the leading accuracy of deep equivariant models to biomolecular simu- lations of realistic size. In Proceedings of the International Conference for High Performance Computing, Networking, Stor...
2023
-
[29]
Ilyes Batatia, Dávid Péter Kovács, Gregor N. C. Simm, Christoph Ortner, and Gábor Csányi. 2022. MACE: higher order equivariant message passing neural networks for fast and accurate force fields. (2022). doi:10.48550/ARXIV.2206.07 697
2022 doi
-
[30]
Yutack Park, Jaesun Kim, Seungwoo Hwang, and Seungwu Han. 2024. Scalable parallel algorithm for graph neural network interatomic potentials in molecular dynamics simulations. Journal of Chemical Theory and Computation , 20, 11, (May 2024), 4857–4868. doi:10.1021/acs.jctc.4c00190
2024 doi
- [31]
- [32]
-
[33]
A. P. Thompson et al. 2022. LAMMPS - a flexible simulation tool for particle- based materials modeling at the atomic, meso, and continuum scales. Comput. Phys. Commun., 271, 108171
2022
- [34]
-
[35]
Kevin Han, Bowen Deng, Amir Barati Farimani, and Gerbrand Ceder. 2025. Distmlip: a distributed inference platform for machine learning interatomic potentials. (2025). doi:10.48550/ARXIV.2506.02023
2025 doi
-
[36]
Owen, Mordechai Kornbluth, and Boris Kozinsky
Albert Musaelian, Simon Batzner, Anders Johansson, Lixin Sun, Cameron J. Owen, Mordechai Kornbluth, and Boris Kozinsky. 2023. Learning local equivari- ant representations for large-scale atomistic dynamics.Nature Communications, 14, 1, (Feb. 2023). doi:10.1038/s41467-023-36329-y
2023 doi
-
[37]
W. Kohn. 1996. Density functional and density matrix method scaling linearly with the number of atoms. Physical Review Letters, 76, 17, (Apr. 1996), 3168–
1996
- [38]
-
[39]
Jason Ansel et al. 2024. PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph Compilation. In 29th ACM Inter- national Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 (ASPLOS ’24) . ACM, (Apr....
2024 doi
-
[40]
Ryosuke Okuta, Yuya Unno, Daisuke Nishino, Shohei Hido, and Crissman Loomis. 2017. CuPy: a NumPy-compatible library for NVIDIA GPU calculations. In Proceedings of Workshop on Machine Learning Systems (LearningSys) in The Thirty-first Annual Conference on Neural Information Pro...
2017
- [41]
-
[42]
d.] Karypis, george; kumar, vipin
[n. d.] Karypis, george; kumar, vipin. (1997). METIS: a software package for partitioning unstructured graphs, partitioning meshes, and computing fill- reducing orderings of sparse matrices. retrieved from the university digital conservancy, https://hdl.handle.net/11299/215346
1997
-
[43]
PyMETIS: a python wrapper for METIS
2024. PyMETIS: a python wrapper for METIS. https://github.com/inducer/pym etis. (2024). Distributed Equivariant Graph Neural Networks for Large-Scale Electronic Structure Prediction
2024
-
[44]
Thomas M. J. Fruchterman and Edward M. Reingold. 1991. Graph drawing by force-directed placement. Software: Practice and Experience , 21, 11, (Nov. 1991), 1129–1164. doi:10.1002/spe.4380211102
1991 doi
-
[45]
Owen, and Rodrigo Freitas
Killian Sheriff, Daniel Xiao, Yifan Cao, Lewis R. Owen, and Rodrigo Freitas
-
[46]
Burr et al
Geoffrey W. Burr et al. 2010. Phase change memory technology. Journal of Vacuum Science & Technology B, 28, 2, (Mar. 2010), 223–262. eprint: https://p ubs.aip.org/avs/jvb/article- pdf /28/2/223/16127414/223\_1\_online.pdf. doi:10.1116/1.3301579
2010 doi
-
[47]
Manuel Le Gallo and Abu Sebastian. 2020. An overview of phase-change mem- ory device physics. Journal of Physics D: Applied Physics , 53, 21, (Mar. 2020), 213002. doi:10.1088/1361-6463/ab7794
2020 doi
-
[48]
Intel Announces Optane Memory M15: 3D XPoint On M.2 PCIe 3.0 x4
2019. Intel Announces Optane Memory M15: 3D XPoint On M.2 PCIe 3.0 x4. https://www.anandtech.com/show/14437/intel-announces-optane-memory- m15-3d-xpoint-on-m2-pcie-30-x4. Accessed online. (2019)
2019
-
[49]
Khaddam-Aljameh et al
R. Khaddam-Aljameh et al. 2021. HERMES core – a 14nm CMOS and PCM-based in-memory compute core using an array of 300ps/LSB linearized CCO-based ADCs and local digital processing. In 2021 Symposium on VLSI Circuits . IEEE, (June 2021), 1–2. doi:10.23919/vlsicircuits52068.2021.9492362
2021
-
[50]
Deringer
Yuxing Zhou, Wei Zhang, En Ma, and Volker L. Deringer. 2023. Device-scale atomistic modelling of phase-change memory materials. Nature Electronics, 6, 10, 746–754
2023
-
[51]
Nils Holle, Sebastian Walfort, Riccardo Mazzarello, and Martin Salinga. 2025. Effect of peierls-like distortions on transport in amorphous phase change devices. Communications Materials, 6, 1, (Mar. 2025). doi:10.1038/s43246-025-0 0776-5
2025 doi
-
[52]
Bin Chen, Xue-Peng Wang, Fangying Jiao, Long Ning, Jiaen Huang, Jiatao Xie, Shengbai Zhang, Xian-Bin Li, and Feng Rao. 2023. Suppressing structural re- laxation in nanoscale antimony to enable ultralow-drift phase-change memory applications. Advanced Science, 10, 25, (June 202...
2023 doi
-
[53]
Ryotaro Okabe et al. 2024. Virtual node graph neural network for full phonon prediction. Nature Computational Science, 4, 7, (July 2024), 522–531. doi:10.103 8/s43588-024-00661-0
2024
-
[2022]
Nature Communications, 13, 1, (May 2022)
E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nature Communications, 13, 1, (May 2022). doi:10.1038 /s41467-022-29939-5
2022
-
[2023]
Nature Communications, 14, 1, (May 2023)
General framework for E(3)-equivariant neural network representation of density functional theory hamiltonian. Nature Communications, 14, 1, (May 2023). doi:10.1038/s41467-023-38468-8
2023 doi
- [2025]
-
[3171]
doi:10.1103/physrevlett.76.3168
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.