pith. sign in

arxiv: 2605.18404 · v2 · pith:USNR7PI7new · submitted 2026-05-18 · 💻 cs.DC

JanusPipe: Efficient Pipeline Parallel Training for Machine Learning Interatomic Potentials

Hongyu Wang , Weijian Liu , Hongtao Xu , Yan Wang , Mingzhen Li , Weile Jia , Guangming Tan This is my paper

Pith reviewed 2026-05-20 08:31 UTC · model grok-4.3

classification 💻 cs.DC
keywords machine learning interatomic potentialspipeline parallelismdistributed trainingconservative MLIPs3D parallelismSymFoldWaveKmolecular dynamics
0
0 comments X

The pith

JanusPipe introduces a tailored pipeline parallelism approach that handles the double-backward execution of conservative MLIPs to improve distributed training efficiency.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Conservative machine learning interatomic potentials require computing gradients as part of the forward pass, creating a double-backward pattern that clashes with standard pipeline parallel training systems designed for typical neural networks. The paper presents JanusPipe as a 3D parallel system that incorporates SymFold for memory-efficient pipeline execution and WaveK for balancing the four phases of computation to minimize idle time in the pipeline. If this works, it would make it feasible to train larger MLIP models on clusters of GPUs, following the scaling trends seen in other large models. This matters for researchers because more scalable training could support longer and more accurate molecular dynamics simulations at the atomic level without prohibitive computational costs.

Core claim

The authors develop JanusPipe, an efficient 3D-parallel training system for conservative MLIPs. It integrates SymFold to support memory-efficient pipeline parallelism despite the double-backward pattern and WaveK to reduce pipeline bubbles through balanced four-phase compute times. On 32 GPUs, this yields 1.51 times higher throughput than 1F1B and 1.45 times higher than Hanayo on average for conservative MLIP training.

What carries the argument

SymFold for memory-efficient pipeline parallelism adapted to double-backward execution and WaveK for balancing the four-phase compute time to reduce bubbles in the pipeline schedule.

Load-bearing premise

The double-backward execution pattern is the dominant source of inefficiency in existing pipeline-parallel systems for these models, and the overhead introduced by SymFold and WaveK remains negligible across the tested model sizes and GPU counts.

What would settle it

Running JanusPipe and a baseline like 1F1B on the same conservative MLIP model with 32 GPUs and comparing the measured training throughput; if the improvement is absent or reversed, the central claim would be falsified.

Figures

Figures reproduced from arXiv: 2605.18404 by Guangming Tan, Hongtao Xu, Hongyu Wang, Mingzhen Li, Weijian Liu, Weile Jia, Yan Wang.

Figure 1
Figure 1. Figure 1: (a) First-order workloads perform one forward pass and one backward pass per micro-batch. (b) Conservative MLIPs com￾pute forces by differentiating the predicted energy in the forward stage (F = −∇xE), which introduces a double-backward exe￾cution pattern with four phases (FE/FF/BF/BE). See [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Naively applying first-order PP schedules to conservative MLIPs causes redundant FE recomputation and residual pipeline bubbles. tFF < tBE. BF backpropagates the force loss through the force computation, and it is typically the most expensive phase because it involves double-backward. 2) Additional bubbles. This four-phase execution causes more bubbles in the steady state of the pipeline. As shown in [PIT… view at source ↗
Figure 3
Figure 3. Figure 3: SymFold transforms a first-order PP schedule into a correct second-order schedule. For simplicity in this figure, we assume that the four phases have identical execution times. the first-order pipeline schedule into a second-order one, en￾suring training correctness through four optimization passes (i.e., passes 0–3). It places FE and FF on the same device, reusing FE’s activations locally to eliminate red… view at source ↗
Figure 4
Figure 4. Figure 4: WaveK organizes the instructions into WaveK units and overlaps unit boundaries to reduce pipeline bubbles under the four￾phase partial order. Pass 4: WaveK Decomposition. Pass 4 takes the Sym￾Fold schedule as input and decomposes the four-phase exe￾cution into two parts: WaveK-F and WaveK-B. As shown in [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: WaveK schedules with different k (fixed Nmb=12). Top: k=4. Bottom: k=6. 𝑂𝑢𝑡: 𝐾!"#$ ①. Measure 𝑀!"#$%!#$&', 𝑀(#!#$" ⏱ Throughput profiler 𝑘 ∗ = 𝑎𝑟𝑔𝑚𝑎𝑥(𝑡ℎ𝑟𝑜𝑢𝑔ℎ𝑝𝑢𝑡(𝑘)) GPU profiler MLIP Model Input Records memory 𝑘%&' = 𝑀&(&"! − 𝑀#$&$") 𝑀𝑎𝑐𝑡𝑖𝑣𝑎𝑡𝑖𝑜𝑛 𝑘%"* = 𝑃𝑃 𝐹𝑜𝑟 𝑒𝑎𝑐ℎ 𝑘 𝑖𝑛 𝐾!"#$, generate a WaveK schedule, measure throughput. ②. Determine K Search Space ③. Measure Throughput 𝑭𝒊𝒍𝒕𝒆𝒓 𝑹𝒂𝒏𝒈𝒆 Micro-batch (Atomic gr… view at source ↗
Figure 6
Figure 6. Figure 6: Offline tuning selects the WaveK unit size k under a memory constraint. micro-batches. The top shows the case of k = 4 with three WaveK units, and the bottom shows the case of k = 6 with two WaveK units, resulting in fewer pipeline bubbles and achieving higher throughput. Increasing k reduces the num￾ber of unit boundaries, thereby confining bubbles to unit boundaries and improving steady-state overlap. Bu… view at source ↗
Figure 7
Figure 7. Figure 7: End-to-end throughput of 1F1B-2nd, Hanayo-2nd, and JanusPipe across MLIP models and PP/GP/DP settings. training, we adopt two widely used first-order baselines and adapt them accordingly for second-order training. 1F1B￾2nd is based on Megatron-LM’s 1F1B pipeline schedule, extended to support second-order training (Narayanan et al., 2021). Hanayo-2nd adopts the wave-style schedule from Hanayo (Liu et al., 2… view at source ↗
Figure 8
Figure 8. Figure 8: Peak device memory across 32 GPUs (violin plots), with absolute throughput (atoms/sec) and relative speedup annotated above each violin. UMA-2.3B (G=2) UMA-2.3B (G=4) UMA-1.2B (G=2) UMA-1.2B (G=4) 0 10 20 30 40 Peak GPU Memory (GB) OOM 38.7 23.6 18.9 23.6 17.6 13.0 8.0 20.0 14.1 11.2 6.2 PP=1 PP=4 PP=8 [PITH_FULL_IMAGE:figures/full_fig_p008_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Peak GPU memory versus pipeline degree P on UMA￾1.2B and UMA-2.3B under G ∈ {2, 4}. 5.4. WaveK Sensitivity The WaveK unit size k reveals a clear throughput–memory trade-off. Initially, throughput increases with larger k be￾cause pipeline bubbles decrease, but larger k is constrained by device memory. This motivates selecting k under a mem￾ory budget. On UMA-1.2B (P=4, G=D=1), k=8 achieves the largest speed… view at source ↗
Figure 10
Figure 10. Figure 10: Impact of micro-batch heterogeneity under PP and DP: bubbles and synchronization stalls. A.2. Lightweight Solver: Heuristic Algorithm GARS reduces step-time variance by repacking graphs into better-balanced micro-batches and tagging each micro-batch to select an efficient GP execution mode: comm-free local execution for small-graph micro-batches, and dist execution that splits oversized graphs across GP r… view at source ↗
Figure 11
Figure 11. Figure 11: reports the normalized throughput improvement over 1F1B-2nd. Overall, the three components are complementary. SymFold improves throughput by up to 23% by eliminating redundant recomputation and avoiding cross-device replicated parameter synchronization at optimizer-step boundaries. WaveK further improves throughput by 0–18% under a fixed memory budget by selecting an effective unit size k. GARS contribute… view at source ↗
Figure 12
Figure 12. Figure 12: UMA-1.2B (P=4, G=D=1): throughput and peak memory under varying wave size k. B.5. Bubble Analysis To evaluate scheduling efficiency, we analyze the pipeline bubble ratio on UMA-1.2B with P = 4 and Nmb = 12 using profiler traces. As shown in [PITH_FULL_IMAGE:figures/full_fig_p016_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Pipeline execution timelines on UMA-1.2B (P = 4). B.6. GARS Micro-benchmarks We micro-benchmark the impact of GARS on communication and load balance. We use UMA-1.2B and compare GARS against the same schedule without repacking, under identical global batch and parallelism settings [PITH_FULL_IMAGE:figures/full_fig_p016_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: UMA-1.2B: throughput (left y-axis) and halo All-Gather time (right y-axis) with SymFold+WaveK. GARS mitigates micro-batch imbalance. Across 1,000 iterations, GARS maintains a consistently low standard deviation of per-micro-batch atom counts ( [PITH_FULL_IMAGE:figures/full_fig_p017_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Per-iteration standard deviation of micro-batch atom counts over 1,000 iterations, with and without GARS. Lower values indicate more balanced micro-batches. B.7. Scalability Analysis We report strong and weak scaling results on UMA-2.3B. In strong scaling, we fix the total problem size and increase the number of devices. In weak scaling, we proportionally increase the global batch size with the number of … view at source ↗
Figure 16
Figure 16. Figure 16: Scalability analysis: strong scaling (left) and weak scaling (right). B.8. Correctness Validation Gradient Computation Correctness. Equation 2 shows that our gradient merging preserves the mathematical correctness of parameter updates. In non-pipelined training, the total gradient ∂Ltotal ∂θ naturally combines contributions from both energy and force losses. As shown in Equation 1, the parameter gradients… view at source ↗
Figure 17
Figure 17. Figure 17: plots MAE trajectories over 1,000 training iterations. The trajectories closely match, with mean absolute percentage errors of 0.84% for energy MAE and 0.21% for force MAE. The small residual discrepancies are attributable to non-associativity in floating-point arithmetic under distributed execution (e.g., different reduction/aggregation orders across pipeline stages), which is expected; empirically, both… view at source ↗
read the original abstract

Discovering atom-level phenomena requires molecular dynamics (MD) simulations with ab initio accuracy. Machine learning interatomic potentials (MLIPs) enable stable, high-accuracy MD simulations, and their models exhibit scaling-law trends similar to large language models. However, the lack of scalable and efficient distributed training systems for conservative MLIPs makes them difficult to scale. This is because conservative MLIPs inherently follow a double-backward execution pattern, which involves computing gradients during the forward pass. This pattern creates a mismatch with existing distributed training systems, especially for pipeline parallelism. Therefore, we present JanusPipe, an efficient 3D-parallel (PP/DP/GP) training system tailored for conservative MLIPs. It integrates SymFold to enable memory-efficient pipeline parallelism for conservative MLIPs, and WaveK to reduce pipeline bubbles by balancing the four-phase compute time. Experimental results on 32 GPUs show that JanusPipe improves throughput by $1.51\times$ and $1.45\times$ on average over 1F1B and Hanayo, respectively.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript introduces JanusPipe, a 3D-parallel (PP/DP/GP) training system for conservative machine learning interatomic potentials (MLIPs). It integrates SymFold to support memory-efficient pipeline parallelism that accommodates the double-backward execution pattern and WaveK to balance four-phase compute times and reduce pipeline bubbles. The central experimental claim is that JanusPipe delivers average throughput gains of 1.51× over 1F1B and 1.45× over Hanayo on 32 GPUs.

Significance. If the reported speedups are robust, the work would be significant for distributed systems supporting scalable MLIP training, which is needed for high-accuracy molecular dynamics simulations that follow scaling-law behavior. The paper receives credit for targeting a concrete mismatch between conservative MLIP computation and existing pipeline-parallel frameworks and for supplying named-baseline throughput numbers on a fixed GPU count.

major comments (2)
  1. [Experimental evaluation] Experimental evaluation: the abstract and results section report concrete 1.51×/1.45× throughput numbers on 32 GPUs but supply no information on model architectures, dataset sizes, exact hardware configuration, or statistical variance; without these the central performance claim cannot be fully evaluated.
  2. [Method (SymFold/WaveK)] SymFold and WaveK descriptions: the attribution of gains to resolution of the double-backward mismatch assumes that the overheads of SymFold folding and WaveK phase scheduling remain negligible, yet no ablation studies or per-component timing breakdowns are provided to confirm this for the evaluated MLIP sizes and GPU counts.
minor comments (2)
  1. [Abstract] The abstract would be clearer if it briefly indicated the scale or type of MLIP models used in the 32-GPU experiments.
  2. [Background] Notation for the four-phase execution pattern could be introduced with a small diagram or timing table to aid readers unfamiliar with conservative MLIP gradients.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on our manuscript. We appreciate the emphasis on strengthening the experimental claims and method validation. Below we respond point-by-point to the major comments and indicate the revisions made.

read point-by-point responses

  1. Referee: [Experimental evaluation] Experimental evaluation: the abstract and results section report concrete 1.51×/1.45× throughput numbers on 32 GPUs but supply no information on model architectures, dataset sizes, exact hardware configuration, or statistical variance; without these the central performance claim cannot be fully evaluated.

    Authors: We agree that these details are essential for full evaluation and reproducibility of the reported speedups. In the revised manuscript we have expanded the Experimental Setup section to specify the MLIP model architectures (including network depth, feature dimensions, and equivariant layers), the training dataset sizes and sources, the precise hardware configuration (32 NVIDIA A100 GPUs with NVLink interconnect), and statistical variance (mean and standard deviation across five independent runs). These additions directly support assessment of the 1.51× and 1.45× throughput gains. revision: yes

  2. Referee: [Method (SymFold/WaveK)] SymFold and WaveK descriptions: the attribution of gains to resolution of the double-backward mismatch assumes that the overheads of SymFold folding and WaveK phase scheduling remain negligible, yet no ablation studies or per-component timing breakdowns are provided to confirm this for the evaluated MLIP sizes and GPU counts.

    Authors: We acknowledge that explicit ablations and breakdowns would strengthen attribution of the gains. The original manuscript explains the design choices in SymFold and WaveK to keep overheads low for the double-backward pattern, but we have added a new subsection with per-component timing breakdowns on the 32-GPU configurations. These show SymFold and WaveK overheads remain below 4% of total time for the evaluated MLIP sizes, confirming the assumptions. A partial ablation isolating each component is also included based on existing experimental logs. revision: partial

Circularity Check

0 steps flagged

No circularity: throughput claims rest on external empirical measurements against 1F1B and Hanayo baselines.

full rationale

The paper introduces SymFold and WaveK as engineering mechanisms to address the double-backward pattern in conservative MLIPs under pipeline parallelism. Reported speedups (1.51× and 1.45×) are direct runtime measurements on 32 GPUs, not quantities derived from internal parameters, fitted constants, or self-referential equations. No load-bearing step reduces a claimed result to a definition or prior self-citation by construction; the central attribution is to measured net throughput after adding the new components. The derivation chain is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 2 invented entities

The central claim rests on the premise that the double-backward pattern creates a unique mismatch with existing pipeline schedulers and that the two new modules can be added with low overhead. No free parameters are fitted to data in the abstract; the invented entities are the two algorithmic modules themselves.

axioms (1)
  • domain assumption Existing pipeline-parallel frameworks assume a single forward-then-backward execution pattern.
    Invoked to motivate the need for SymFold and WaveK.
invented entities (2)
  • SymFold no independent evidence
    purpose: Enable memory-efficient pipeline parallelism for double-backward conservative MLIPs
    New module introduced to fold the computation graph.
  • WaveK no independent evidence
    purpose: Reduce pipeline bubbles by balancing four-phase compute times
    New scheduling component introduced to balance phases.

pith-pipeline@v0.9.0 · 5726 in / 1400 out tokens · 50964 ms · 2026-05-20T08:31:39.626201+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

94 extracted references · 94 canonical work pages · 10 internal anchors

  1. [1]

    A foundation model for atomistic materials chemistry

    A foundation model for atomistic materials chemistry. arXiv e-prints , keywords =. doi:10.48550/arXiv.2401.00096 , archivePrefix =. 2401.00096 , primaryClass =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2401.00096

  2. [2]

    Open Materials 2024 (OMat24) Inorganic Materials Dataset and Models

    Open Materials 2024 (OMat24) Inorganic Materials Dataset and Models. arXiv e-prints , keywords =. doi:10.48550/arXiv.2410.12771 , archivePrefix =. 2410.12771 , primaryClass =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2410.12771 2024

  3. [3]

    MatterSim: A Deep Learning Atomistic Model Across Elements, Temperatures and Pressures

    MatterSim: A Deep Learning Atomistic Model Across Elements, Temperatures and Pressures. arXiv e-prints , keywords =. doi:10.48550/arXiv.2405.04967 , archivePrefix =. 2405.04967 , primaryClass =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2405.04967

  4. [4]

    Forty-second International Conference on Machine Learning , year=

    Learning Smooth and Expressive Interatomic Potentials for Physical Property Prediction , author=. Forty-second International Conference on Machine Learning , year=

    work page

  5. [5]

    , title =

    Tuckerman, Mark E. , title =. 2010 , address =

    work page 2010

  6. [6]

    Scaling deep learning for materials discovery

    Merchant, Amil and Batzner, Simon and Schoenholz, Samuel S and Aykol, Muratahan and Cheon, Gowoon and Cubuk, Ekin Dogus. Scaling deep learning for materials discovery. Nature

    work page

  7. [7]

    , title =

    Qu, Eric and Krishnapriyan, Aditi S. , title =. Proceedings of the 38th International Conference on Neural Information Processing Systems , articleno =. 2025 , isbn =

    work page 2025

  8. [8]

    arXiv e-prints , keywords =

    Matbench Discovery -- A framework to evaluate machine learning crystal stability predictions. arXiv e-prints , keywords =. doi:10.48550/arXiv.2308.14920 , archivePrefix =. 2308.14920 , primaryClass =

    work page doi:10.48550/arxiv.2308.14920

  9. [9]

    Ilyes Batatia and David Peter Kovacs and Gregor N. C. Simm and Christoph Ortner and Gabor Csanyi , booktitle=. 2022 , url=

    work page 2022

  10. [10]

    Kitchin and Daniel S

    Brandon M Wood and Misko Dzamba and Xiang Fu and Meng Gao and Muhammed Shuaibi and Luis Barroso-Luque and Kareem Abdelmaqsoud and Vahe Gharakhanyan and John R. Kitchin and Daniel S. Levine and Kyle Michel and Anuroop Sriram and Taco Cohen and Abhishek Das and Sushree Jagriti Sahoo and Ammar Rizvi and Zachary Ward Ulissi and C. Lawrence Zitnick , booktitle...

    work page 2025

  11. [11]

    Scaling Laws for Neural Language Models

    Scaling Laws for Neural Language Models. arXiv e-prints , keywords =. doi:10.48550/arXiv.2001.08361 , archivePrefix =. 2001.08361 , primaryClass =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2001.08361 2001

  12. [12]

    International Conference on Learning Representations , year=

    Towards Training Billion Parameter Graph Neural Networks for Atomic Simulations , author=. International Conference on Learning Representations , year=

    work page

  13. [13]

    2022 , eprint=

    Towards Training Billion Parameter Graph Neural Networks for Atomic Simulations , author=. 2022 , eprint=

    work page 2022

  14. [14]

    Zhao, Yanli and Gu, Andrew and Varma, Rohan and Luo, Liang and Huang, Chien-Chin and Xu, Min and Wright, Less and Shojanazeri, Hamid and Ott, Myle and Shleifer, Sam and Desmaison, Alban and Balioglu, Can and Damania, Pritam and Nguyen, Bernard and Chauhan, Geeta and Hao, Yuchen and Mathews, Ajit and Li, Shen , title =. Proc. VLDB Endow. , month = aug, pag...

    work page doi:10.14778/3611540.3611569 2023

  15. [15]

    A brief review on importance of DFT in drug design , author=. Res. Med. Eng. Sci , volume=

    work page

  16. [16]

    Drug Discovery Today , volume=

    Applications of density functional theory in COVID-19 drug modeling , author=. Drug Discovery Today , volume=. 2022 , publisher=

    work page 2022

  17. [17]

    npj Computational Materials , volume=

    Computational understanding of Li-ion batteries , author=. npj Computational Materials , volume=. 2016 , publisher=

    work page 2016

  18. [18]

    Energy & Environmental Materials , volume=

    Density functional theory for battery materials , author=. Energy & Environmental Materials , volume=. 2019 , publisher=

    work page 2019

  19. [19]

    ACS Catalysis , volume=

    The Open Catalyst 2022 (OC22) dataset and challenges for oxide electrocatalysts , author=. ACS Catalysis , volume=. 2023 , publisher=

    work page 2022

  20. [20]

    Brabson and Abhishek Das and Zachary Ulissi and Matt Uyttendaele and Andrew J

    Anuroop Sriram and Sihoon Choi and Xiaohan Yu and Logan M. Brabson and Abhishek Das and Zachary Ulissi and Matt Uyttendaele and Andrew J. Medford and David S. Sholl , title =. 2023 , journal=

    work page 2023

  21. [21]

    Levine, Muhammed Shuaibi, Evan Walter Clark Spotte-Smith, Michael G

    The Open Molecules 2025 (OMol25) Dataset, Evaluations, and Models. arXiv e-prints , keywords =. doi:10.48550/arXiv.2505.08762 , archivePrefix =. 2505.08762 , primaryClass =

    work page doi:10.48550/arxiv.2505.08762 2025

  22. [22]

    Nature Machine Intelligence , volume=

    CHGNet as a pretrained universal neural network potential for charge-informed atomistic modelling , author=. Nature Machine Intelligence , volume=. 2023 , publisher=

    work page 2023

  23. [23]

    The Journal of Physical Chemistry Letters , volume=

    Accurate band gaps for semiconductors from density functional theory , author=. The Journal of Physical Chemistry Letters , volume=. 2011 , publisher=

    work page 2011

  24. [24]

    Lawrence and Ulissi, Zachary , title =

    Chanussot*, Lowik and Das*, Abhishek and Goyal*, Siddharth and Lavril*, Thibaut and Shuaibi*, Muhammed and Riviere, Morgane and Tran, Kevin and Heras-Domingo, Javier and Ho, Caleb and Hu, Weihua and Palizhati, Aini and Sriram, Anuroop and Wood, Brandon and Yoon, Junwoong and Parikh, Devi and Zitnick, C. Lawrence and Ulissi, Zachary , title =. ACS Catalysi...

    work page

  25. [25]

    Advances in neural information processing systems , volume=

    Large scale distributed deep networks , author=. Advances in neural information processing systems , volume=

    work page

  26. [26]

    PyTorch Distributed: Experiences on Accelerating Data Parallel Training

    Pytorch distributed: Experiences on accelerating data parallel training , author=. arXiv preprint arXiv:2006.15704 , year=

    work page internal anchor Pith review Pith/arXiv arXiv 2006

  27. [27]

    Horovod: fast and easy distributed deep learning in TensorFlow

    Horovod: fast and easy distributed deep learning in TensorFlow , author=. arXiv preprint arXiv:1802.05799 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  28. [28]

    SC20: International Conference for High Performance Computing, Networking, Storage and Analysis , pages=

    Zero: Memory optimizations toward training trillion parameter models , author=. SC20: International Conference for High Performance Computing, Networking, Storage and Analysis , pages=. 2020 , organization=

    work page 2020

  29. [29]

    Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

    Megatron-lm: Training multi-billion parameter language models using model parallelism , author=. arXiv preprint arXiv:1909.08053 , year=

    work page internal anchor Pith review Pith/arXiv arXiv 1909

  30. [30]

    Proceedings of the 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming , pages=

    DAPPLE: A pipelined data parallel approach for training large models , author=. Proceedings of the 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming , pages=

    work page

  31. [31]

    2025 , eprint=

    Orb-v3: atomistic simulation at scale , author=. 2025 , eprint=

    work page 2025

  32. [32]

    Nature Computational Science , volume=

    A universal graph deep learning interatomic potential for the periodic table , author=. Nature Computational Science , volume=. 2022 , publisher=

    work page 2022

  33. [33]

    Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis , articleno =

    Jia, Weile and Wang, Han and Chen, Mohan and Lu, Denghui and Lin, Lin and Car, Roberto and E, Weinan and Zhang, Linfeng , title =. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis , articleno =. 2020 , isbn =

    work page 2020

  34. [34]

    On the Opportunities and Risks of Foundation Models

    On the opportunities and risks of foundation models , author=. arXiv preprint arXiv:2108.07258 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  35. [35]

    ACS Applied Materials & Interfaces , year=

    Performance assessment of universal machine learning interatomic potentials: Challenges and directions for materials’ surfaces , author=. ACS Applied Materials & Interfaces , year=

    work page

  36. [36]

    Advances in neural information processing systems , volume=

    Gpipe: Efficient training of giant neural networks using pipeline parallelism , author=. Advances in neural information processing systems , volume=

    work page

  37. [37]

    Journal of machine learning research , volume=

    Automatic differentiation in machine learning: a survey , author=. Journal of machine learning research , volume=

    work page

  38. [38]

    arXiv preprint arXiv:2003.03123 , year=

    Directional message passing for molecular graphs , author=. arXiv preprint arXiv:2003.03123 , year=

    work page arXiv 2003

  39. [39]

    Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 , pages =

    Wang, Yujie and Wang, Shiju and Zhu, Shenhan and Fu, Fangcheng and Liu, Xinyi and Xiao, Xuefeng and Li, Huixia and Li, Jiashi and Wu, Faming and Cui, Bin , title =. Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 , pages =. 2025 , isbn =. doi:10.1145/3676641.3715998 , ...

    work page doi:10.1145/3676641.3715998 2025

  40. [40]

    The Journal of Physical Chemistry A , volume=

    Machine learning interatomic potentials and long-range physics , author=. The Journal of Physical Chemistry A , volume=. 2023 , publisher=

    work page 2023

  41. [41]

    arXiv e-prints , keywords =

    A Graph Neural Network for the Era of Large Atomistic Models. arXiv e-prints , keywords =. doi:10.48550/arXiv.2506.01686 , archivePrefix =. 2506.01686 , primaryClass =

    work page doi:10.48550/arxiv.2506.01686

  42. [42]

    Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

    Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. arXiv e-prints , keywords =. doi:10.48550/arXiv.1703.03400 , archivePrefix =. 1703.03400 , primaryClass =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.1703.03400

  43. [43]

    Improved Training of Wasserstein GANs

    Improved Training of Wasserstein GANs. arXiv e-prints , keywords =. doi:10.48550/arXiv.1704.00028 , archivePrefix =. 1704.00028 , primaryClass =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.1704.00028

  44. [44]

    Raissi, P

    Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics , keywords =. doi:10.1016/j.jcp.2018.10.045 , adsurl =

    work page doi:10.1016/j.jcp.2018.10.045 2018

  45. [45]

    2017 , eprint=

    Adam: A Method for Stochastic Optimization , author=. 2017 , eprint=

    work page 2017

  46. [46]

    2019 , eprint=

    Decoupled Weight Decay Regularization , author=. 2019 , eprint=

    work page 2019

  47. [47]

    Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis , articleno =

    Liu, Ziming and Cheng, Shenggan and Zhou, Haotian and You, Yang , title =. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis , articleno =. 2023 , isbn =. doi:10.1145/3581784.3607073 , abstract =

    work page doi:10.1145/3581784.3607073 2023

  48. [48]

    Nature , volume=

    DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning , author=. Nature , volume=. 2025 , publisher=

    work page 2025

  49. [49]

    2009 , month =

    Hey, Tony and Tansley, Stewart and Tolle, Kristin and Gray, Jim , title =. 2009 , month =

    work page 2009

  50. [50]

    1998 , publisher=

    The Feynman Lectures on Physics: The Complete Audio Collection , author=. 1998 , publisher=

    work page 1998

  51. [51]

    2020 , eprint=

    PairNorm: Tackling Oversmoothing in GNNs , author=. 2020 , eprint=

    work page 2020

  52. [52]

    Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis , articleno =

    Li, Shigang and Hoefler, Torsten , title =. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis , articleno =. 2021 , isbn =. doi:10.1145/3458817.3476145 , abstract =

    work page doi:10.1145/3458817.3476145 2021

  53. [53]

    Xing and Joseph E

    Lianmin Zheng and Zhuohan Li and Hao Zhang and Yonghao Zhuang and Zhifeng Chen and Yanping Huang and Yida Wang and Yuanzhong Xu and Danyang Zhuo and Eric P. Xing and Joseph E. Gonzalez and Ion Stoica , title =. 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22) , year =

    work page

  54. [54]

    Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis , articleno =

    Rajbhandari, Samyam and Ruwase, Olatunji and Rasley, Jeff and Smith, Shaden and He, Yuxiong , title =. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis , articleno =. 2021 , isbn =. doi:10.1145/3458817.3476205 , abstract =

    work page doi:10.1145/3458817.3476205 2021

  55. [55]

    Lee , booktitle=

    Seung Yul Lee and Hojoon Kim and Yutack Park and Dawoon Jeong and Seungwu Han and Yeonhong Park and Jae W. Lee , booktitle=. Flash. 2025 , url=

    work page 2025

  56. [56]

    , author=

    Scalable Parallel Algorithm for Graph Neural Network Interatomic Potentials in Molecular Dynamics Simulations. , author=. Journal of chemical theory and computation , year=

    work page

  57. [57]

    Kohn-Sham equations for multiplets , author =. Phys. Rev. A , volume =. 1998 , month =. doi:10.1103/PhysRevA.57.1672 , url =

    work page doi:10.1103/physreva.57.1672 1998

  58. [58]

    Computer Physics Communications , volume=

    The analysis of a plane wave pseudopotential density functional theory code on a GPU machine , author=. Computer Physics Communications , volume=. 2013 , publisher=

    work page 2013

  59. [59]

    Journal of Computational Physics , volume=

    Fast plane wave density functional theory molecular dynamics calculations on multi-GPU machines , author=. Journal of Computational Physics , volume=. 2013 , publisher=

    work page 2013

  60. [60]

    , title =

    Zhao, Zhengji and Austin, Brian and Rrapaj, Ermal and Wright, Nicholas J. , title =. Proceedings of the SC '24 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis , pages =. 2025 , isbn =. doi:10.1109/SCW63240.2024.00189 , abstract =

    work page doi:10.1109/scw63240.2024.00189 2025

  61. [61]

    Physical review letters , volume=

    Generalized neural-network representation of high-dimensional potential-energy surfaces , author=. Physical review letters , volume=. 2007 , publisher=

    work page 2007

  62. [62]

    Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 , pages =

    Yang, Shuangyan and Zhang, Minjia and Dong, Wenqian and Li, Dong , title =. Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 , pages =. 2023 , isbn =. doi:10.1145/3575693.3575725 , abstract =

    work page doi:10.1145/3575693.3575725 2023

  63. [63]

    ACM Trans

    Chen, Rong and Shi, Jiaxin and Chen, Yanzhe and Zang, Binyu and Guan, Haibing and Chen, Haibo , title =. ACM Trans. Parallel Comput. , month = jan, articleno =. 2019 , issue_date =. doi:10.1145/3298989 , abstract =

    work page doi:10.1145/3298989 2019

  64. [64]

    Forty-second International Conference on Machine Learning , year=

    The dark side of the forces: assessing non-conservative force models for atomistic machine learning , author=. Forty-second International Conference on Machine Learning , year=

    work page

  65. [65]

    2024 , url=

    Yi-Lun Liao and Brandon Wood and Abhishek Das* and Tess Smidt* , booktitle=. 2024 , url=

    work page 2024

  66. [66]

    The Twelfth International Conference on Learning Representations , year=

    EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations , author=. The Twelfth International Conference on Learning Representations , year=

    work page

  67. [67]

    npj Computational Materials , volume=

    DPA-2: a large atomic model as a multi-task learner , author=. npj Computational Materials , volume=. 2024 , publisher=

    work page 2024

  68. [68]

    OpenAI blog , volume=

    Language models are unsupervised multitask learners , author=. OpenAI blog , volume=

    work page

  69. [69]

    The Thirty-eighth Annual Conference on Neural Information Processing Systems , year=

    The Importance of Being Scalable: Improving the Speed and Accuracy of Neural Network Interatomic Potentials Across Chemical Domains , author=. The Thirty-eighth Annual Conference on Neural Information Processing Systems , year=

    work page

  70. [70]

    Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis , articleno =

    Narayanan, Deepak and Shoeybi, Mohammad and Casper, Jared and LeGresley, Patrick and Patwary, Mostofa and Korthikanti, Vijay and Vainbrand, Dmitri and Kashinkunti, Prethvi and Bernauer, Julie and Catanzaro, Bryan and Phanishayee, Amar and Zaharia, Matei , title =. Proceedings of the International Conference for High Performance Computing, Networking, Stor...

    work page doi:10.1145/3458817.3476209 2021

  71. [71]

    2025 62nd ACM/IEEE Design Automation Conference (DAC) , year=

    Scaling Laws of Graph Neural Networks for Atomistic Materials Modeling* , author=. 2025 62nd ACM/IEEE Design Automation Conference (DAC) , year=

    work page 2025

  72. [72]

    2025 , eprint=

    Scaling Laws of Graph Neural Networks for Atomistic Materials Modeling , author=. 2025 , eprint=

    work page 2025

  73. [73]

    Materials Horizons , year=

    Machine learning pipelines for the design of solid-state electrolytes , author=. Materials Horizons , year=

    work page

  74. [74]

    Nature , volume=

    Scaling deep learning for materials discovery , author=. Nature , volume=. 2023 , publisher=

    work page 2023

  75. [75]

    Journal of the American Chemical Society , volume=

    Mace-off: Short-range transferable machine learning force fields for organic molecules , author=. Journal of the American Chemical Society , volume=. 2025 , publisher=

    work page 2025

  76. [76]

    Journal of Medicinal Chemistry , volume=

    Innovative Medicinal Chemistry Strategies for Improving Target Binding Kinetics in Drug Discovery , author=. Journal of Medicinal Chemistry , volume=. 2025 , publisher=

    work page 2025

  77. [77]

    Proceedings of the National Academy of Sciences , volume=

    Following the dynamics of industrial catalysts under operando conditions , author=. Proceedings of the National Academy of Sciences , volume=. 2024 , publisher=

    work page 2024

  78. [78]

    Chemical Society Reviews , volume=

    Computational approach inspired advancements of solid-state electrolytes for lithium secondary batteries: from first-principles to machine learning , author=. Chemical Society Reviews , volume=. 2024 , publisher=

    work page 2024

  79. [79]

    Nature communications , volume=

    E (3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials , author=. Nature communications , volume=. 2022 , publisher=

    work page 2022

  80. [80]

    Forty-second International Conference on Machine Learning , year=

    PipeOffload: Improving Scalability of Pipeline Parallelism with Memory Optimization , author=. Forty-second International Conference on Machine Learning , year=

    work page

Showing first 80 references.