REVIEW 2 major objections 4 minor 1 cited by
The relativistic N-body code gevolution has been rewritten for GPU hardware, and the new version cuts the node-hours needed for a standard cosmology run by a factor of about 4.5 compared with the previous CPU-only release.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 16:34 UTC pith:77BUODCZ
load-bearing objection A solid, honest code-release paper: the GPU port is real, the benchmarks are transparent, and the main caveat is a CPU-baseline comparison that is fair but could be better documented. the 2 major comments →
gevolution 2.0: GPU-accelerated relativistic N-body simulations for cosmology
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that gevolution 2.0 provides a practical, publicly available route to fully relativistic N-body simulations on GPU hardware, without giving up the physics that made the original code distinctive. The implementation moves essentially all compute-intensive work—particle projection, kick, drift, and the one-dimensional FFTs inside the distributed three-dimensional transform—onto GPUs, keeps Fourier-space fields in host memory, and reorganises the particle ensemble into sorted rows so that thread blocks touch only a few mesh rows. The performance story is that the code spends most of its time in the 3D FFT and its MPI transpositions; particle updates take less than ten perce
What carries the argument
The load-bearing mechanism is the three-layer parallelisation: MPI ranks are bound one-to-one to GPUs, OpenMP threads fill the CPU cores of each rank, and CUDA kernels execute the particle-mesh operations. The distributed 3D FFT, which dominates runtime, is handled by running 1D FFTs in cuFFT on device with a fallback to CPU FFTW when data alignment does not satisfy cuFFT, while the MPI transpositions between 1D passes become the main communication cost. A rewritten particle handler sorts particles into row bins and uses CUDA's radix sort implementations so that each thread block processes a small number of mesh rows, making the projection, kick, and drift steps GPU-resident. The code keeps
Load-bearing premise
The ~4.5x speed-up over the CPU version assumes that using 96 MPI ranks per node (a network limit) is a fair stand-in for the CPU-only code's best performance; if the old code could have used the full 288 cores per node, the speed-up would be smaller.
What would settle it
Run the identical strong-scaling benchmark with the previous version of gevolution on the same machine using 288 MPI ranks per node (or otherwise removing the network restriction) and recompute the node-hour ratio; if the CPU-only runtime drops by enough to reduce the gain below ~4.5x, the headline speed-up claim is overturned. Alternatively, rerun the GPU code on a cluster with a different network topology and check whether the FFT transposition time still dominates, which would confirm or refute the communication-bound explanation.
If this is right
- Relativistic N-body simulations can now be run on affordable GPU machines, including consumer hardware, at resolutions that previously required large CPU clusters.
- The public release provides a stable base for adding new relativistic fields, such as dynamical dark energy models, with confidence that the GPU layer will scale.
- The FFT and communication bottleneck implies that on even larger GPU systems, further gains will come from reducing the cost of distributed transpositions or overlapping communication with computation.
- The power-spectrum agreement within about two percent on large scales suggests uniform-mesh FFT codes can serve as fast, accurate forward models for galaxy surveys across that scale range.
- The drop-in replacement of the particle handler, from function pointers to templated kernels, provides a template for other mesh-based simulation codes migrating to GPU hardware.
Where Pith is reading between the lines
- The reported 4.5x speed-up is computed relative to a CPU-only run using 96 of the 288 cores per node, a restriction imposed by the network configuration; if the CPU code could have used all cores, the gain would likely shrink. The authors' node-hour normalisation partially compensates, but the headline number is sensitive to this baseline.
- Because distributed 3D FFT transpositions dominate runtime, the code's scaling on even larger GPU systems will be governed by network and GPU-direct communication rather than by arithmetic. A concrete test would be to run the same benchmark with a different transpose schedule or with computation/communication overlap and measure whether strong scaling improves beyond the current I/O-limited platea
- The redesign from function-pointer callbacks to compile-time templated kernels is a transferable pattern for other mesh-based cosmology codes that want to become GPU-native without changing their physics.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. gevolution 2.0 is a GPU-accelerated rewrite of the relativistic N-body code gevolution. The paper describes three levels of parallelism (MPI, OpenMP, CUDA), a new particle handler, GPU-enabled mesh updates, and modifications to the LATfield2 3D FFT to use cuFFT with a CPU fallback when alignment requirements are not met. It reviews the relativistic field equations and the Newtonian limit, then presents benchmarks: strong scaling on a 1080^3 problem, weak scaling up to 2400^3, component-wise timing, a comparison with the CPU-only gevolution 1.3 reporting about a 4.5x node-hour speed-up, and a Newtonian comparison against PKDGRAV3 and Gadget-4 with matter power-spectrum cross-checks. The code, the GPU-ready LATfield2 library, and benchmark configuration/profiler data are publicly released.
Significance. The public release of a GPU-accelerated relativistic N-body code is a valuable community resource, and the design choices—thread-block/row-based particle sorting, device-side particle-mesh operations, and careful staging of FFT data—are technically well motivated. The benchmark methodology is transparent: configuration files and profiler reports are deposited on Zenodo, and the power-spectrum agreement with other codes provides a useful physical sanity check. The strong and weak scaling data, together with the component-by-component timings, are informative for users planning large runs. If the performance claims hold, gevolution 2.0 should be competitive for large cosmological simulations. The main quantitative claim, the ~4.5x speed-up over the CPU-only version, rests on a CPU baseline that is not fully validated, which tempers confidence in that specific number.
major comments (2)
- [Section 4.3, Eq. (19)] The ~4.5x speed-up claim depends on an unvalidated CPU baseline. The CPU run uses 3 nodes with 96 MPI ranks/node, while the GPU run uses 4 nodes with 16 MPI ranks. Equation (19) discounts CPU node hours by a node-utilisation factor of 1/3, equivalent to assuming gevolution 1.3 wall-time scales linearly with active cores. No CPU strong-scaling data are provided to support this linearity; particle-mesh codes typically show sub-linear MPI scaling. The claim that the two configurations have a 'roughly comparable' mix of inter- and intra-node communication is not substantiated given different node and rank counts. Please provide a CPU strong-scaling curve for gevolution 1.3 (at 96 ranks/node and ideally 288 ranks/node) or explicitly present the 4.5x factor as configuration-specific.
- [Section 4.4, Tables 2-4] The cross-code comparison inherits the same normalisation issue. Gadget-4 is run on 3 nodes with 96 MPI ranks per node and its node hours are discounted by the same 1/3 utilisation factor. The resulting large node-hour ratios are therefore subject to the same linear-scaling assumption about CPU performance, which is not demonstrated for any of the CPU runs. Since the conclusion that gevolution is 'very much competitive with the state of the art' relies in part on these ratios, the paper should either provide supporting CPU scaling data or soften the comparative claims to reflect the configuration-specific nature of the measurements.
minor comments (4)
- [Section 4.3] Please report the wall-clock time and node-hour values for the CPU and GPU runs in a small table or in the text. Currently the reader only sees the final factor and cannot gauge the absolute runtime difference.
- [Table 4] The note 'The run was executed on 3 nodes using 288 MPI ranks' is potentially confusing. Since the text says only 1/3 of CPUs are used, it is clearer to write '96 MPI ranks per node' for consistency with Section 4.3.
- [Figures 2 and 3] Figure 3's y-axis label is 'wall-time (s)', while Figure 2 uses 'speed-up'. It may help to explicitly state in the captions that the 'total' line in Figure 2 is speed-up relative to the single-node run, and that in Figure 3 constant wall-time indicates ideal weak scaling.
- [Data Availability] The Zenodo deposit is a good practice. For reproducibility, please also include the exact version tag or commit hash of gevolution 2.0 and LATfield2 used in the benchmarks, and the CUDA/compiler versions if they affect performance.
Circularity Check
No circular derivation; benchmarks are empirical and self-contained.
full rationale
The paper's central claims are about a GPU-accelerated code release and its measured performance, not about deriving a prediction from fitted inputs. The strong/weak scaling results and the ~4.5x speed-up are empirical benchmarks on specific hardware, with configuration files and profiler reports deposited at Zenodo. The physics equations in Section 3 are reviewed from prior work (including the authors' own earlier papers), but they are not re-derived to match the benchmarks, nor are any parameters fitted to make the benchmarks come out. The only near-concern is Section 4.3, where the CPU-only baseline uses 96 of 288 available cores per node and is normalized through Eq. (19) with a node-utilisation factor; this is an acknowledged benchmarking limitation that could affect the robustness of the speed-up number, but it is not a definitional identity or a fitted parameter renamed as a prediction. Self-citations to earlier gevolution papers describe the code's heritage and equation set; they are not load-bearing in the sense of forcing the reported performance results, which are independently measured. No circular step of the enumerated kinds is present.
Axiom & Free-Parameter Ledger
axioms (6)
- domain assumption Weak-field conditions: linearization in the spin-2 amplitude h_ij and the frame-dragging potential B^i.
- domain assumption The conformal factor a is introduced for computational convenience and does not introduce background dependence for observables.
- domain assumption The predictor-corrector loop for coupled phi and psi converges without iterations because their evolution is slow.
- domain assumption Particle scattering on gravitational waves is negligible and can be omitted from the N-body updates.
- domain assumption Sub-horizon assumption (L H << 1) in the Newtonian limit reduces the Hamiltonian constraint to the Poisson equation.
- domain assumption The N-body gauge requires a nearly time-independent large-scale potential H_T predicted from linear perturbation theory.
read the original abstract
High-performance computing is increasingly dominated by hardware acceleration using Graphics Processing Units (GPUs). To take advantage of this long-term trend, we implement a major overhaul of the parallelisation approach in the relativistic particle-mesh N-body code gevolution. The new version 2.0 of gevolution employs three layers of parallelisation: MPI for scalability on a distributed memory system, shared memory parallelisation on each MPI rank using OpenMP, and offloading all compute intensive tasks to GPUs using CUDA. The code also includes many new features that have been developed over the past years. We provide an overview of the code structure and show key performance benchmarks. The public release of gevolution 2.0 can be found at https://github.com/gevolution-code/gevolution-2.0. We also release a GPU-ready version of the LATfield2 library which provides the parallelisation backend, available at https://github.com/gevolution-code/LATfield2.
Figures
Forward citations
Cited by 1 Pith paper
-
Signatures of kinetic gravity braiding in cosmological probes of the gravitational field
KGB braiding boosts weak-lensing power by ~10–12% at ℓ~10²–10³ and flips the ISW–RS difference from suppression to excess once nonlinear Rees–Sciama physics dominates.
Reference graph
Works this paper leans on
-
[1]
General relativity and cosmic structure formation
Adamek, Julian and Daverio, David and Durrer, Ruth and Kunz, Martin. General relativity and cosmic structure formation. Nature Phys. 2016. doi:10.1038/nphys3673. arXiv:1509.01699
Pith/arXiv arXiv 2016
-
[2]
Cosmic strings, domain walls and environment-dependent clustering
Christiansen, yvind and Adamek, Julian and Kunz, Martin. Cosmic strings, domain walls and environment-dependent clustering. JCAP. 2026. doi:10.1088/1475-7516/2026/06/028. arXiv:2601.15234
arXiv 2026
-
[3]
gevolution: a cosmological N-body code based on General Relativity
Adamek, Julian and Daverio, David and Durrer, Ruth and Kunz, Martin. gevolution: a cosmological N-body code based on General Relativity. JCAP. 2016. doi:10.1088/1475-7516/2016/07/053. arXiv:1604.06065
Pith/arXiv arXiv 2016
-
[4]
General Relativistic N -body simulations in the weak field limit
Adamek, Julian and Daverio, David and Durrer, Ruth and Kunz, Martin. General Relativistic N -body simulations in the weak field limit. Phys. Rev. D. 2013. doi:10.1103/PhysRevD.88.103527. arXiv:1308.6524
Pith/arXiv arXiv 2013
-
[5]
Relativistic second-order initial conditions for simulations of large-scale structure
Adamek, Julian and Calles, Juan and Montandon, Thomas and Nore \ n a, Jorge and Stahl, Cl \'e ment. Relativistic second-order initial conditions for simulations of large-scale structure. JCAP. 2022. doi:10.1088/1475-7516/2022/04/001. arXiv:2110.11249
Pith/arXiv arXiv 2022
-
[6]
Relativistic matter bispectrum of cosmic structures on the light cone
Montandon, Thomas and Adamek, Julian and Hahn, Oliver and Nore \ n a, Jorge and Rampf, Cornelius and Stahl, Cl \'e ment and van Tent, Bartjan. Relativistic matter bispectrum of cosmic structures on the light cone. JCAP. 2023. doi:10.1088/1475-7516/2023/08/043. arXiv:2212.06799
Pith/arXiv arXiv 2023
-
[7]
Latfield2: A c++ library for classical lattice field theory
Daverio, David and Hindmarsh, Mark and Bevis, Neil. Latfield2: A c++ library for classical lattice field theory. 2015. arXiv:1508.05610
Pith/arXiv arXiv 2015
-
[8]
The effect of early radiation in N-body simulations of cosmic structure formation
Adamek, Julian and Brandbyge, Jacob and Fidler, Christian and Hannestad, Steen and Rampf, Cornelius and Tram, Thomas. The effect of early radiation in N-body simulations of cosmic structure formation. Mon. Not. Roy. Astron. Soc. 2017. doi:10.1093/mnras/stx1157. arXiv:1703.08585
Pith/arXiv arXiv 2017
-
[9]
Relativistic N-body simulations with massive neutrinos
Adamek, Julian and Durrer, Ruth and Kunz, Martin. Relativistic N-body simulations with massive neutrinos. JCAP. 2017. doi:10.1088/1475-7516/2017/11/004. arXiv:1707.06938
Pith/arXiv arXiv 2017
-
[10]
Adamek, J. and others. Euclid: Modelling massive neutrinos in cosmology a code comparison. JCAP. 2023. doi:10.1088/1475-7516/2023/06/035. arXiv:2211.12457
Pith/arXiv arXiv 2023
-
[11]
Weak-lensing observables in relativistic N-body simulations
Lepori, Francesca and Adamek, Julian and Durrer, Ruth and Clarkson, Chris and Coates, Louis. Weak-lensing observables in relativistic N-body simulations. Mon. Not. Roy. Astron. Soc. 2020. doi:10.1093/mnras/staa2024. arXiv:2002.04024
Pith/arXiv arXiv 2020
-
[12]
The Cosmic Linear Anisotropy Solving System (CLASS) II: Approximation schemes
Blas, Diego and Lesgourgues, Julien and Tram, Thomas. The Cosmic Linear Anisotropy Solving System (CLASS) II: Approximation schemes. JCAP. 2011. doi:10.1088/1475-7516/2011/07/034. arXiv:1104.2933
Pith/arXiv arXiv 2011
-
[13]
Green, Stephen R. and Wald, Robert M. Newtonian and Relativistic Cosmologies. Phys. Rev. D. 2012. doi:10.1103/PhysRevD.85.063512. arXiv:1111.2997
Pith/arXiv arXiv 2012
-
[14]
k -evolution: a relativistic N-body code for clustering dark energy
Hassani, Farbod and Adamek, Julian and Kunz, Martin and Vernizzi, Filippo. k -evolution: a relativistic N-body code for clustering dark energy. JCAP. 2019. doi:10.1088/1475-7516/2019/12/011. arXiv:1910.01104
Pith/arXiv arXiv 2019
-
[15]
grgadget: an N-body TreePM relativistic code for cosmological simulations
Quintana-Miranda, Eduardo and Monaco, Pierluigi and Tornatore, Luca. grgadget: an N-body TreePM relativistic code for cosmological simulations. Mon. Not. Roy. Astron. Soc. 2023. doi:10.1093/mnras/stad1174. arXiv:2301.11854
Pith/arXiv arXiv 2023
-
[16]
Simulating cosmic structure formation with the gadget-4 code
Springel, Volker and Pakmor, R. Simulating cosmic structure formation with the gadget-4 code. Mon. Not. Roy. Astron. Soc. 2021. doi:10.1093/mnras/stab1855. arXiv:2010.03567
Pith/arXiv arXiv 2021
-
[17]
Asevolution: a relativistic N-body implementation of the (a)symmetron
Christiansen, yvind and Hassani, Farbod and Jalilvand, Mona and Mota, David F. Asevolution: a relativistic N-body implementation of the (a)symmetron. JCAP. 2023. doi:10.1088/1475-7516/2023/05/009. arXiv:2302.07857
Pith/arXiv arXiv 2023
-
[18]
Gravitational waves from dark domain walls
Christiansen, yvind and Adamek, Julian and Hassani, Farbod and Mota, David. Gravitational waves from dark domain walls. JCAP. 2025. doi:10.1088/1475-7516/2025/01/149. arXiv:2401.02409
arXiv 2025
-
[19]
KGB-evolution: a relativistic N -body code for kinetic gravity braiding models
Nouri-Zonoz, Ahmad and Hassani, Farbod and Bellini, Emilio and Kunz, Martin. KGB-evolution: a relativistic N -body code for kinetic gravity braiding models. 2025. arXiv:2511.04676
arXiv 2025
-
[20]
The foundation of the general theory of relativity
Einstein, Albert. The foundation of the general theory of relativity. Annalen Phys. 1916. doi:10.1002/andp.19163540702
-
[21]
General relativistic corrections to N -body simulations and the Zel'dovich approximation
Fidler, Christian and Rampf, Cornelius and Tram, Thomas and Crittenden, Robert and Koyama, Kazuya and Wands, David. General relativistic corrections to N -body simulations and the Zel'dovich approximation. Phys. Rev. D. 2015. doi:10.1103/PhysRevD.92.123517. arXiv:1505.04756
Pith/arXiv arXiv 2015
-
[22]
General relativistic weak-field limit and Newtonian N-body simulations
Fidler, Christian and Tram, Thomas and Rampf, Cornelius and Crittenden, Robert and Koyama, Kazuya and Wands, David. General relativistic weak-field limit and Newtonian N-body simulations. JCAP. 2017. doi:10.1088/1475-7516/2017/12/022. arXiv:1708.07769
Pith/arXiv arXiv 2017
-
[23]
Fast simulations of cosmic large-scale structure with massive neutrinos
Partmann, Christian and Fidler, Christian and Rampf, Cornelius and Hahn, Oliver. Fast simulations of cosmic large-scale structure with massive neutrinos. JCAP. 2020. doi:10.1088/1475-7516/2020/09/018. arXiv:2003.07387
Pith/arXiv arXiv 2020
-
[24]
Effective Field Theory of Large Scale Structure and Newtonian Motion Gauges
Fidler, Christian and Lesgourgues, Julien and Mattes, Antonia and Moradinezhad Dizgah, Azadeh and Neuland, Simon. Effective Field Theory of Large Scale Structure and Newtonian Motion Gauges. 2026. arXiv:2605.05114
Pith/arXiv arXiv 2026
-
[25]
Perturbed redshifts from N-body simulations
Adamek, Julian. Perturbed redshifts from N-body simulations. Phys. Rev. D. 2018. doi:10.1103/PhysRevD.97.021302. arXiv:1708.07552
Pith/arXiv arXiv 2018
-
[26]
The large-scale general-relativistic correction for Newtonian mocks
Adamek, Julian and Fidler, Christian. The large-scale general-relativistic correction for Newtonian mocks. JCAP. 2019. doi:10.1088/1475-7516/2019/09/026. arXiv:1905.11721
Pith/arXiv arXiv 2019
-
[27]
PKDGRAV3: beyond trillion particle cosmological simulations for the next era of galaxy surveys
Potter, Douglas and Stadel, Joachim and Teyssier, Romain. PKDGRAV3: beyond trillion particle cosmological simulations for the next era of galaxy surveys. Comput. Astrophys. Cosmol. 2017. doi:10.1186/s40668-017-0021-1. arXiv:1609.08621
Pith/arXiv arXiv 2017
-
[28]
Michaux, Micha. Accurate initial conditions for cosmological N-body simulations: Minimizing truncation and discreteness errors. Mon. Not. Roy. Astron. Soc. 2020. doi:10.1093/mnras/staa3149. arXiv:2008.09588
Pith/arXiv arXiv 2020
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.