Pith. sign in

REVIEW 2 major objections 4 minor 1 cited by

The relativistic N-body code gevolution has been rewritten for GPU hardware, and the new version cuts the node-hours needed for a standard cosmology run by a factor of about 4.5 compared with the previous CPU-only release.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · deepseek-v4-flash

2026-08-01 16:34 UTC pith:77BUODCZ

load-bearing objection A solid, honest code-release paper: the GPU port is real, the benchmarks are transparent, and the main caveat is a CPU-baseline comparison that is fair but could be better documented. the 2 major comments →

arxiv 2607.17929 v1 pith:77BUODCZ submitted 2026-07-20 astro-ph.IM astro-ph.CO

gevolution 2.0: GPU-accelerated relativistic N-body simulations for cosmology

classification astro-ph.IM astro-ph.CO
keywords relativistic N-body simulationGPU accelerationCUDAparticle-mesh methodcosmological simulationslarge-scale structureparallel computingweak-field general relativity
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that a full rewrite of the parallelisation layer can make a relativistic particle-mesh N-body code competitive with state-of-the-art cosmological simulation tools on GPU supercomputers. The new gevolution 2.0 offloads all particle-mesh operations and most one-dimensional Fourier transforms to GPUs, adds shared-memory parallelisation on top of MPI, and replaces the old linked-list particle handler with a sorted-row, templated-kernel design. Benchmarks show near-perfect strong scaling for particle updates, good weak scaling for the FFT-dominated metric solvers, and a roughly 4.5-fold reduction in node-hours versus the CPU-only version on the same hardware. The code's matter power spectra agree with two established community N-body codes to within about 2% on large scales, suggesting the speed does not come at an obvious cost in accuracy for typical large-scale structure analyses.

Core claim

The central claim is that gevolution 2.0 provides a practical, publicly available route to fully relativistic N-body simulations on GPU hardware, without giving up the physics that made the original code distinctive. The implementation moves essentially all compute-intensive work—particle projection, kick, drift, and the one-dimensional FFTs inside the distributed three-dimensional transform—onto GPUs, keeps Fourier-space fields in host memory, and reorganises the particle ensemble into sorted rows so that thread blocks touch only a few mesh rows. The performance story is that the code spends most of its time in the 3D FFT and its MPI transpositions; particle updates take less than ten perce

What carries the argument

The load-bearing mechanism is the three-layer parallelisation: MPI ranks are bound one-to-one to GPUs, OpenMP threads fill the CPU cores of each rank, and CUDA kernels execute the particle-mesh operations. The distributed 3D FFT, which dominates runtime, is handled by running 1D FFTs in cuFFT on device with a fallback to CPU FFTW when data alignment does not satisfy cuFFT, while the MPI transpositions between 1D passes become the main communication cost. A rewritten particle handler sorts particles into row bins and uses CUDA's radix sort implementations so that each thread block processes a small number of mesh rows, making the projection, kick, and drift steps GPU-resident. The code keeps

Load-bearing premise

The ~4.5x speed-up over the CPU version assumes that using 96 MPI ranks per node (a network limit) is a fair stand-in for the CPU-only code's best performance; if the old code could have used the full 288 cores per node, the speed-up would be smaller.

What would settle it

Run the identical strong-scaling benchmark with the previous version of gevolution on the same machine using 288 MPI ranks per node (or otherwise removing the network restriction) and recompute the node-hour ratio; if the CPU-only runtime drops by enough to reduce the gain below ~4.5x, the headline speed-up claim is overturned. Alternatively, rerun the GPU code on a cluster with a different network topology and check whether the FFT transposition time still dominates, which would confirm or refute the communication-bound explanation.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Relativistic N-body simulations can now be run on affordable GPU machines, including consumer hardware, at resolutions that previously required large CPU clusters.
  • The public release provides a stable base for adding new relativistic fields, such as dynamical dark energy models, with confidence that the GPU layer will scale.
  • The FFT and communication bottleneck implies that on even larger GPU systems, further gains will come from reducing the cost of distributed transpositions or overlapping communication with computation.
  • The power-spectrum agreement within about two percent on large scales suggests uniform-mesh FFT codes can serve as fast, accurate forward models for galaxy surveys across that scale range.
  • The drop-in replacement of the particle handler, from function pointers to templated kernels, provides a template for other mesh-based simulation codes migrating to GPU hardware.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The reported 4.5x speed-up is computed relative to a CPU-only run using 96 of the 288 cores per node, a restriction imposed by the network configuration; if the CPU code could have used all cores, the gain would likely shrink. The authors' node-hour normalisation partially compensates, but the headline number is sensitive to this baseline.
  • Because distributed 3D FFT transpositions dominate runtime, the code's scaling on even larger GPU systems will be governed by network and GPU-direct communication rather than by arithmetic. A concrete test would be to run the same benchmark with a different transpose schedule or with computation/communication overlap and measure whether strong scaling improves beyond the current I/O-limited platea
  • The redesign from function-pointer callbacks to compile-time templated kernels is a transferable pattern for other mesh-based cosmology codes that want to become GPU-native without changing their physics.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. gevolution 2.0 is a GPU-accelerated rewrite of the relativistic N-body code gevolution. The paper describes three levels of parallelism (MPI, OpenMP, CUDA), a new particle handler, GPU-enabled mesh updates, and modifications to the LATfield2 3D FFT to use cuFFT with a CPU fallback when alignment requirements are not met. It reviews the relativistic field equations and the Newtonian limit, then presents benchmarks: strong scaling on a 1080^3 problem, weak scaling up to 2400^3, component-wise timing, a comparison with the CPU-only gevolution 1.3 reporting about a 4.5x node-hour speed-up, and a Newtonian comparison against PKDGRAV3 and Gadget-4 with matter power-spectrum cross-checks. The code, the GPU-ready LATfield2 library, and benchmark configuration/profiler data are publicly released.

Significance. The public release of a GPU-accelerated relativistic N-body code is a valuable community resource, and the design choices—thread-block/row-based particle sorting, device-side particle-mesh operations, and careful staging of FFT data—are technically well motivated. The benchmark methodology is transparent: configuration files and profiler reports are deposited on Zenodo, and the power-spectrum agreement with other codes provides a useful physical sanity check. The strong and weak scaling data, together with the component-by-component timings, are informative for users planning large runs. If the performance claims hold, gevolution 2.0 should be competitive for large cosmological simulations. The main quantitative claim, the ~4.5x speed-up over the CPU-only version, rests on a CPU baseline that is not fully validated, which tempers confidence in that specific number.

major comments (2)
  1. [Section 4.3, Eq. (19)] The ~4.5x speed-up claim depends on an unvalidated CPU baseline. The CPU run uses 3 nodes with 96 MPI ranks/node, while the GPU run uses 4 nodes with 16 MPI ranks. Equation (19) discounts CPU node hours by a node-utilisation factor of 1/3, equivalent to assuming gevolution 1.3 wall-time scales linearly with active cores. No CPU strong-scaling data are provided to support this linearity; particle-mesh codes typically show sub-linear MPI scaling. The claim that the two configurations have a 'roughly comparable' mix of inter- and intra-node communication is not substantiated given different node and rank counts. Please provide a CPU strong-scaling curve for gevolution 1.3 (at 96 ranks/node and ideally 288 ranks/node) or explicitly present the 4.5x factor as configuration-specific.
  2. [Section 4.4, Tables 2-4] The cross-code comparison inherits the same normalisation issue. Gadget-4 is run on 3 nodes with 96 MPI ranks per node and its node hours are discounted by the same 1/3 utilisation factor. The resulting large node-hour ratios are therefore subject to the same linear-scaling assumption about CPU performance, which is not demonstrated for any of the CPU runs. Since the conclusion that gevolution is 'very much competitive with the state of the art' relies in part on these ratios, the paper should either provide supporting CPU scaling data or soften the comparative claims to reflect the configuration-specific nature of the measurements.
minor comments (4)
  1. [Section 4.3] Please report the wall-clock time and node-hour values for the CPU and GPU runs in a small table or in the text. Currently the reader only sees the final factor and cannot gauge the absolute runtime difference.
  2. [Table 4] The note 'The run was executed on 3 nodes using 288 MPI ranks' is potentially confusing. Since the text says only 1/3 of CPUs are used, it is clearer to write '96 MPI ranks per node' for consistency with Section 4.3.
  3. [Figures 2 and 3] Figure 3's y-axis label is 'wall-time (s)', while Figure 2 uses 'speed-up'. It may help to explicitly state in the captions that the 'total' line in Figure 2 is speed-up relative to the single-node run, and that in Figure 3 constant wall-time indicates ideal weak scaling.
  4. [Data Availability] The Zenodo deposit is a good practice. For reproducibility, please also include the exact version tag or commit hash of gevolution 2.0 and LATfield2 used in the benchmarks, and the CUDA/compiler versions if they affect performance.

Circularity Check

0 steps flagged

No circular derivation; benchmarks are empirical and self-contained.

full rationale

The paper's central claims are about a GPU-accelerated code release and its measured performance, not about deriving a prediction from fitted inputs. The strong/weak scaling results and the ~4.5x speed-up are empirical benchmarks on specific hardware, with configuration files and profiler reports deposited at Zenodo. The physics equations in Section 3 are reviewed from prior work (including the authors' own earlier papers), but they are not re-derived to match the benchmarks, nor are any parameters fitted to make the benchmarks come out. The only near-concern is Section 4.3, where the CPU-only baseline uses 96 of 288 available cores per node and is normalized through Eq. (19) with a node-utilisation factor; this is an acknowledged benchmarking limitation that could affect the robustness of the speed-up number, but it is not a definitional identity or a fitted parameter renamed as a prediction. Self-citations to earlier gevolution papers describe the code's heritage and equation set; they are not load-bearing in the sense of forcing the reported performance results, which are independently measured. No circular step of the enumerated kinds is present.

Axiom & Free-Parameter Ledger

0 free parameters · 6 axioms · 0 invented entities

The paper's central claim is performance, not new physics. The equations are reviewed from earlier gevolution publications and involve standard weak-field cosmology assumptions. No new free parameters are fitted, and no new entities are introduced. The benchmarks depend on hardware and configuration choices, not on tuning physics parameters.

axioms (6)
  • domain assumption Weak-field conditions: linearization in the spin-2 amplitude h_ij and the frame-dragging potential B^i.
    Section 3.1 states 'weak-field conditions are crucial' and the code linearizes in h_ij and B^i. This is carried over from prior gevolution papers, not newly justified here.
  • domain assumption The conformal factor a is introduced for computational convenience and does not introduce background dependence for observables.
    Section 3.1 argues a is chosen to absorb a large smooth contribution to extrinsic curvature, with homogeneous modes of phi and psi adjusting accordingly. This is assumed from earlier work.
  • domain assumption The predictor-corrector loop for coupled phi and psi converges without iterations because their evolution is slow.
    Section 3.1: 'the evolution of phi and psi is sufficiently slow that the initial guess can be obtained from the previous time step and no actual loop iterations are required.' This is an assumption about the time evolution.
  • domain assumption Particle scattering on gravitational waves is negligible and can be omitted from the N-body updates.
    Section 3.1: 'The scattering of particles on gravitational waves is typically so small that it would be lost in numerical round-off errors.' This is used to exclude h_ij from particle kick/drift.
  • domain assumption Sub-horizon assumption (L H << 1) in the Newtonian limit reduces the Hamiltonian constraint to the Poisson equation.
    Section 3.2: 'This is justified under the additional assumption that we are deep inside the horizon, so that Δφ ≫ H∂τφ + H^2ψ.' This is a standard weak-field cosmological approximation.
  • domain assumption The N-body gauge requires a nearly time-independent large-scale potential H_T predicted from linear perturbation theory.
    Section 3.2 defines H_T and states it is 'a nearly time-independent large-scale potential that can be predicted from linear perturbation theory.' The code assumes this external input when switching gauges.

pith-pipeline@v1.3.0-alltime-deepseek · 15189 in / 10725 out tokens · 85691 ms · 2026-08-01T16:34:41.032308+00:00 · methodology

0 comments
read the original abstract

High-performance computing is increasingly dominated by hardware acceleration using Graphics Processing Units (GPUs). To take advantage of this long-term trend, we implement a major overhaul of the parallelisation approach in the relativistic particle-mesh N-body code gevolution. The new version 2.0 of gevolution employs three layers of parallelisation: MPI for scalability on a distributed memory system, shared memory parallelisation on each MPI rank using OpenMP, and offloading all compute intensive tasks to GPUs using CUDA. The code also includes many new features that have been developed over the past years. We provide an overview of the code structure and show key performance benchmarks. The public release of gevolution 2.0 can be found at https://github.com/gevolution-code/gevolution-2.0. We also release a GPU-ready version of the LATfield2 library which provides the parallelisation backend, available at https://github.com/gevolution-code/LATfield2.

Figures

Figures reproduced from arXiv: 2607.17929 by Julian Adamek, {\O}yvind Christiansen.

Figure 1
Figure 1. Figure 1: — Illustration of the domain decomposition in LATfield2. The ‘fast’ index of the arrays used to store 3D mesh quantities is aligned with the 𝑥 coordinate. while fields that are defined in Fourier space may reside in CPU (host) memory, i.e. regular dynamic random access memory (DRAM). In this way, we try to make optimal use of limited resources: memory is the primary limiting factor of large 𝑁- body simulat… view at source ↗
Figure 2
Figure 2. Figure 2: — Results of a strong scaling test. The black dashed line indicates the overall performance for a simulation with 10803 particles on a 10803 mesh. The different coloured lines show the scaling behaviour of individual code components as indicated in the legend. Scaling is mainly limited by I/O: on 81 nodes, the simulation runs in ∼ 93 s while producing 400 GB of data that are mostly written using MPI collec… view at source ↗
Figure 3
Figure 3. Figure 3: — Results of a weak scaling test. The black dashed line indicates the overall performance as the problem size increases proportionately to the number of nodes. The different coloured bands show the relative contribution and the scaling behaviour of individual code components as indicated in the legend. only. Specifically, we run the simulation used for the strong￾scaling analysis on three nodes with 96 MPI… view at source ↗
Figure 4
Figure 4. Figure 4: — Comparison of matter power spectra for Newtonian simulations run with different codes. The top panel refers to redshift 𝑧 = 1 and the bottom panel to redshift 𝑧 = 0.25. The rose-coloured lines show the relative differ￾ence between gevolution and Gadget-4, whereas the teal-coloured lines show the relative difference between gevolution and PKDGRAV3. Positive numbers mean that the result from gevolution is … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Signatures of kinetic gravity braiding in cosmological probes of the gravitational field

    gr-qc 2026-07 accept novelty 5.0

    KGB braiding boosts weak-lensing power by ~10–12% at ℓ~10²–10³ and flips the ISW–RS difference from suppression to excess once nonlinear Rees–Sciama physics dominates.

Reference graph

Works this paper leans on

28 extracted references · 24 linked inside Pith · cited by 1 Pith paper

  1. [1]

    General relativity and cosmic structure formation

    Adamek, Julian and Daverio, David and Durrer, Ruth and Kunz, Martin. General relativity and cosmic structure formation. Nature Phys. 2016. doi:10.1038/nphys3673. arXiv:1509.01699

  2. [2]

    Cosmic strings, domain walls and environment-dependent clustering

    Christiansen, yvind and Adamek, Julian and Kunz, Martin. Cosmic strings, domain walls and environment-dependent clustering. JCAP. 2026. doi:10.1088/1475-7516/2026/06/028. arXiv:2601.15234

  3. [3]

    gevolution: a cosmological N-body code based on General Relativity

    Adamek, Julian and Daverio, David and Durrer, Ruth and Kunz, Martin. gevolution: a cosmological N-body code based on General Relativity. JCAP. 2016. doi:10.1088/1475-7516/2016/07/053. arXiv:1604.06065

  4. [4]

    General Relativistic N -body simulations in the weak field limit

    Adamek, Julian and Daverio, David and Durrer, Ruth and Kunz, Martin. General Relativistic N -body simulations in the weak field limit. Phys. Rev. D. 2013. doi:10.1103/PhysRevD.88.103527. arXiv:1308.6524

  5. [5]

    Relativistic second-order initial conditions for simulations of large-scale structure

    Adamek, Julian and Calles, Juan and Montandon, Thomas and Nore \ n a, Jorge and Stahl, Cl \'e ment. Relativistic second-order initial conditions for simulations of large-scale structure. JCAP. 2022. doi:10.1088/1475-7516/2022/04/001. arXiv:2110.11249

  6. [6]

    Relativistic matter bispectrum of cosmic structures on the light cone

    Montandon, Thomas and Adamek, Julian and Hahn, Oliver and Nore \ n a, Jorge and Rampf, Cornelius and Stahl, Cl \'e ment and van Tent, Bartjan. Relativistic matter bispectrum of cosmic structures on the light cone. JCAP. 2023. doi:10.1088/1475-7516/2023/08/043. arXiv:2212.06799

  7. [7]

    Latfield2: A c++ library for classical lattice field theory

    Daverio, David and Hindmarsh, Mark and Bevis, Neil. Latfield2: A c++ library for classical lattice field theory. 2015. arXiv:1508.05610

  8. [8]

    The effect of early radiation in N-body simulations of cosmic structure formation

    Adamek, Julian and Brandbyge, Jacob and Fidler, Christian and Hannestad, Steen and Rampf, Cornelius and Tram, Thomas. The effect of early radiation in N-body simulations of cosmic structure formation. Mon. Not. Roy. Astron. Soc. 2017. doi:10.1093/mnras/stx1157. arXiv:1703.08585

  9. [9]

    Relativistic N-body simulations with massive neutrinos

    Adamek, Julian and Durrer, Ruth and Kunz, Martin. Relativistic N-body simulations with massive neutrinos. JCAP. 2017. doi:10.1088/1475-7516/2017/11/004. arXiv:1707.06938

  10. [10]

    and others

    Adamek, J. and others. Euclid: Modelling massive neutrinos in cosmology a code comparison. JCAP. 2023. doi:10.1088/1475-7516/2023/06/035. arXiv:2211.12457

  11. [11]

    Weak-lensing observables in relativistic N-body simulations

    Lepori, Francesca and Adamek, Julian and Durrer, Ruth and Clarkson, Chris and Coates, Louis. Weak-lensing observables in relativistic N-body simulations. Mon. Not. Roy. Astron. Soc. 2020. doi:10.1093/mnras/staa2024. arXiv:2002.04024

  12. [12]

    The Cosmic Linear Anisotropy Solving System (CLASS) II: Approximation schemes

    Blas, Diego and Lesgourgues, Julien and Tram, Thomas. The Cosmic Linear Anisotropy Solving System (CLASS) II: Approximation schemes. JCAP. 2011. doi:10.1088/1475-7516/2011/07/034. arXiv:1104.2933

  13. [13]

    and Wald, Robert M

    Green, Stephen R. and Wald, Robert M. Newtonian and Relativistic Cosmologies. Phys. Rev. D. 2012. doi:10.1103/PhysRevD.85.063512. arXiv:1111.2997

  14. [14]

    k -evolution: a relativistic N-body code for clustering dark energy

    Hassani, Farbod and Adamek, Julian and Kunz, Martin and Vernizzi, Filippo. k -evolution: a relativistic N-body code for clustering dark energy. JCAP. 2019. doi:10.1088/1475-7516/2019/12/011. arXiv:1910.01104

  15. [15]

    grgadget: an N-body TreePM relativistic code for cosmological simulations

    Quintana-Miranda, Eduardo and Monaco, Pierluigi and Tornatore, Luca. grgadget: an N-body TreePM relativistic code for cosmological simulations. Mon. Not. Roy. Astron. Soc. 2023. doi:10.1093/mnras/stad1174. arXiv:2301.11854

  16. [16]

    Simulating cosmic structure formation with the gadget-4 code

    Springel, Volker and Pakmor, R. Simulating cosmic structure formation with the gadget-4 code. Mon. Not. Roy. Astron. Soc. 2021. doi:10.1093/mnras/stab1855. arXiv:2010.03567

  17. [17]

    Asevolution: a relativistic N-body implementation of the (a)symmetron

    Christiansen, yvind and Hassani, Farbod and Jalilvand, Mona and Mota, David F. Asevolution: a relativistic N-body implementation of the (a)symmetron. JCAP. 2023. doi:10.1088/1475-7516/2023/05/009. arXiv:2302.07857

  18. [18]

    Gravitational waves from dark domain walls

    Christiansen, yvind and Adamek, Julian and Hassani, Farbod and Mota, David. Gravitational waves from dark domain walls. JCAP. 2025. doi:10.1088/1475-7516/2025/01/149. arXiv:2401.02409

  19. [19]

    KGB-evolution: a relativistic N -body code for kinetic gravity braiding models

    Nouri-Zonoz, Ahmad and Hassani, Farbod and Bellini, Emilio and Kunz, Martin. KGB-evolution: a relativistic N -body code for kinetic gravity braiding models. 2025. arXiv:2511.04676

  20. [20]

    The foundation of the general theory of relativity

    Einstein, Albert. The foundation of the general theory of relativity. Annalen Phys. 1916. doi:10.1002/andp.19163540702

  21. [21]

    General relativistic corrections to N -body simulations and the Zel'dovich approximation

    Fidler, Christian and Rampf, Cornelius and Tram, Thomas and Crittenden, Robert and Koyama, Kazuya and Wands, David. General relativistic corrections to N -body simulations and the Zel'dovich approximation. Phys. Rev. D. 2015. doi:10.1103/PhysRevD.92.123517. arXiv:1505.04756

  22. [22]

    General relativistic weak-field limit and Newtonian N-body simulations

    Fidler, Christian and Tram, Thomas and Rampf, Cornelius and Crittenden, Robert and Koyama, Kazuya and Wands, David. General relativistic weak-field limit and Newtonian N-body simulations. JCAP. 2017. doi:10.1088/1475-7516/2017/12/022. arXiv:1708.07769

  23. [23]

    Fast simulations of cosmic large-scale structure with massive neutrinos

    Partmann, Christian and Fidler, Christian and Rampf, Cornelius and Hahn, Oliver. Fast simulations of cosmic large-scale structure with massive neutrinos. JCAP. 2020. doi:10.1088/1475-7516/2020/09/018. arXiv:2003.07387

  24. [24]

    Effective Field Theory of Large Scale Structure and Newtonian Motion Gauges

    Fidler, Christian and Lesgourgues, Julien and Mattes, Antonia and Moradinezhad Dizgah, Azadeh and Neuland, Simon. Effective Field Theory of Large Scale Structure and Newtonian Motion Gauges. 2026. arXiv:2605.05114

  25. [25]

    Perturbed redshifts from N-body simulations

    Adamek, Julian. Perturbed redshifts from N-body simulations. Phys. Rev. D. 2018. doi:10.1103/PhysRevD.97.021302. arXiv:1708.07552

  26. [26]

    The large-scale general-relativistic correction for Newtonian mocks

    Adamek, Julian and Fidler, Christian. The large-scale general-relativistic correction for Newtonian mocks. JCAP. 2019. doi:10.1088/1475-7516/2019/09/026. arXiv:1905.11721

  27. [27]

    PKDGRAV3: beyond trillion particle cosmological simulations for the next era of galaxy surveys

    Potter, Douglas and Stadel, Joachim and Teyssier, Romain. PKDGRAV3: beyond trillion particle cosmological simulations for the next era of galaxy surveys. Comput. Astrophys. Cosmol. 2017. doi:10.1186/s40668-017-0021-1. arXiv:1609.08621

  28. [28]

    Accurate initial conditions for cosmological N-body simulations: Minimizing truncation and discreteness errors

    Michaux, Micha. Accurate initial conditions for cosmological N-body simulations: Minimizing truncation and discreteness errors. Mon. Not. Roy. Astron. Soc. 2020. doi:10.1093/mnras/staa3149. arXiv:2008.09588