pith. sign in

arxiv: 2509.08971 · v1 · pith:CZQKPRVBnew · submitted 2025-09-10 · ⚛️ physics.comp-ph · astro-ph.IM· cs.DC

HARD: A Performance Portable Radiation Hydrodynamics Code based on FleCSI Framework

Julien Loiseau , Hyun Lim , Andr\'es Yag\"ue L\'opez , Mammadbaghir Baghirzade , Shihab Shahriar Khan , Yoonsoo Kim , Sudarshan Neopane , Alexander Strack
show 2 more authors
Farhana Taiyebah Benjamin K. Bergen
This is my paper

Pith reviewed 2026-05-21 22:38 UTC · model grok-4.3

classification ⚛️ physics.comp-ph astro-ph.IMcs.DC
keywords radiation hydrodynamicsperformance portabilitytask-based parallelismFleCSIKokkoshigh-performance computingverification tests
0
0 comments X

The pith

HARD delivers a single code base for radiation hydrodynamics that runs efficiently on laptops through the largest heterogeneous supercomputers.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces HARD as an open-source application for compressible hydrodynamics with radiation-diffusion coupling. It builds the code on the FleCSI framework so that computational units are expressed as tasks whose execution can be handed to multiple back-end runtimes including Legion, MPI, and HPX. Node-level work is assigned to Kokkos, keeping the source unchanged across platforms. A built-in regression suite reproduces standard verification problems such as the Sod shock tube and Sedov blast wave to confirm numerical accuracy. The design therefore claims to supply performance portability without platform-specific rewrites.

Core claim

HARD expresses its computational units as tasks whose execution can be orchestrated by multiple back-end runtimes, including Legion, MPI, and HPX, while node-level parallelism is delegated to Kokkos, providing a single, portable code base that runs efficiently on laptops, small homogeneous clusters, and the largest heterogeneous supercomputers currently available.

What carries the argument

FleCSI task-based framework combined with Kokkos, which lets the same source express computations that multiple runtimes can schedule across different hardware.

Load-bearing premise

Delegating execution to multiple back-end runtimes and Kokkos will preserve numerical accuracy and performance portability without forcing platform-specific changes that would break the single-code-base claim.

What would settle it

Compile and run the unchanged HARD source on a new heterogeneous supercomputer, execute the Sedov blast-wave test, and check whether the numerical solution still matches the known analytical result while achieving competitive wall-clock time without any code edits.

Figures

Figures reproduced from arXiv: 2509.08971 by Alexander Strack, Andr\'es Yag\"ue L\'opez, Benjamin K. Bergen, Farhana Taiyebah, Hyun Lim, Julien Loiseau, Mammadbaghir Baghirzade, Shihab Shahriar Khan, Sudarshan Neopane, Yoonsoo Kim.

Figure 1
Figure 1. Figure 1: HARD within the FleCSI ecosystem. 2.2. Software functionalities HARD provides a set of core capabilities for solving RHD problems on mod￾ern HPC systems. At a high level, these include: • implementation of the coupled hydrodynamics and radiation–diffusion equations, • modular numerical algorithms (finite-volume solvers, reconstruction methods, Riemann solvers, and implicit diffusion solvers), • verificatio… view at source ↗
Figure 2
Figure 2. Figure 2: A v-cycle, the most basic structure of a GMG solver. Every circle represents a [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Numerical and analytical solution to the Sod problem at simulation time [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Tests of the approach to radiative equilibrium in a radiation dominated gas. [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: The Temperature induced shock test problem with different initial temperatures [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Density plot of Kelvin-Helmholtz instability with (right) and without (left) [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Convergence order for an acoustic wave problem with weno5z as limiter [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Performance of the 3D radiation benchmark on Chicoma. The notation (128/1) [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗
read the original abstract

Hydrodynamics And Radiation Diffusion} (HARD) is an open-source application for high-performance simulations of compressible hydrodynamics with radiation-diffusion coupling. Built on the FleCSI (Flexible Computational Science Infrastructure) framework, HARD expresses its computational units as tasks whose execution can be orchestrated by multiple back-end runtimes, including Legion, MPI, and HPX. Node-level parallelism is delegated to Kokkos, providing a single, portable code base that runs efficiently on laptops, small homogeneous clusters, and the largest heterogeneous supercomputers currently available. To ensure scientific reliability, HARD includes a regression-test suite that automatically reproduces canonical verification problems such as the Sod and LeBlanc shock tubes and the Sedov blast wave, comparing numerical solutions against known analytical results. The project is distributed under an OSI-approved license, hosted on GitHub, and accompanied by reproducible build scripts and continuous integration workflows. This combination of performance portability, verification infrastructure, and community-focused development makes HARD a sustainable platform for advancing radiation hydrodynamics research across multiple domains.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 3 minor

Summary. The manuscript describes HARD, an open-source code for compressible hydrodynamics with radiation-diffusion coupling, implemented on the FleCSI framework. Computational units are expressed as tasks orchestrated by multiple back-end runtimes (Legion, MPI, HPX) with node-level parallelism via Kokkos, enabling a single portable source base. Verification relies on an automated regression suite reproducing the Sod and LeBlanc shock tubes and Sedov blast wave, with direct comparisons to analytical solutions; the project includes reproducible build scripts, CI workflows, and an OSI-approved license.

Significance. If the portability claims hold, the work supplies a verified, maintainable platform for radiation-hydrodynamics simulations that can target laptops through heterogeneous supercomputers from one code base. The combination of task abstraction, Kokkos delegation, and built-in regression tests against analytical results strengthens its utility for domains such as astrophysics and high-energy-density physics, while the open distribution and CI lower barriers to adoption and extension.

major comments (1)
  1. [Performance and portability discussion (likely §4 or §5)] The central portability claim (single source without platform-specific tuning) rests on the assumption that FleCSI task orchestration plus Kokkos preserves both numerical fidelity and performance across Legion/MPI/HPX and heterogeneous nodes. The manuscript provides regression results for the Sod, LeBlanc, and Sedov problems but does not report quantitative performance or scaling data on heterogeneous supercomputers; this gap is load-bearing for the efficiency assertion in the abstract and introduction.
minor comments (3)
  1. [Abstract] Abstract contains a stray closing brace: 'Hydrodynamics And Radiation Diffusion} (HARD)'.
  2. [Verification / Regression tests] The verification section would benefit from explicit statement of the spatial discretization, time-stepping scheme, and radiation solver (e.g., diffusion approximation details) to allow readers to assess the numerical methods independently of the portability layer.
  3. [Figures] Figure captions and axis labels for any performance plots should include hardware details (node count, GPU/CPU configuration) to make scaling claims immediately interpretable.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the positive summary, significance assessment, and recommendation of minor revision. We address the single major comment below and will incorporate changes to strengthen the manuscript.

read point-by-point responses

  1. Referee: The central portability claim (single source without platform-specific tuning) rests on the assumption that FleCSI task orchestration plus Kokkos preserves both numerical fidelity and performance across Legion/MPI/HPX and heterogeneous nodes. The manuscript provides regression results for the Sod, LeBlanc, and Sedov problems but does not report quantitative performance or scaling data on heterogeneous supercomputers; this gap is load-bearing for the efficiency assertion in the abstract and introduction.

    Authors: We agree that the absence of quantitative performance and scaling results on heterogeneous systems leaves the efficiency claims in the abstract and introduction less fully supported than the portability and verification aspects. The manuscript's primary emphasis is on the task-based formulation, radiation-diffusion coupling, and automated regression against analytical solutions to demonstrate correctness and reproducibility. Performance portability is inherited from the FleCSI back-ends and Kokkos delegation, which are documented elsewhere to maintain efficiency without source changes. To directly address the referee's concern, we will add a concise performance subsection (likely in §4) that reports wall-clock timings and weak-scaling behavior for the Sedov test on a heterogeneous platform using the Legion backend with Kokkos GPU execution. These data will be obtained from additional runs on an available system and will be accompanied by the corresponding input decks in the repository to maintain reproducibility. revision: yes

Circularity Check

0 steps flagged

No significant circularity

full rationale

The manuscript is a software description paper presenting the HARD radiation-hydrodynamics code, its task-based abstraction over FleCSI, and its use of Kokkos for node-level parallelism. No derivation chain, equations, fitted parameters, or predictive claims exist that could reduce to self-definitions, fitted inputs, or self-citation load-bearing steps. Verification relies on external analytical solutions for Sod, LeBlanc, and Sedov problems plus reproducible build/CI infrastructure, keeping the presentation self-contained and independent of any internal circular reduction.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

This is a software-engineering paper describing an implementation; the abstract introduces no free parameters, mathematical axioms, or new physical entities.

pith-pipeline@v0.9.0 · 5756 in / 1185 out tokens · 62024 ms · 2026-05-21T22:38:13.920857+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

28 extracted references · 28 canonical work pages · 2 internal anchors

  1. [1]

    Bergen, I

    B. Bergen, I. Demeshko, C. Ferenbaugh, D. Herring, L.-T. Lo, J. Loiseau, N. Ray, A. Reisner, Flecsi 2.0: The flexible computational science infrastructure project, in: European Conference on Parallel Pro- cessing, Springer, 2021, pp. 480–495

    work page 2021

  2. [2]

    Bauer, S

    M. Bauer, S. Treichler, E. Slaughter, A. Aiken, Legion: expressing local- ity and independence with logical regions, in: SC’12: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, IEEE, 2012, pp. 1–11

    work page 2012

  3. [3]

    M. P. I. Forum, Mpi: A message-passing interface standard, Tech. rep., USA (1994)

    work page 1994

  4. [4]

    Kaiser, P

    H. Kaiser, P. Diehl, A. S. Lemoine, B. A. Lelbach, P. Amini, A. Berge, J. Biddiscombe, S. R. Brandt, N. Gupta, T. Heller, K. Huck, Z. Khatami, A. Kheirkhahan, A. Reverdell, S. Shirzad, M. Simberg, B. Wagle, W. Wei, T. Zhang, Hpx - the c++ standard library for paral- lelism and concurrency, Journal of Open Source Software 5 (53) (2020) 2352.doi:10.21105/jo...

    work page doi:10.21105/joss.02352 2020

  5. [5]

    H. C. Edwards, C. R. Trott, D. Sunderland, Kokkos: enabling manycore performance portability through polymorphic memory access patterns, Journal of Parallel and Distributed Computing 74 (12) (2014) 3202– 3216. xvi

    work page 2014

  6. [6]

    N. J. Turner, J. M. Stone, A module for radiation hydrodynamic cal- culations with zeus-2d using flux-limited diffusion, The Astrophysical Journal Supplement Series 135 (1) (2001) 95.doi:10.1086/321779. URLhttps://dx.doi.org/10.1086/321779

    work page doi:10.1086/321779 2001

  7. [7]

    Crash: A Block-Adaptive-Mesh Code for Radiative Shock Hydrodynamics - Implementation and Verification

    B. van der Holst, G. Toth, I. V. Sokolov, K. G. Powell, J. P. Holloway, E. S. Myra, Q. Stout, M. L. Adams, J. E. Morel, R. P. Drake, Crash: A Block-Adaptive-Mesh Code for Radiative Shock Hydrodynamics - Implementation and Verification, Astrophys. J. Suppl. 194 (2011) 23. arXiv:1101.3758,doi:10.1088/0067-0049/194/2/23

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.1088/0067-0049/194/2/23 2011

  8. [8]

    W.Zhang, L.Howell, A.Almgren, A.Burrows, J.Bell, CASTRO:ANew Compressible Astrophysical Solver. II. Gray Radiation Hydrodynamics, Astrophys. J. Suppl. 196 (2011) 20.arXiv:1105.2466,doi:10.1088/ 0067-0049/196/2/20

    work page internal anchor Pith review Pith/arXiv arXiv 2011

  9. [9]

    Jiang, Multigroup Radiation Magnetohydrodynamics Based on Discrete Ordinates including Compton Scattering, Astrophys

    Y.-F. Jiang, Multigroup Radiation Magnetohydrodynamics Based on Discrete Ordinates including Compton Scattering, Astrophys. J. Supp. 263 (1) (2022) 4.arXiv:2209.06240,doi:10.3847/1538-4365/ ac9231

    work page doi:10.3847/1538-4365/ 2022

  10. [10]

    Moens, J

    N. Moens, J. O. Sundqvist, I. El Mellah, L. Poniatowski, J. Teu- nissen, R. Keppens, Radiation-hydrodynamics with MPI-AMRVAC . Flux-limited diffusion,A&A657 (2022) A81.arXiv:2104.03968,doi: 10.1051/0004-6361/202141023

    work page doi:10.1051/0004-6361/202141023 2022

  11. [11]

    B. D. Wibking, M. R. Krumholz, QUOKKA: a code for two-moment AMR radiation hydrodynamics on GPUs,MNRAS512 (1) (2022) 1430– 1449.arXiv:2110.01792,doi:10.1093/mnras/stac439

    work page doi:10.1093/mnras/stac439 2022

  12. [12]

    Tanase, Hyesoon Kim, and Ching-Yung Lin

    T. Gamblin, M. LeGendre, M. R. Collette, G. L. Lee, A. Moody, B. R. de Supinski, S. Futral, The Spack Package Manager: Bringing Order to HPC Software Chaos, Supercomputing 2015 (SC’15), Austin, Texas, USA, 2015, lLNL-CONF-669890.doi:10.1145/2807591.2807623. URLhttps://github.com/spack/spack

    work page doi:10.1145/2807591.2807623 2015

  13. [13]

    Ayachit, The ParaView Guide: A Parallel Visualization Application, Kitware, Inc., Clifton Park, NY, USA, 2015

    U. Ayachit, The ParaView Guide: A Parallel Visualization Application, Kitware, Inc., Clifton Park, NY, USA, 2015. URLhttps://www.paraview.org/paraview-guide/

    work page 2015

  14. [14]

    Childs, E

    H. Childs, E. Brugger, B. Whitlock, J. Meredith, S. Ahern, D. Pugmire, K. Biagas, M. Miller, C. Harrison, G. H. Weber, H. Krishnan, T. Fogal, A. Sanderson, C. Garth, E. W. Bethel, D. Camp, O. Rübel, M. Du- rant, J. M. Favre, P. Navrátil, VisIt: An End-User Tool For Visualizing xvii and Analyzing Very Large Data, in: High Performance Visualization– Enablin...

    work page 2012

  15. [15]

    Mihalas, B

    D. Mihalas, B. W. Mihalas, Foundations of radiation hydrodynamics, Dover Publications, INC., 1984

    work page 1984

  16. [16]

    URLhttps://github.com/lanl/singularity-eos

    singularity-eos, https://github.com/lanl/singularity-eos (aug 25 2025). URLhttps://github.com/lanl/singularity-eos

    work page 2025

  17. [17]

    C. D. Levermore, G. C. Pomraning, A flux-limited diffusion theory, As- trophysical Journal 248 (1981) 321–334.doi:10.1086/159157

    work page doi:10.1086/159157 1981

  18. [18]

    Borges, M

    R. Borges, M. Carmona, B. Costa, W. S. Don, An improved weighted es- sentially non-oscillatory scheme for hyperbolic conservation laws, Jour- nal of Computational Physics 227 (6) (2008) 3191–3211.doi:10.1016/ j.jcp.2007.11.038

    work page 2008

  19. [19]

    Lax, and Bram van Leer

    A. Harten, P. D. Lax, B. v. Leer, On upstream differencing and godunov- type schemes for hyperbolic conservation laws, SIAM Review 25 (1) (1983) 35–61.doi:10.1137/1025002. URLhttps://doi.org/10.1137/1025002

    work page doi:10.1137/1025002 1983

  20. [20]

    V. Rusanov, The calculation of the interaction of non-stationary shock waves and obstacles, USSR Computational Mathematics and Mathematical Physics 1 (2) (1962) 304–320.doi:https: //doi.org/10.1016/0041-5553(62)90062-9. URLhttps://www.sciencedirect.com/science/article/pii/ 0041555362900629

    work page doi:10.1016/0041-5553(62)90062-9 1962

  21. [21]

    W. L. Briggs, V. E. Henson, S. F. McCormick, A Multigrid Tutorial, Second Edition, 2nd Edition, Society for Industrial and Applied Mathe- matics, 2000.arXiv:https://epubs.siam.org/doi/pdf/10.1137/1. 9780898719505,doi:10.1137/1.9780898719505. URLhttps://epubs.siam.org/doi/abs/10.1137/1.9780898719505

    work page doi:10.1137/1 2000

  22. [22]

    Trottenberg, C

    U. Trottenberg, C. W. Oosterlee, S. Anton, Multigrid, Elsevier, 2000

    work page 2000

  23. [23]

    Gist (grid impedance scan tool) [swr-22-73],

    A. Reisner, C. Mauney, flecsolve (03 2025).doi:10.11578/dc. 20250418.2. URLhttps://www.osti.gov/biblio/code-154403

    work page doi:10.11578/dc 2025

  24. [24]

    N. Roth, D. Kasen, Monte carlo radiation-hydrodynamics with implicit methods, The Astrophysical Journal Supplement Series 217 (1) (2015) 9.doi:10.1088/0067-0049/217/1/9. URLhttps://dx.doi.org/10.1088/0067-0049/217/1/9 xviii

    work page doi:10.1088/0067-0049/217/1/9 2015

  25. [25]

    rep., LANL (2023)

    ASC, xrage user manual, Tech. rep., LANL (2023)

    work page 2023

  26. [26]

    Chandrasekhar, Hydrodynamic and hydromagnetic stability, 1961

    S. Chandrasekhar, Hydrodynamic and hydromagnetic stability, 1961

    work page 1961

  27. [27]

    Shadmehri, Z

    M. Shadmehri, Z. Enayati, M. Khajavi, Magnetized kelvin-helmholtz instability in the presence of a radiation field, Astrophysics and Space Science 341 (2) (2012) 369–374.doi:10.1007/s10509-012-1096-4. URLhttps://doi.org/10.1007/s10509-012-1096-4

    work page doi:10.1007/s10509-012-1096-4 2012

  28. [28]

    H. Peng, F. Yu, Y. Huliuta, L. Wei, Z.-X. Wang, Y. Liu, Effect of Transverse Magnetic Field on Kelvin–Helmholtz Instability in the Pres- ence of a Radiation Field, Astrophysical Journal 970 (1) (2024) 3. doi:10.3847/1538-4357/ad5312. xix

    work page doi:10.3847/1538-4357/ad5312 2024