REVIEW 3 major objections 4 minor 18 references
Development of Atlas, a flexible data structure framework
T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read One library now unifies weather-model data structures for CPU and GPU
desk verdict Solid design report for a genuinely useful NWP data-structure library; no measurements, so judge it as an engineering document, not as evidence of performance. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing machinery is a four-concept data model. Grid is a (possibly structured, reduced, or unstructured) list of points with a projection and a domain; Mesh is a collection of polygonal elements linking grid points, with explicit connectivity tables and the ability to be partitioned across MPI tasks with halos; Field is a contiguous array of values with metadata and a pluggable memory layout; FunctionSpace interprets a field's discretisation—node columns, edge columns, structured columns, or spectral coefficients—and supplies parallel primitives including halo exchange and gather/scatter. The design also rests on two implementation choices: a Fortran derived type whose only member is a pointer to a matching C++ object, with calls delegated through C interfaces, and a storage backend that can be switched between a native implementation and the accelerator-aware storage provided by the DSL library (reference [6]) so that fields and mesh connectivity can be synchronised between CPU and GPU.
What would settle it
Run a representative Fortran NWP kernel (for example, a Jacobi or stencil update over columns and levels) with the same data access performed through Atlas's Fortran API inside the inner loop versus fetched once before the loop, on a current CPU and GPU; if the Atlas API version is measurably slower by more than a few percent, the assertion that delegation overhead can be disregarded would fail for that usage pattern.
Extended reading notes
Core claim
Atlas's central claim is that the four concepts—Grid, Mesh, Field, and FunctionSpace—form a complete and flexible foundation for NWP and climate applications. A Grid is an ordered list of points under a spatial projection; a Mesh adds elements and connectivity and may be split into MPI partitions with overlapping halo regions; a Field is a contiguous array of values with metadata that can be laid out in different memory orders; and a FunctionSpace ties them together by dictating how a field is discretised (nodes, edges, columns, or spherical harmonics) and provides the parallel operations, such as halo exchange and gather/scatter, that keep distributed fields consistent. The report argues that this separation lets the same numerical methods—finite volume, spectral element, spectral transform—be expressed once and run on emerging architectures, and that the Fortran API plus the GPU-aware storage backend make it possible to port Fortran operators to accelerators without replacing the underlying data structures.
Load-bearing premise
The efficiency claim depends on the assumption that delegating calls from the Fortran API to the C++ implementation has negligible overhead because users fetch field data before entering computational loops; the report does not test what happens when Atlas calls occur inside hot loops.
Editorial extensions
If this is right
- Numerical methods written against Atlas's FunctionSpace and Field concepts—finite volume, spectral element, spectral transform—can be reused across CPU and GPU backends without rewriting the data-structure layer.
- Models that adopt Atlas gain ready-made parallel communication (halo exchange, gather/scatter) through the FunctionSpace, so the burden of hand-coding MPI for field synchronisation is reduced.
- The grid catalogue covers the main global grid families used in the community (regular lon-lat, classic and octahedral reduced Gaussian) plus regional projected grids, so model intercomparison and limited-area modelling can share one framework.
- Because Atlas fields can be created with a host/device storage backend, Fortran kernels can be ported to GPUs with OpenACC while keeping the same data structures, as demonstrated by the report's example.
Reading between the lines
- If Atlas becomes the shared layer for several models, optimisations to its halo exchange, memory layout, or partitioners would benefit every model at once; the paper does not attempt to quantify this leverage.
- The report's assumption that Fortran-to-C++ delegation overhead is negligible outside computational loops implies a discipline—extract field data before looping—that users must follow; the paper gives no benchmark measuring the threshold where that overhead becomes material.
- The MatchingMesh partitioner points toward a future where interpolation and coupling between models on different grids can be done partition-locally without global communication, but scaling and load-balance of that approach are not reported.
- The planned generalisation of StructuredColumns beyond Gaussian grids (stated as a future release) would let the same field object serve both grid-point and spectral-transform codes; the current report only supports Gaussian grids for that function space.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This deliverable report describes Atlas, a C++ library with a Fortran API for managing data structures used in numerical weather prediction and climate modelling: grids (structured, reduced, unstructured), meshes, fields, function spaces, parallel communication (halo exchange, gather/scatter), and accelerator storage via the GridTools storage layer. The paper presents the design rationale, UML class diagrams, usage examples, installation instructions, and describes work done in the ESCAPE project to extend Atlas to regional grids and GPU support. The stated central claim is that Atlas provides a versatile framework for the future development of efficient NWP and climate applications on emerging HPC architectures.
Significance. The design is thoughtful and addresses real needs of the NWP community: a common abstraction for grid and mesh data structures, separation of grid/distribution/mesh, a function-space concept that manages parallelisation and memory layout, and an array abstraction with host/device views and synchronisation. The report is honest about current limitations (no GPU halo exchange, no projection-aware operators for regional grids, restricted access to the code). However, the central efficiency and scalability claims are not supported by benchmarks, scaling studies, or profiling data. The only performance-related assertion is the assumption in Section 4.1 that Fortran-to-C++ delegation overhead is negligible when calls are made outside computational loops. Thus, if the design and claims are taken at face value, Atlas could be a significant step for the community, but the evidence presented is insufficient to verify the efficiency promise.
major comments (3)
- [§4.1, §1] The claim that the Fortran API delegation overhead "can be disregarded if performed outside of a computational loop" (§4.1) is load-bearing for the Executive Summary's promise of "efficient" applications, but it is an unquantified assumption. No microbenchmark or profiling data are provided, and no evidence is given that realistic Fortran codes (e.g., with halo exchanges or metadata queries inside time-stepping loops) can always follow the recommended pattern of retrieving field data before entering loops. If per-call overhead is non-negligible, the efficiency claim is eroded. Please either measure the overhead or soften the claim.
- [§4.4, §5] The report describes parallel communication classes (GatherScatter, HaloExchange) and GPU support (§4.4, §5), but provides no scaling results, communication timings, GPU kernel timings, or comparisons with non-Atlas data structures. The assertion that Atlas enables "massively parallel" and "emerging HPC architectures" is therefore not substantiated by the evidence in the paper. At minimum, a weak scaling test on a representative grid/mesh workload and a microbenchmark of the Fortran API delegation would be needed to support the central claim.
- [§3.2, §6] The code is currently restricted to ESCAPE partners (§3.2), and the Conclusions state that the library has been "tested and compiled with various compilers and computer architectures" without describing any test suite, validation results, or error metrics. This limits the reproducibility of the claimed capabilities and makes it difficult for readers to assess the maturity of the library. Please provide a description of the tests performed or make the test suite available.
minor comments (4)
- [§2.2.1, §2.2.2] There are spelling errors: "accomodate" should be "accommodate" in §2.2.1, and "succesfully" should be "successfully" in §2.2.2.
- [§4.2.3.7, Figure 15] The note inside the class diagram says "RegularGaussianGrid is valid when it is a valid ReducedGrid..." but should refer to "ReducedGaussianGrid".
- [Listing 15] The kernel code contains garbled characters "t hr ea dI dx .x" instead of "threadIdx.x".
- [§3] Several sections contain duplicated words such as "Section section3.1" in §3, and the code listings have inconsistent spacing; a careful proofread is needed.
Circularity Check
No circularity: Atlas is a software design report whose claims are supported by implementation and design, not by derivation from the claims themselves.
full rationale
This document is a deliverable report describing the design, implementation, and installation of the Atlas library. It makes no fitted predictions and contains no derivation chain: the central claim that Atlas provides a versatile framework for efficient NWP and climate applications on emerging HPC architectures is a design goal supported by the presented architecture (Grid, Mesh, FunctionSpace, Field, partitioners, halo exchange, GPU storage integration) rather than by a calculation that reduces to its own inputs. Section 4.1 does state an assumption that Fortran-to-C++ delegation overhead can be disregarded outside computational loops, and the paper contains no benchmarks validating this assumption; however, that is a correctness or evidence gap, which is explicitly out of scope for the circularity pass. The self-citations to ESCAPE deliverables D1.1, D1.2, D2.4, and D4.4 are used for context and for referencing prior project work, not to import an unverified premise that already contains the central claim. The citations to the fvm finite-volume scheme (reference 15) and to the equal-regions partitioner (references 11-13) are ordinary references to published algorithms and implementation choices; they are not invoked as uniqueness theorems or as proofs of the library's efficiency. No step in the report defines, fits, or renames a result in terms of the output it is supposed to establish. The report is therefore self-contained against external benchmarks in the sense relevant to circularity analysis: there is no circular reduction to exhibit.
Assumptions & free parameters
assumptions (3)
- domain assumption Vertical discretisation is orthogonal to horizontal discretisation.
- standard math Gaussian grid latitudes are the roots of Legendre polynomials.
- ad hoc to paper Fortran-C++ delegation overhead is negligible when calls are made outside computational loops.
Cite this review
Pith. "Pith review of Development of Atlas, a flexible data structure framework." pith.science (2026). https://pith.science/paper/ZII6IAYU
@misc{pith2026190806091,
author = {Pith},
title = {Pith review of: Development of Atlas, a flexible data structure framework},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZII6IAYU}},
note = {Machine review of arXiv:1908.06091}
}
read the original abstract
This document is one of the deliverable reports created for the ESCAPE project. ESCAPE stands for Energy-efficient Scalable Algorithms for Weather Prediction at Exascale. The project develops world-class, extreme-scale computing capabilities for European operational numerical weather prediction and future climate models. This is done by identifying Weather & Climate dwarfs which are key patterns in terms of computation and communication (in the spirit of the Berkeley dwarfs). These dwarfs are then optimised for different hardware architectures (single and multi-node) and alternative algorithms are explored. Performance portability is addressed through the use of domain specific languages. In this deliverable report, we present Atlas, a new software library that is currently being developed at the European Centre for Medium-Range Weather Forecasts (ECMWF), with the scope of handling data structures required for NWP applications in a flexible and massively parallel way. Atlas provides a versatile framework for the future development of efficient NWP and climate applications on emerging HPC architectures. The applications range from full Earth system models, to specific tools required for post-processing weather forecast products. Atlas provides data structures for building various numerical strategies to solve equations on the sphere or limited area's on the sphere. These data structures may contain a distribution of points (grid) and, possibly, a composition of elements (mesh), required to implement the numerical operations required. Atlas can also represent a given field within a specific spatial projection. Atlas is capable of mapping fields between different grids as part of pre- and post-processing stages or as part of coupling processes whose respective fields are discretised on different grids or meshes.
Figures
Figures from the paper (30 more)
Reference graph
Works this paper leans on
-
[1]
In order to inter-operate with the DSL a converter between GridTools and Atlas is required
Allocate mirror storages for the accelerator memory within the existing data structures using the CUDA API for memory management and provide functions to synchronise the CPU and accelerator memory spaces. In order to inter-operate with the DSL a converter between GridTools and Atlas is required
-
[2]
Replace the existing Atlas data structures by GPU capable GridTools storages
-
[3]
Hybrid solution where both options, the Atlas native data structures and GridTools storages are supported. Using the GridTools storage has the advantage that its storage management framework already solves the problem of supporting storages in the accelerator memory and provides an API to operate on them and synchronise the CPU and accelerator copies. Add...
-
[4]
ESCAPE Deliverable D1.1, http : / / www . hpc - escape . eu / media - hub / escape-pub/escape-deliverables, Accessed: 31-03-2017
work page 2017
-
[5]
ESCAPE Deliverable D1.2: Batch 2: Definition of novel Weather & Climate Dwarfs, provision of prototype implementations and dissemination to other WPs, http : / / www . hpc - escape . eu / media - hub / escape - pub / escape - deliverables, Available: 31-12-2017
work page 2017
-
[6]
eu / media - hub / escape - pub / escape - deliverables, Accessed: 31-03-2017
ESCAPE Deliverable D4.4: Atlas extension for LAM use,http://www.hpc- escape . eu / media - hub / escape - pub / escape - deliverables, Accessed: 31-03-2017
work page 2017
-
[7]
eu/media-hub/escape-pub/escape-deliverables, Available: 31-12-2017
ESCAPE Deliverable D2.4: Domain-specific language (DSL) for dynamical cores on unstructured meshes/structured grids, http://www.hpc- escape. eu/media-hub/escape-pub/escape-deliverables, Available: 31-12-2017
2017
-
[8]
NVIDIA CUDA toolkit documentation, http://docs.nvidia.com/cuda/ , Accessed: 31-03-2017
work page 2017
Show all 18 references
-
[9]
Stella: A domain-specific tool for structured grid methods in weather and climate models
T. Gysi, C. Osuna, O. Fuhrer, M. Bianco, and T. C. Schulthess, “Stella: A domain-specific tool for structured grid methods in weather and climate models”, inProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, ser. SC ’15...
2015 doi
-
[10]
ESCAPE Deliverable D5.6: Establish software collaboration platform,http: //www.hpc- escape.eu/media- hub/escape- pub/escape- deliverables , Accessed: 31-03-2017
2017
-
[11]
Unified Modeling Language main webpage,http://www.uml.org, Accessed: 31-03-2017
2017
-
[12]
Use of reduced Gaussian grids in spectral models
M. Hortal and A. Simmons, “Use of reduced Gaussian grids in spectral models.”,Monthly Weather Review, vol. 119, pp. 1057–1074, 1991. 53 D1.3 – Development of Atlas, a flexible data structure framework
1991
-
[13]
A new grid for the IFS
S. Malardel, N. Wedi, W. Deconinck, M. Diamantakis, C. Kühnlein, G. Mozdzynski, M. Hamrud, and P. Smolarkiewicz, “A new grid for the IFS”, ECMWF Newsletter, vol. 146, pp. 23–28, 2016
2016
-
[14]
Accelerating extreme-scale numerical weather prediction
W. Deconinck, M. Hamrud, C. Kühnlein, G. Mozdzynski, P. Smolarkiewicz, J. Szmelter, and N. Wedi, “Accelerating extreme-scale numerical weather prediction”, inParallel Processing and Applied Mathematics, Springer, 2016, pp. 583–593
2016
-
[15]
A partition of the unit sphere into regions of equal area and small diameter
P. Leopardi, “A partition of the unit sphere into regions of equal area and small diameter”,Electronic Transactions on Numerical Analysis, vol. 25, no. 12, pp. 309–327, 2006
2006
-
[16]
A new partitioning approach for ECMWF’s integrated forecasting system (IFS)
G. Mozdzynski, “A new partitioning approach for ECMWF’s integrated forecasting system (IFS)”, inProceedings of the Twelfth ECMWF Workshop: Use of High Performance Computing in Meteorology, vol. 273, World Scientific, 2007, pp. 148–166
2007
-
[17]
A general method for modeling on irregular grids
A. MacDonald, J. Middlecoff, T. Henderson, and L. J.-L., “A general method for modeling on irregular grids”,High Performance Computing Applications, vol. 25, no. 1, pp. 392–403, 2011
2011
-
[18]
A finite-volume module for simulating global all-scale atmospheric flows
P. K. Smolarkiewicz, W. Deconinck, M. Hamrud, C. Kühnlein, G. Mozdzynski, J. Szmelter, and N. P. Wedi, “A finite-volume module for simulating global all-scale atmospheric flows”,Journal of Computational Physics, vol. 314, pp. 287–304, 2016. 54 D1.3 – Development of Atlas, a flexi...
2016
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.