Pith. sign in

REVIEW 4 major objections 6 minor 59 references

REDI turns raw scientific datasets into AI-ready training assets through one five-stage pipeline that also tracks provenance and works as an agent skill.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 07:08 UTC pith:4VV4KZPQ

load-bearing objection Solid multi-domain systems paper that ships a real IPTSO+provenance factory and expert-level fidelity on four paths; the soft spot is equating re-hosting known pipelines with a general readiness engine. the 4 major comments →

arxiv 2607.02771 v1 pith:4VV4KZPQ submitted 2026-07-02 cs.AI cs.CE

Automated Data Readiness for Scientific AI

Sean R. Wilkinson , Valentine G. Anantharaj , Jong Youl Choi , Ketan Maheshwari , Marshall McDonnell , Massimiliano Lupo Pasini , Polina Shpilker , Renan Souza
show 3 more authors
Patrick Widener Sarp Oral Wesley Brewer
This is my paper
classification cs.AI cs.CE
keywords data readinessscientific AIIPTSO pipelineprovenanceFAIRHPC preprocessingagentic workflowsfile I/O
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Scientific AI is limited less by models than by the messy, domain-specific work of turning archived instrument and simulation data into something a training loop can actually consume. This paper presents REDI, an open-source system that runs a fixed five-stage readiness pipeline—ingest, preprocess, transform, structure, and output—with built-in readiness scoring, step-level provenance, and a companion tool (SetGo) that handles FAIR metadata and catalog publishing. Across climate grids, protein structures, materials graphs, and fusion particle-mesh data, REDI advances each corpus from raw to AI-ready and matches the corresponding domain-expert reference outputs to numerical precision. Profiling of those runs shows that file I/O, not floating-point work, is the main cost, so format choice becomes a first-order design decision, and a climate case already scales near-ideally to 100 nodes. The practical claim is that data-preparation bottlenecks can be turned into shared, auditable community assets instead of one-off lab scripts.

Core claim

No prior framework fully unifies automated cross-domain transformation, quantitative readiness assessment, data-state provenance, and agent-callable deployment for scientific AI. REDI supplies that unification via a domain-agnostic IPTSO pipeline with per-stage instrumentation; evaluated on climate, proteomics, materials science, and nuclear fusion, it produces AI-ready outputs that validate against expert reference pipelines and, for climate, scales near-ideally to 100 nodes, while revealing file I/O as the dominant cost.

What carries the argument

The IPTSO five-stage pipeline (ingest → preprocess → transform → structure → output), implemented as domain-agnostic PipelineStep modules over a shared PipelineContext, with Flowcept capturing before/after data state at every step and redi assess quantifying domain-aware readiness deltas.

Load-bearing premise

That matching existing domain-expert preprocessing pipelines is a sufficient definition of AI readiness, so re-hosting those pipelines under one orchestrator with provenance generalizes as a true cross-domain readiness platform.

What would settle it

Run REDI end-to-end on a held-out scientific corpus that has a published AI-ready reference and check whether feature-level Pearson correlation and MAE stay at the claimed near-1.0 / near-0 levels, or whether a new domain forces substantial hand-written steps outside the five-stage template.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents REDI, an open-source framework that automates scientific AI data readiness via a domain-agnostic five-stage IPTSO pipeline (ingest, preprocess, transform, structure, output), with Flowcept provenance, a built-in redi assess readiness scorer, agent-callable skill packaging, and the companion SetGo tool for FAIR/metadata catalog publication. It is evaluated on four leadership-scale corpora—ClimaX/climate (NetCDF regrid), OpenFold/proteomics (mmCIF+MSA features), HydraGNN/materials (cutoff graphs to ADIOS), and XGC1/fusion (particle–mesh graphs)—claiming raw-to-AI-ready transformation with outputs matching domain-expert reference pipelines (Pearson ≈1.0 / MAE ≈0 on validated features), near-ideal strong scaling to 100 nodes on Frontier for the climate case, and the finding that file I/O (and format choice) dominates pipeline cost.

Significance. If the systems claims hold, this is a practically valuable contribution for DOE-scale AI data factories: a reusable orchestration layer that unifies format handling, stage-level provenance (PROVENANCE_CARD.md), quantitative before/after readiness deltas, and FAIR publication—areas that domain tools (Anemoi, PhysicsNeMo-Curator, HydraGNN pipelines) and general orchestrators (Nextflow, Snakemake, Dask) address only in isolation. Strengths that should be credited include open-source release, multi-domain fidelity validation against external expert pipelines (not self-defined targets), explicit stage timing/I/O profiling, and concrete parallel backends (Python futures, MPI, GNU Parallel, Slurm wrappers). The work is engineering rather than theoretical; its impact depends on whether REDI is a general readiness engine or primarily a well-instrumented rehost of four known workflows.

major comments (4)
  1. Section IV selection criteria and Section V-C validation design make fidelity to existing domain-expert preprocessing scripts (ClimaX Snakemake/xESMF, OpenFold make_mmcif/msa_features, HydraGNN cutoff graphs, XGC1 mesh projection) the operational definition of AI readiness. All four corpora were chosen because such ground-truth pipelines already exist; reported Pearson≈1.0 / MAE≈0 therefore shows that REDI can reproduce those pipelines under IPTSO+provenance, not that it discovers or generalizes readiness for corpora without a reference. Section VI already notes limited novelty for mature pipelines, but the abstract and contributions still claim a cross-domain platform that solves the readiness bottleneck. Either add a held-out/novel corpus without a published reference pipeline (or a domain where redi discover must drive execution), or reframe the central claim to orchestration, consist
  2. Abstract and Section V-E advertise near-ideal parallel scaling to 100 nodes on Frontier, but the experiment is climate-only (ClimaX/AWI-ESM), preliminary, and uses a fixed 4-processes-per-node layout; Fusion, OpenFold, and HydraGNN scaling are deferred. Given that Section V-D shows domain-dependent bottlenecks (NPZ output for climate, LMDB ingest for materials, ADIOS ingest for fusion), climate NetCDF scaling does not establish leadership-scale behavior for the other three modalities. Qualify the abstract claim to climate-only preliminary results, or provide multi-domain scaling (even on smaller node counts) before asserting platform-level parallel readiness.
  3. Section III-B.2 and V-A present redi discover as the path for novel data without workflows, yet it only emits non-executing plans and is described as early-stage; the fully automated redi run path requires a known domain and predefined steps. The introduction and abstract nonetheless market unified automated transformation and agent-native deployment as solved. Without an end-to-end agent experiment (e.g., Claude Code/Codex invoking REDI as a skill on a novel dataset with measured failure modes vs model-generated readiness) or an executed discover→run loop on a held-out corpus, the agentic and novel-data claims remain aspirational relative to the mature redi run results.
  4. Table II and Section V-B readiness deltas partly measure that the pipeline applied the stages it was designed to apply (e.g., none→z-score, positions-only→radius graph, raw dynamic range→[0,1]). That is useful closed-loop instrumentation, but it is weaker evidence of independent readiness improvement than the external reference comparisons in V-C. Clarify which assess metrics are domain-grounded quality criteria versus stage-completion indicators, and avoid treating categorical tags (UNIFORM, ✓, NORMALIZED) as quantitative readiness gains in the same way as continuous deltas.
minor comments (6)
  1. Figure 3 caption notes normalization omitted for XGC1 because it is applied at training time, yet Table I lists Normalize (min-max) under Transform for XGC1 and Table II reports eden min–max compression—align the pipeline description across Table I, Fig. 3, and Table II.
  2. Section V-C climate validation samples 10M elements for matrix comparison; state the sampling seed/reproducibility protocol and whether MAE ~0.01 K is within expected bilinear regrid error bounds for the chosen xESMF configuration.
  3. HydraGNN cutoff r=6 Å and OpenFold 23-token vocabulary / HHBLITS gap index are domain-critical constants; document them as configuration (not hard-coded silent defaults) and surface them in PROVENANCE_CARD.md examples.
  4. Related work (Section II-C) could more sharply contrast REDI with AIDRIN (assessment only) and Flowcept (provenance only) in a single comparison table of transformation / readiness / provenance / agent skill / multi-domain columns.
  5. Typographical/consistency: “F . File I/O” spacing; “Y AML” → YAML; ensure NPZ vs Zarr recommendation in V-F is consistent with ClimaX’s historical NPZ choice discussed in V-B.
  6. SetGo is central to the lifecycle figure and abstract but is only briefly specified in III-E; a short example of metadata.json → setgo publish to Hugging Face/CKAN would strengthen reproducibility claims.

Circularity Check

2 steps flagged

Engineering evaluation against external domain pipelines; only mild closed-loop assess and self-cited DRAI/SetGo scaffolding, not a forced derivation.

specific steps
  1. self definitional [Section V-B (Readiness Assessment), Table II and surrounding prose]
    "The gaps reported by redi assess before processing directly correspond to the pipeline stages that REDI applies, thereby providing a closed-loop verification that each gap has been addressed. This before/after assessment serves as a reproducible data quality metric that can be rerun at any point in the data lifecycle."

    redi assess is built to flag missing IPTSO-style readiness operations; REDI then applies those same stages. Reporting that assess gaps close after the pipeline runs is therefore partly true by construction of the assessor–pipeline pair, not an independent discovery that the data became AI-ready. Domain metrics in Table II still have external content (e.g., MSA depth, energy units), so this is mild and non-central.

  2. self citation load bearing [Section II-A; also Abstract/Intro framing of IPTSO and SetGo [4],[5]]
    "The Data Readiness for AI (DRAI) construct [5] maps each band transition onto a canonical IPTSO (Ingest, Preprocess, Transform, Structure, Output) pipeline stage, and defines five quantitative readiness levels from Level 1 (RAW) through Level 5 (FULLY AI READY), thus enabling reproducible quantification of readiness improvement."

    The operational readiness vocabulary and stage model that REDI implements and that redi assess quantifies come from DRAI by overlapping authors (Brewer, Widener, Anantharaj, Oral, et al.); SetGo [4] is likewise same-group companion work. This is scaffolding for the architecture and for the closed-loop assess claim, not a uniqueness theorem that forces the empirical fidelity/scaling results. Non-load-bearing for the main evaluation; raises score only modestly.

full rationale

REDI is a systems/engineering paper, not a fitted theory. Its load-bearing empirical claims—fidelity of outputs to ClimaX, OpenFold, HydraGNN, and XGC1 reference pipelines (Pearson ≈1.0 / MAE ≈0), I/O-dominated stage timings, and climate strong-scaling on Frontier—are checked against external corpora and domain-expert scripts, not against quantities defined by free parameters of REDI itself. That is independent evidence under the hard rules. Mild circularity appears only in two non-central places: (1) redi assess is designed around the same IPTSO stages REDI applies, so the before/after “readiness improvement” partly verifies by construction that the pipeline ran the stages it was built to run; (2) the IPTSO/DRAI readiness vocabulary and the companion SetGo tool are self-cited prior work by overlapping authors, which supplies scaffolding rather than a uniqueness theorem that forces the result. Neither reduces the central cross-domain fidelity or scaling claims to a fit or a self-citation chain. Score 2 is proportionate: one minor closed-loop and non-load-bearing self-citation, with the evaluation still self-contained against external benchmarks. The skeptic concern that matching four known pipelines does not prove generalization beyond re-hosting them is a correctness/scope issue, not circularity.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 3 invented entities

Systems paper: load-bearing content is software design plus empirical match to domain pipelines. Free parameters are engineering knobs (sample sizes, cutoffs, worker layouts). Axioms are FAIR/DRAI readiness vocabulary and the assumption that expert pipelines define ground truth. Invented entities are the REDI/SetGo products and the operational IPTSO readiness factory—not physical entities.

free parameters (4)
  • Climate validation subsample size (10M elements)
    For unkeyed climate arrays, validation randomly samples 10 million elements with shared indices; choice affects reported MAE/Corr stability though not claimed as a physical constant.
  • HydraGNN graph cutoff radius r = 6 Å
    Edge construction uses a fixed cutoff (Table II); domain convention but a free engineering choice for connectivity metrics.
  • Parallelism layout (e.g., 4 processes/node; 16 workers for OpenFold subsample)
    Scaling and timing results depend on chosen worker counts and backends; not derived from theory.
  • Readiness quality threshold for iterative loop
    Fig. 1 iteration continues until a quality threshold is met; threshold values are not fixed universally in the paper.
axioms (4)
  • domain assumption FAIR compliance is necessary but not sufficient for AI readiness; AI-ready means domain-appropriate cleaning/validation/feature engineering plus metadata for reproducible training.
    Stated in Introduction and DRAI framing; defines the problem REDI claims to solve.
  • domain assumption The IPTSO (Ingest–Preprocess–Transform–Structure–Output) stage model is an adequate operational backbone for cross-domain readiness.
    From cited DRAI work; architecture Section III builds entirely on this decomposition (authors note stage boundaries can be ambiguous).
  • domain assumption Existing published domain preprocessing pipelines for ClimaX, OpenFold, HydraGNN, and XGC1 are correct ground truth for scientific fidelity.
    Section IV selection criteria and Section V-C validation design rest on this.
  • standard math Standard numerical agreement metrics (Pearson, MAE, KL, etc.) on features/tensors suffice to certify no substantive scientific distortion.
    Used throughout redi validate comparisons.
invented entities (3)
  • REDI (Readiness Engine for Data Integration) independent evidence
    purpose: Unified five-stage automated transformation, assessment, provenance, and agent-callable skill for scientific AI data readiness.
    Primary software artifact; independent evidence is the open-source release and multi-domain runs, not an external physical prediction.
  • SetGo metadata readiness companion no independent evidence
    purpose: FAIR/license/provenance/governance checks and catalog publish for REDI outputs.
    Companion product (cited as to-appear SSDBM 2026); evidence is described workflow, not independent physics.
  • Operational DRAI readiness levels mapped onto IPTSO with PROVENANCE_CARD.md no independent evidence
    purpose: Make descriptive readiness bands executable and provenance-linked.
    Conceptual packaging of prior DRAI ideas into REDI assess/run; not a new natural entity.

pith-pipeline@v1.1.0-grok45 · 23171 in / 3512 out tokens · 41630 ms · 2026-07-12T07:08:01.931350+00:00 · methodology

0 comments
read the original abstract

Leadership computing facilities steward large-scale scientific datasets that routinely require substantial transformation before serving as AI training data. However, no existing framework fully unifies automated transformation, readiness assessment, provenance tracking, and agent-native deployment. We present REDI, an open-source framework that addresses this gap through a unified five-stage pipeline (ingest, preprocess, transform, structure, and output) with per-stage instrumentation for reproducibility and deployment as an agent-callable skill; companion tool SetGo automates FAIR compliance and catalog publication. Evaluated across climate, proteomics, materials science, and nuclear fusion, REDI transforms all datasets from raw to AI-ready, with outputs validated against domain-expert references, and preliminary results show near-ideal parallel scaling to 100 nodes on Frontier for the climate case. Provenance-instrumented profiling reveals file I/O as the dominant pipeline cost, with format selection a first-order optimization lever. These results establish REDI as a cross-domain platform providing automated data readiness for scientific AI, transforming data preparation bottlenecks into reproducible, reusable community assets.

Figures

Figures reproduced from arXiv: 2607.02771 by Jong Youl Choi, Ketan Maheshwari, Marshall McDonnell, Massimiliano Lupo Pasini, Patrick Widener, Polina Shpilker, Renan Souza, Sarp Oral, Sean R. Wilkinson, Valentine G. Anantharaj, Wesley Brewer.

Figure 1
Figure 1. Figure 1: The REDI-SetGo iterative data readiness lifecycle. Raw data goes through a sequence of transformations via REDI, followed by domain-aware readiness [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Domain-agnostic transformation architecture within REDI. Top ( [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: REDI pipeline stage timings based on dataset subsamples, each run on a single compute node. Normalization is omitted for XGC1 because it is [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Preliminary REDI parallel scalability results for the Climate [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

59 extracted references · 15 canonical work pages · 2 internal anchors

  1. [1]

    On the opportunities and risks of foundation models,

    R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill et al., “On the opportunities and risks of foundation models,” arXiv preprint arXiv:2108.07258, 2021. [Online]. Available: https: //doi.org/10.48550/arXiv.2108.07258

    Pith/arXiv arXiv doi:10.48550/arxiv.2108.07258 2021

  2. [2]

    The FAIR guiding principles for scientific data management and stewardship,

    M. D. Wilkinson, M. Dumontier, I. J. Aalbersberg, G. Appleton, M. Axton, A. Baak, N. Blomberg, J.-W. Boiten, L. B. da Silva Santos, P. E. Bourneet al., “The FAIR guiding principles for scientific data management and stewardship,”Scientific data, vol. 3, no. 1, pp. 1–9,

  3. [3]

    Available: https://doi.org/10.1038/sdata.2016.18

    [Online]. Available: https://doi.org/10.1038/sdata.2016.18

    doi:10.1038/sdata.2016.18 2016

  4. [4]

    AI-readiness for biomedical data: Bridge2AI recommendations,

    T. Clark, H. Caufield, J. A. Parker, S. Al Manir, E. Amorim, J. Eddy, N. Gim, B. Gow, W. Goar, M. Haendelet al., “AI-readiness for biomedical data: Bridge2AI recommendations,” Oct. 2024. [Online]. Available: https://doi.org/10.1101/2024.10.23.619844

    doi:10.1101/2024.10.23.619844 2024

  5. [5]

    SetGo: Metadata readiness for scientific AI datasets,

    S. R. Wilkinson, P. Shpilker, and W. Brewer, “SetGo: Metadata readiness for scientific AI datasets,” inThe International Conference on Scalable Scientific Data Management 2026 (SSDBM 2026). New York, NY , USA: Association for Computing Machinery, Aug. 2026, to appear. [Online]. Available: https://doi.org/10.1145/3828820.3828827

    work page doi:10.1145/3828820.3828827 2026

  6. [6]

    Data readiness pipeline patterns for scientific AI at scale: Insights from climate, fusion, life sciences, and materials,

    W. Brewer, P. Widener, V . Anantharaj, F. Wang, T. Beck, A. Shankar, and S. Oral, “Data readiness pipeline patterns for scientific AI at scale: Insights from climate, fusion, life sciences, and materials,” AI Magazine, vol. 47, no. 1, 2026. [Online]. Available: https: //doi.org/10.1002/aaai.70056

    work page doi:10.1002/aaai.70056 2026

  7. [7]

    Data readiness levels,

    N. D. Lawrence, “Data readiness levels,”arXiv preprint arXiv:1705.02245, 2017. [Online]. Available: https://doi.org/10.48550/ arXiv.1705.02245

    Pith/arXiv arXiv 2017

  8. [8]

    Washington, DC: National Academies Press, 2026, prepublication copy—uncorrected proofs

    National Academies of Sciences, Engineering, and Medicine,Frontiers of Statistics in Science and Engineering: 2035 and Beyond. Washington, DC: National Academies Press, 2026, prepublication copy—uncorrected proofs. [Online]. Available: https://doi.org/10.17226/29292

    doi:10.17226/29292 2035

  9. [9]

    Nvidia physicsnemo: An open-source framework for physics-based deep learning in science and engineering,

    S. K. Chandrasekar, C. Adams, M. A. Nabian, S. Nidhan, R. Cherukuri, and A. Kamenev, “Nvidia physicsnemo: An open-source framework for physics-based deep learning in science and engineering,” 2023, open-source framework for physics-based deep learning. [Online]. Available: https://github.com/NVIDIA/physicsnemo

    2023

  10. [10]

    AIFS – ECMWF’s data-driven forecasting system,

    S. Lang, M. Alexe, M. Chantry, J. Dramsch, F. Pinault, B. Raoult, M. C. A. Clare, C. Lessig, M. Maier-Gerber, L. Magnusson, Z. B. Bouall`egue, A. P. Nemesio, P. D. Dueben, A. Brown, F. Pappenberger, and F. Rabier, “AIFS – ECMWF’s data-driven forecasting system,”

  11. [11]

    Available: https://doi.org/10.48550/arXiv.2406.01465

    [Online]. Available: https://doi.org/10.48550/arXiv.2406.01465

    Pith/arXiv arXiv doi:10.48550/arxiv.2406.01465

  12. [12]

    Nature language model: deciphering the language of nature for scientific discovery,

    Y . Xia, P. Jin, S. Xie, L. He, C. Cao, R. Luo, G. Liu, Y . Wang, Z. Liu, Y .-J. Chenet al., “Nature language model: deciphering the language of nature for scientific discovery,”arXiv preprint arXiv:2502.07527, 2025. [Online]. Available: https://doi.org/10.48550/arXiv.2502.07527

    Pith/arXiv arXiv doi:10.48550/arxiv.2502.07527 2025

  13. [13]

    On scientific foundation models: Rigorous definitions, key applications, and a comprehensive survey,

    S. S. Menon, T. Mondal, S. Brahmachary, A. Panda, S. M. Joshi, K. Kalyanaraman, and A. D. Jagtap, “On scientific foundation models: Rigorous definitions, key applications, and a comprehensive survey,” Neural Networks, vol. 198, p. 108567, 2026. [Online]. Available: https://doi.org/10.1016/j.neunet.2026.108567

    work page doi:10.1016/j.neunet.2026.108567 2026

  14. [14]

    Towards a foundation model for partial differential equations across physics domains,

    E. Soares, E. V . Brazil, V . Shirasuna, B. W. S. R. de Carvalho, and C. Malossi, “Towards a foundation model for partial differential equations across physics domains,” 2025. [Online]. Available: https: //doi.org/10.48550/ARXIV .2511.21861

    doi:10.48550/arxiv 2025

  15. [15]

    2023 DOE Public Access Plan,

    United States Department of Energy, “2023 DOE Public Access Plan,” 2023. [Online]. Available: https://doi.org/10.11578/ 2023DOEPUBLICACCESSPLAN

    2023

  16. [16]

    Applying the FAIR principles to computational workflows,

    S. R. Wilkinson, M. Aloqalaa, K. Belhajjame, M. R. Crusoe, B. de Paula Kinoshita, L. Gadelha, D. Garijo, O. J. R. Gustafsson, N. Juty, S. Kanwal, F. Z. Khan, J. K ¨oster, K. Peters-von Gehlen, L. Pouchard, R. K. Rannow, S. Soiland-Reyes, N. Soranzo, S. Sufi, Z. Sun, B. Vilne, M. A. Wouters, D. Yuen, and C. Goble, “Applying the FAIR principles to computati...

    doi:10.1038/s41597-025-04451-9 2025

  17. [17]

    ClimaX: A foundation model for weather and climate,

    T. Nguyen, J. Brandstetter, A. Kapoor, J. K. Gupta, and A. Grover, “ClimaX: A foundation model for weather and climate,”arXiv preprint arXiv:2301.10343, 2023. [Online]. Available: https://doi.org/10.48550/ arXiv.2301.10343

    Pith/arXiv arXiv 2023

  18. [18]

    Openfold: Retraining alphafold2 yields new insights into its learning mechanisms and capacity for generalization,

    G. Ahdritz, N. Bouatta, C. Floristean, S. Kadyan, Q. Xia, W. Gerecke, T. J. O’Donnell, D. Berenberg, I. Fisk, N. Zanichelliet al., “Openfold: Retraining alphafold2 yields new insights into its learning mechanisms and capacity for generalization,”Nature methods, vol. 21, no. 8, pp. 1514–1524, 2024. [Online]. Available: https: //doi.org/10.1038/s41592-024-02272-z

    doi:10.1038/s41592-024-02272-z 2024

  19. [19]

    Orbit: Oak ridge base foundation model for earth system predictability,

    X. Wang, S. Liu, A. Tsaris, J.-Y . Choi, A. M. Aji, M. Fan, W. Zhang, J. Yin, M. Ashfaq, D. Luet al., “Orbit: Oak ridge base foundation model for earth system predictability,” in SC24: International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, 2024, pp. 1–11. [Online]. Available: https://doi.org/10.1109/SC41406.2024.00007

    Pith/arXiv arXiv doi:10.1109/sc41406.2024.00007 2024

  20. [20]

    Data readiness for AI: A 360-degree survey,

    K. Hiniduma, S. Byna, and J. L. Bez, “Data readiness for AI: A 360-degree survey,”ACM Computing Surveys, vol. 57, no. 9, pp. 1–39,

  21. [21]

    Available: https://doi.org/10.1145/3722214

    [Online]. Available: https://doi.org/10.1145/3722214

    doi:10.1145/3722214

  22. [22]

    Lustre unveiled: Evolution, design, advancements, and current trends,

    A. George, A. Dilger, M. J. Brim, R. Mohr, A. Shehata, J. Y . Choi, A. M. Karimi, J. Hanley, J. Simmons, D. Manno, V . M. Vergara, S. Oral, and C. Zimmer, “Lustre unveiled: Evolution, design, advancements, and current trends,”ACM Transactions on Storage, vol. 21, no. 3,

  23. [23]

    Available: https://doi.org/10.1145/3736583

    [Online]. Available: https://doi.org/10.1145/3736583

    work page doi:10.1145/3736583

  24. [24]

    An overview of the HDF5 technology suite and its applications,

    M. Folk, G. Heber, Q. Koziol, E. Pourmal, and D. Robinson, “An overview of the HDF5 technology suite and its applications,” inProceedings of the EDBT/ICDT 2011 Workshop on Array Databases. ACM, 2011, pp. 36–47. [Online]. Available: https: //doi.org/10.1145/1966895.1966900

    work page doi:10.1145/1966895.1966900 2011

  25. [25]

    NetCDF: An interface for scientific data access,

    R. Rew and G. Davis, “NetCDF: An interface for scientific data access,”IEEE Computer Graphics and Applications, vol. 10, no. 4, pp. 76–82, 1990. [Online]. Available: https://doi.org/10.1109/38.56302

    doi:10.1109/38.56302 1990

  26. [26]

    ADIOS 2: The adaptable input output system. a framework for high-performance data management,

    W. F. Godoy, N. Podhorszki, R. Wang, C. Atkins, G. Eisenhauer, J. Gu, P. Davis, J. Choi, K. Germaschewski, K. Hucket al., “ADIOS 2: The adaptable input output system. a framework for high-performance data management,”SoftwareX, vol. 12, p. 100561, 2020. [Online]. Available: https://doi.org/10.1016/j.softx.2020.100561

    doi:10.1016/j.softx.2020.100561 2020

  27. [27]

    Zarr: A cloud-optimized storage for interactive access of large arrays,

    J. Moore and S. Kunis, “Zarr: A cloud-optimized storage for interactive access of large arrays,” inProceedings of the Conference on Research Data Infrastructure, vol. 1, 2023. [Online]. Available: https://doi.org/10.52825/cordi.v1i.285

    work page doi:10.52825/cordi.v1i.285 2023

  28. [28]

    LMDB: Lightning memory-mapped database,

    H. Chu, “LMDB: Lightning memory-mapped database,” http://www. lmdb.tech/doc/, 2011, symas Corporation

    2011

  29. [29]

    Great expectations,

    A. Gong, J. Campbell, and G. Expectations, “Great expectations,”

  30. [30]

    Available: https://doi.org/10.5281/zenodo.5683574

    [Online]. Available: https://doi.org/10.5281/zenodo.5683574

    work page doi:10.5281/zenodo.5683574

  31. [31]

    AI data readiness inspector (aidrin) for quantitative assessment of data readiness for AI,

    K. Hiniduma, S. Byna, J. L. Bez, and R. Madduri, “AI data readiness inspector (aidrin) for quantitative assessment of data readiness for AI,” inProceedings of the 36th International Conference on Scientific and Statistical Database Management, 2024, pp. 1–12. [Online]. Available: https://doi.org/10.1145/3676288.3676296

    work page doi:10.1145/3676288.3676296 2024

  32. [32]

    A terminology for scientific workflow systems,

    F. Suter, T. Coleman, ˙I. Altintas ¸, R. M. Badia, B. Balis, K. Chard, I. Colonnelli, E. Deelman, P. Di Tommaso, T. Fahringer, C. Goble, S. Jha, D. S. Katz, J. K ¨oster, U. Leser, K. Mehta, H. Oliver, J.-L. Peterson, G. Pizzi, L. Pottier, R. Sirvent, E. Suchyta, D. Thain, S. R. Wilkinson, J. M. Wozniak, and R. Ferreira da Silva, “A terminology for scienti...

    doi:10.1016/j.future.2025.107974 2026

  33. [33]

    Nextflow enables reproducible computational workflows,

    P. Di Tommaso, M. Chatzou, E. W. Floden, P. P. Barja, E. Palumbo, and C. Notredame, “Nextflow enables reproducible computational workflows,”Nature Biotechnology, vol. 35, no. 4, pp. 316–319, 2017. [Online]. Available: https://doi.org/10.1038/nbt.3820

    doi:10.1038/nbt.3820 2017

  34. [34]

    Sustainable data analysis with snakemake,

    F. M ¨older, K. P. Jablonski, B. Letcher, M. B. Hall, C. H. Tomkins-Tinch, V . Sochat, J. Forster, S. Lee, S. O. Twardziok, A. Kanitz, A. Wilm, M. Holtgrewe, S. Rahmann, S. Nahnsen, and J. K ¨oster, “Sustainable data analysis with snakemake,”F1000Research, vol. 10, p. 33, 2021. [Online]. Available: https://doi.org/10.12688/f1000research.29032.2

    doi:10.12688/f1000research.29032.2 2021

  35. [35]

    Dask: Parallel computation with blocked algorithms and task scheduling,

    M. Rocklin, “Dask: Parallel computation with blocked algorithms and task scheduling,” inProceedings of the 14th Python in Science Conference, 2015, pp. 130–136. [Online]. Available: https: //doi.org/10.25080/MAJORA-7B98E3ED-013

    doi:10.25080/majora-7b98e3ed-013 2015

  36. [36]

    Ray: A distributed framework for emerging AI applications,

    P. Moritz, R. Nishihara, S. Wang, A. Tumanov, R. Liaw, E. Liang, M. Elibol, Z. Yang, W. Paul, M. I. Jordan, and I. Stoica, “Ray: A distributed framework for emerging AI applications,” in13th USENIX Symposium on Operating Systems Design and Implementation, 2018, pp. 561–577. [Online]. Available: https: //dl.acm.org/doi/10.5555/3291168.3291210

    doi:10.5555/3291168.3291210 2018

  37. [37]

    Apache Spark: A unified engine for big data processing,

    M. Zaharia, R. S. Xin, P. Wendell, T. Das, M. Armbrust, A. Dave, X. Meng, J. Rosen, S. Venkataraman, M. J. Franklin, A. Ghodsi, J. Gonzalez, S. Shenker, and I. Stoica, “Apache Spark: A unified engine for big data processing,”Communications of the ACM, vol. 59, no. 11, pp. 56–65, 2016. [Online]. Available: https://doi.org/10.1145/2934664

    doi:10.1145/2934664 2016

  38. [38]

    Parsl: Pervasive parallel programming in Python,

    Y . Babuji, A. Woodard, Z. Li, D. S. Katz, B. Clifford, R. Kumar, L. Lacinski, R. Chard, J. M. Wozniak, I. Foster, M. Wilde, and K. Chard, “Parsl: Pervasive parallel programming in Python,” in Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing, 2019, pp. 25–36. [Online]. Available: https://doi.org/10.114...

    doi:10.1145/3307681.3325400 2019

  39. [39]

    Cloud- native repositories for big scientific data,

    R. P. Abernathey, T. Augspurger, A. Banihirwe, C. C. Blackmon-Luca, T. J. Crone, C. L. Gentemann, J. J. Hamman, N. Henderson, C. Lepore, T. A. McCaie, N. H. Robinson, and R. P. Signell, “Cloud- native repositories for big scientific data,”Computing in Science & Engineering, vol. 23, no. 2, pp. 26–35, 2021. [Online]. Available: https://doi.org/10.1109/MCSE...

    doi:10.1109/mcse.2021.3059437 2021

  40. [40]

    Scalable training of trustworthy and energy-efficient predictive graph foundation models for atomistic materials modeling: a case study with HydraGNN,

    M. Lupo Pasini, J. Y . Choi, K. Mehta, P. Zhang, D. Rogers, J. Bae, K. Z. Ibrahim, A. M. Aji, K. W. Schulz, J. Poloet al., “Scalable training of trustworthy and energy-efficient predictive graph foundation models for atomistic materials modeling: a case study with HydraGNN,”The Journal of Supercomputing, vol. 81, no. 4, p. 618, 2025. [Online]. Available: ...

    work page doi:10.1007/s11227-025-07029-9 2025

  41. [41]

    Accelerating the machine learning lifecycle with MLflow,

    M. Zaharia, A. Chen, A. Davidson, A. Ghodsi, S. A. Hong, A. Konwinski, S. Murching, T. Nykodym, P. Ogilvie, M. Parkheet al., “Accelerating the machine learning lifecycle with MLflow,”IEEE Data Engineering Bulletin, vol. 41, no. 4, pp. 39–45, 2018. [Online]. Available: https://people.eecs.berkeley.edu/∼matei/papers/2018/ieee mlflow.pdf

    2018

  42. [42]

    Towards lightweight data integration using multi- workflow provenance and data observability,

    R. Souza, T. J. Skluzacek, S. R. Wilkinson, M. Ziatdinov, and R. F. da Silva, “Towards lightweight data integration using multi- workflow provenance and data observability,” inIEEE International Conference on e-Science, 2023. [Online]. Available: https://doi.org/10. 1109/e-Science58273.2023.10254822

    arXiv 2023

  43. [43]

    SWE-agent: Agent-computer interfaces enable automated software engineering,

    J. Yang, C. E. Jimenez, A. Wettig, K. Lieret, S. Yao, K. Narasimhan, and O. Press, “SWE-agent: Agent-computer interfaces enable automated software engineering,” inAdvances in Neural Information Processing Systems, vol. 37, 2024. [Online]. Available: https://dl.acm.org/doi/10. 5555/3737916.3739517

    arXiv 2024

  44. [44]

    Do large language models speak scientific workflows?

    O. Yildiz and T. Peterka, “Do large language models speak scientific workflows?” inProceedings of the SC ’25 Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis, ser. SC Workshops ’25. New York, NY , USA: Association for Computing Machinery, 2025, p. 2225–2233. [Online]. Available: https://doi.org/10....

    work page doi:10.1145/3731599.3767578 2025

  45. [45]

    Towards generating contracts for scientific data analysis workflows,

    A. D. Vu and T. Kehrer, “Towards generating contracts for scientific data analysis workflows,” inProceedings of the SC ’24 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis, ser. SC-W ’24. IEEE Press, 2025, p. 2048–2055. [Online]. Available: https://doi.org/10.1109/SCW63240.2024.00256

    doi:10.1109/scw63240.2024.00256 2025

  46. [46]

    LLM agents for interactive workflow provenance: Reference architecture and evaluation methodology,

    R. Souza, T. Poteet, B. Etz, D. Rosendo, A. Gueroudji, W. Shin, P. Balaprakash, and R. F. da Silva, “LLM agents for interactive workflow provenance: Reference architecture and evaluation methodology,” inProceedings of the SC ’25 Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis, ser. SC Workshops ’2...

    work page doi:10.1145/3731599.3767582 2025

  47. [47]

    Leakage in data mining: Formulation, detection, and avoidance,

    S. Kaufman, S. Rosset, C. Perlich, and O. Stitelman, “Leakage in data mining: Formulation, detection, and avoidance,”ACM Transactions on Knowledge Discovery from Data, vol. 6, no. 4, p. 15, 2012. [Online]. Available: https://doi.org/10.1145/2382577.2382579

    doi:10.1145/2382577.2382579 2012

  48. [48]

    Enabling low-overhead ht-hpc workflows at extreme scale using gnu parallel,

    K. Maheshwari, W. Arndt, A. M. Karimi, J. Yin, F. Suter, S. Johnson, and R. F. Da Silva, “Enabling low-overhead ht-hpc workflows at extreme scale using gnu parallel,” inSC24-W: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2024, pp. 2056–2063. [Online]. Available: https://doi.org/10.1109/SCW632...

    doi:10.1109/scw63240.2024.00257 2024

  49. [49]

    Whole- volume integrated gyrokinetic simulation of plasma turbulence in realistic diverted-tokamak geometry,

    C. Chang, S. Ku, P. Diamond, M. Adams, R. Barreto, Y . Chen, J. Cummings, E. D’Azevedo, G. Dif-Pradalier, S. Ethieret al., “Whole- volume integrated gyrokinetic simulation of plasma turbulence in realistic diverted-tokamak geometry,” inJournal of Physics: Conference Series, vol. 180, no. 1, 2009, p. 012057. [Online]. Available: https://doi.org/10.1088/174...

    work page doi:10.1088/1742-6596/180/1/012057 2009

  50. [50]

    Highly accurate protein structure prediction with AlphaFold,

    J. Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov, O. Ronneberger, K. Tunyasuvunakool, R. Bates, A. ˇZ´ıdek, A. Potapenkoet al., “Highly accurate protein structure prediction with AlphaFold,” nature, vol. 596, no. 7873, pp. 583–589, 2021. [Online]. Available: https://doi.org/10.1038/s41586-021-03819-2

    doi:10.1038/s41586-021-03819-2 2021

  51. [51]

    The open catalyst 2020 (oc20) dataset and community challenges,

    L. Chanussot, A. Das, S. Goyal, T. Lavril, M. Shuaibi, M. Riviere, K. Tran, J. Heras-Domingo, C. Ho, W. Huet al., “The open catalyst 2020 (oc20) dataset and community challenges,”ACS Catalysis, vol. 11, no. 10, pp. 6059–6072, 2021. [Online]. Available: https://doi.org/10.1021/acscatal.0c04525

    doi:10.1021/acscatal.0c04525 2020

  52. [52]

    The open catalyst 2022 (oc22) dataset and challenges for oxide electrocatalysts,

    R. Tran, J. Lan, M. Shuaibi, B. M. Wood, S. Goyal, A. Das, J. Heras-Domingo, A. Kolluru, A. Rizvi, N. Shoghiet al., “The open catalyst 2022 (oc22) dataset and challenges for oxide electrocatalysts,” ACS Catalysis, vol. 13, no. 5, pp. 3066–3084, 2023. [Online]. Available: https://doi.org/10.1021/acscatal.2c05426

    doi:10.1021/acscatal.2c05426 2022

  53. [53]

    A universal graph deep learning interatomic potential for the elements,

    B. Deng, P. Zhong, K. Jun, J. Riebesell, K. Han, C. J. Bartel, and G. Ceder, “A universal graph deep learning interatomic potential for the elements,”Nature Machine Intelligence, vol. 5, pp. 1031–1041,

  54. [54]

    CHGNet: Pretrained universal neural network potential for charge-informed atomistic modeling

    [Online]. Available: https://doi.org/10.48550/arXiv.2302.14231

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2302.14231

  55. [55]

    The ani-1ccx and ani-1x data sets, coupled-cluster and density functional theory properties for molecules,

    J. S. Smith, R. Zubatyuk, B. Nebgen, N. Lubbers, K. Barros, A. E. Roitberg, O. Isayev, and S. Tretiak, “The ani-1ccx and ani-1x data sets, coupled-cluster and density functional theory properties for molecules,” Scientific Data, vol. 7, no. 1, p. 134, 2020. [Online]. Available: https://doi.org/10.1038/s41597-020-0473-z

    doi:10.1038/s41597-020-0473-z 2020

  56. [56]

    Qm7-x, a comprehensive dataset of quantum-mechanical properties spanning the chemical space of small organic molecules,

    J. Hoja, L. Medrano Sandonas, B. G. Ernst, A. Vazquez-Mayagoitia, R. A. DiStasio Jr., and A. Tkatchenko, “Qm7-x, a comprehensive dataset of quantum-mechanical properties spanning the chemical space of small organic molecules,”Scientific Data, vol. 8, no. 1, p. 43, 2021. [Online]. Available: https://doi.org/10.1038/s41597-021-00812-2

    doi:10.1038/s41597-021-00812-2 2021

  57. [57]

    A fast low-to-high confinement mode bifurcation dynamics in the boundary- plasma gyrokinetic code xgc1,

    S. Ku, C. Chang, R. Hager, R. Churchill, G. Tynan, I. Cziegler, M. Greenwald, J. Hughes, S. E. Parker, M. Adamset al., “A fast low-to-high confinement mode bifurcation dynamics in the boundary- plasma gyrokinetic code xgc1,”Physics of Plasmas, vol. 25, no. 5,

  58. [58]

    Available: https://doi.org/10.1063/1.5020792

    [Online]. Available: https://doi.org/10.1063/1.5020792

    work page doi:10.1063/1.5020792

  59. [59]

    MATEY: multiscale adaptive foundation models for spatiotemporal physical systems

    P. Zhang, M. P. Laiu, M. Norman, D. Stefanski, and J. Gounley, “MATEY: multiscale adaptive foundation models for spatiotemporal physical systems,”arXiv preprint arXiv:2412.20601, 2024. [Online]. Available: https://doi.org/10.48550/arXiv.2412.20601

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2412.20601 2024