astroCAMP: A Community Benchmark and Co-Design Framework for Sustainable SKA-Scale Radio Imaging
Pith reviewed 2026-05-16 21:50 UTC · model grok-4.3
The pith
astroCAMP supplies a metric suite and multi-objective workflow to co-design sustainable SKA-scale radio imaging.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
astroCAMP introduces a unified metric suite and multi-objective co-design formulation for SKA-scale radio imaging. The formulation links quality constraints directly to time-to-solution, energy-to-solution, carbon-to-solution, and cost-to-solution, then applies a design-space exploration workflow to derive Pareto-optimal regions. Demonstrations on AMD EPYC CPUs paired with NVIDIA H100 GPUs reveal orchestration and synchronization bottlenecks despite efficient kernels, limited CPU strong scaling, and location-dependent carbon and cost efficiency; an extension to CPU-FPGA exploration illustrates the workflow's applicability to heterogeneous platforms.
What carries the argument
The multi-objective co-design formulation that connects scientific fidelity constraints to time-, energy-, carbon-, and cost-to-solution metrics, together with the design-space exploration workflow that identifies Pareto-optimal operating regions.
If this is right
- Current radio-interferometric pipelines exhibit only 4-14 percent hardware utilization due to memory and I/O bottlenecks that the framework can now quantify consistently.
- Reproducible cross-platform evaluations become possible through standardized SKA-representative datasets and benchmark configurations.
- Heterogeneous hardware choices can be compared by their location-dependent carbon and cost efficiency alongside performance and fidelity.
- The workflow can derive concrete Pareto fronts that trade image quality against energy and operational cost for specific imaging tasks.
Where Pith is reading between the lines
- The same metric and optimization structure could be adapted to other large-scale scientific computing workloads that face strict power envelopes.
- Widespread use of the benchmarks might standardize evaluation practices across radio-astronomy software projects and reduce duplicated effort.
- If fidelity thresholds are established, the framework could inform procurement decisions for future SKA computing infrastructure by making sustainability trade-offs explicit.
- The approach highlights the value of co-design loops that close the gap between algorithm developers and hardware architects for data-movement-heavy applications.
Load-bearing premise
Quantifiable fidelity thresholds acceptable to the SKA community can be defined and used as hard constraints in the optimization without invalidating the scientific utility of the resulting images.
What would settle it
A demonstration that images produced by the optimization under any community-agreed fidelity thresholds fail to support the key scientific measurements required for SKA observations, or that the framework produces no measurable improvement in the combined resource metrics over existing pipelines.
Figures
read the original abstract
The Square Kilometre Array (SKA) will operate one of the world's largest continuous scientific data systems, sustaining petascale imaging under strict power envelopes. Current radio-interferometric pipelines typically achieve only 4-14% of hardware peak utilization due to memory and I/O bottlenecks, incurring high energy, operational, and carbon costs, further compounded by the absence of standardised cross-layer metrics and fidelity tolerances for principled hardware--software co-design. We present astroCAMP, a reproducible benchmarking and co-design framework for SKA-scale imaging, contributing: (1) a unified metric suite spanning performance, utilisation, memory/data-movement, sustainability, economics, and scientific fidelity; (2) standardised SKA-representative datasets and benchmark configurations for reproducible cross-platform evaluation; (3) a multi-objective co-design formulation linking quality constraints to time-, energy-, carbon-, and cost-to-solution; and (4) a design-space exploration workflow to derive Pareto-optimal operating regions. We evaluate WSClean+IDG on an AMD EPYC 9334 CPU and NVIDIA H100 GPU, revealing orchestration and synchronization bottlenecks despite efficient kernels, limited CPU strong scaling, and location-dependent carbon/cost efficiency. We illustrate astroCAMP for heterogeneous CPU--FPGA exploration and call on the SKA community to define quantifiable fidelity thresholds to accelerate principled optimisation for SKA-scale imaging.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces astroCAMP, a reproducible benchmarking and co-design framework for SKA-scale radio imaging. It contributes (1) a unified metric suite spanning performance, utilisation, memory/data-movement, sustainability, economics, and scientific fidelity; (2) standardised SKA-representative datasets and benchmark configurations; (3) a multi-objective co-design formulation linking quality constraints to time-, energy-, carbon-, and cost-to-solution; and (4) a design-space exploration workflow to derive Pareto-optimal operating regions. Evaluation of WSClean+IDG on AMD EPYC 9334 CPU and NVIDIA H100 GPU reveals orchestration bottlenecks, limited CPU scaling, and location-dependent efficiencies; a CPU-FPGA illustration is provided. The authors defer definition of quantifiable fidelity thresholds to the SKA community.
Significance. If the framework is completed with concrete fidelity constraints, it would provide a valuable standardised toolset for sustainable co-design in radio astronomy, addressing the documented low utilisation (4-14%) and high energy/carbon costs of current pipelines. The metric suite and reproducible datasets could improve cross-platform comparisons and community-driven optimisation for SKA. The multi-objective formulation has potential to link scientific quality to operational costs, but its impact hinges on whether enforceable, non-invalidating fidelity thresholds can be established.
major comments (2)
- [Abstract] Abstract, contribution (3): The multi-objective co-design formulation is described as treating quality constraints as enforceable bounds for deriving Pareto-optimal regions, yet the evaluation on WSClean+IDG reports only performance, utilisation, and sustainability numbers without instantiating any concrete fidelity metric (dynamic range, residual RMS, or source-recovery completeness) or applying an explicit threshold.
- [Evaluation] Evaluation section: The CPU/GPU results and CPU-FPGA illustration demonstrate orchestration bottlenecks and efficiency variations but contain no fidelity measurements, so the workflow cannot be shown to produce operating regions that satisfy the stated quality-constrained multi-objective claim.
minor comments (1)
- [Abstract] The abstract states current pipelines achieve 'only 4-14% of hardware peak utilization' without citing the specific prior measurements or section that establishes this range.
Simulated Author's Rebuttal
We thank the referee for the constructive and detailed review. We address the major comments point by point below. We agree that the current evaluation does not apply concrete fidelity thresholds, as the manuscript explicitly defers their definition to the SKA community. We will make targeted revisions to clarify this distinction in the abstract and evaluation sections without altering the core contributions.
read point-by-point responses
-
Referee: [Abstract] Abstract, contribution (3): The multi-objective co-design formulation is described as treating quality constraints as enforceable bounds for deriving Pareto-optimal regions, yet the evaluation on WSClean+IDG reports only performance, utilisation, and sustainability numbers without instantiating any concrete fidelity metric (dynamic range, residual RMS, or source-recovery completeness) or applying an explicit threshold.
Authors: We acknowledge the observation. The manuscript states that we defer definition of quantifiable fidelity thresholds to the SKA community because no consensus currently exists on enforceable, non-invalidating bounds. The unified metric suite includes fidelity metrics, and the formulation supports quality constraints as bounds, but the evaluation illustrates the workflow using the metrics for which data are available. We will revise the abstract to explicitly distinguish the general formulation from the current evaluation, which derives Pareto regions under the demonstrated metrics while noting the pending fidelity component. revision: partial
-
Referee: [Evaluation] Evaluation section: The CPU/GPU results and CPU-FPGA illustration demonstrate orchestration bottlenecks and efficiency variations but contain no fidelity measurements, so the workflow cannot be shown to produce operating regions that satisfy the stated quality-constrained multi-objective claim.
Authors: We agree that no fidelity measurements appear in the reported results. This follows directly from the paper's call for community-defined thresholds; without them, we cannot enforce or demonstrate satisfaction of quality constraints in the evaluation. The results instead show the multi-objective optimisation for time, energy, carbon, and cost, with the framework designed to incorporate fidelity once thresholds are established. We will add clarifying text in the evaluation section stating that the derived operating regions are with respect to the currently quantified metrics and that fidelity constraints will be integrated upon community input. revision: partial
- Definition of concrete, enforceable fidelity thresholds (e.g., dynamic range or source-recovery completeness) that do not invalidate scientific results, which the manuscript defers to the SKA community as no such standards currently exist.
Circularity Check
No significant circularity in astroCAMP framework
full rationale
The paper introduces a benchmarking framework, metric suite, datasets, multi-objective co-design formulation, and design-space workflow as independent constructs. The formulation explicitly treats quality constraints as external inputs to be supplied by the SKA community rather than deriving or fitting them internally. No equations, predictions, or results are shown that reduce to self-referential definitions, fitted parameters renamed as outputs, or load-bearing self-citations. The reported CPU/GPU and CPU-FPGA evaluations consist of direct empirical measurements on external hardware, not constructed predictions. This is a standard self-contained framework proposal with no circular steps.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
https://app.electricitymaps.com/, 2025
Electricity maps. https://app.electricitymaps.com/, 2025. Accessed: November 2025
work page 2025
-
[2]
R., Constantinescu, D.-A., Peón-Quirós, M., and Atienza, D
Álvarez, R. R., Constantinescu, D.-A., Peón-Quirós, M., and Atienza, D. Ceo-dc: Driving decarbonization in hpc data centers with actionable insights. arXiv preprint arXiv:2507.08923(2025)
-
[3]
Aujoux, C., Kotera, K., and Blanchard, O.Estimating the carbon footprint of the grand project, a multi-decade astrophysics experiment.Astroparticle Physics 131(2021), 102587
work page 2021
-
[4]
Bacon, D. J., Battye, R. A., Bull, P., Camera, S., Ferreira, P. G., Harrison, I., Parkinson, D., Pourtsidou, A., Santos, M. G., Wolz, L., et al.Cosmology with phase 1 of the square kilometre array red book 2018: technical specifications and performance forecasts.Publications of the Astronomical Society of Australia 37 (2020), e007
work page 2018
-
[5]
Datavizta: Environmental Impact Modelling Tool
Boavizta. Datavizta: Environmental Impact Modelling Tool. https://dataviz. boavizta.org/, 2024. Accessed: December 2025
work page 2024
-
[6]
Broekema, P. C., van Nieuwpoort, R. V., and Bal, H. E.The square kilometre array science data processor: Preliminary compute platform design.Journal of Instrumentation 10, 07 (2015), C07004
work page 2015
-
[7]
In 2022 IEEE/ACM International Workshop on HPC User Support Tools (HUST)(2022), IEEE, pp
Corda, S., Veenboer, B., and Tolley, E.Pmt: Power measurement toolkit. In 2022 IEEE/ACM International Workshop on HPC User Support Tools (HUST)(2022), IEEE, pp. 44–47
work page 2022
-
[8]
Corda, S., Veenboer, B., and van Nieuwpoort, R.Reduced-precision accelera- tion of radio-astronomical imaging on reconfigurable hardware.IEEE Access 10 (2022), 104780–104795
work page 2022
-
[9]
Cornwell, T. J., Golap, K., and Bhatnagar, S.The noncoplanar baselines effect in radio interferometry: The w-projection algorithm.IEEE Journal of Selected Topics in Signal Processing 2, 5 (2008), 647–657
work page 2008
-
[10]
Dewdney, P. E., Hall, P. J., Schilizzi, R. T., and Lazio, T. J. W.The square kilometre array.Proceedings of the IEEE 97, 8 (2009), 1482–1496
work page 2009
-
[11]
Nature Astronomy 8, 11 (2024), 1468–1477
dos Santos Ilha, G., Boix, M., Knödlseder, J., Garnier, P., Montastruc, L., Jean, P., Pareschi, G., Steiner, A., and Toussenel, F.Assessment of the environmental impacts of the cherenkov telescope array mid-sized telescope. Nature Astronomy 8, 11 (2024), 1468–1477
work page 2024
-
[12]
Farrell, S., Emani, M., Balma, J., Drescher, L., Drozd, A., Fink, A., Fox, G., Kanter, D., Kurth, T., Mattson, P., et al.Mlperf™hpc: A holistic benchmark suite for scientific machine learning on hpc systems. In2021 IEEE/ACM Workshop on Machine Learning in High Performance Computing Environments (MLHPC) (2021), IEEE, pp. 33–45
work page 2021
-
[13]
In2025 62nd ACM/IEEE Design Automation Conference (DAC)(2025), IEEE, pp
Fu, V., Benazouz, M., Zaourar, L., and Munier-Kordon, A.High-performance computing architecture exploration with stage-enhanced bayesian optimization. In2025 62nd ACM/IEEE Design Automation Conference (DAC)(2025), IEEE, pp. 1–7
work page 2025
-
[14]
Honorat, A., Bourgoin, T., Miomandre, H., Desnos, K., Menard, D., and Nezan, J.-F.Influence of Dataflow Graph Moldable Parameters on Optimization Criteria. InDASIP 2022 - Workshop on Design and Architectures for Signal and Image Processing(Budapest, Hungary, June 2022), vol. 13425 ofLecture Notes in Computer Science, Springer International Publishing, pp. 83–95
work page 2022
-
[15]
Hurley-Walker, N., Callingham, J. R., Hancock, P. J., Franzen, T. M. O., Hindson, L., Kapińska, A. D., Morgan, J., Offringa, A. R., W ayth, R. B., Wu, C., Zheng, Q., Murphy, T., Bell, M. E., Dwarakanath, K. S., For, B., Gaensler, B. M., Johnston-Hollitt, M., Lenc, E., Procopio, P., Staveley-Smith, L., Ekers, R., Bowman, J. D., Briggs, F., Cappallo, R. J.,...
work page 2017
-
[16]
Knödlseder, J., Brau-Nogué, S., Coriat, M., Garnier, P., Hughes, A., Martin, P., and Tibaldo, L.Estimate of the carbon footprint of astronomical research infrastructures.Nature Astronomy 6, 4 (2022), 503–513
work page 2022
-
[17]
Lenovo. ThinkSystem SR675 V3 Server. https://www.lenovo.com/us/en/p/ servers-storage/servers/inferencing/thinksystem-sr675-v3/len21ts0007, 2024. Ac- cessed: December 2025
work page 2024
-
[18]
Monnier, N., Raffin, E., Tasse, C., Nezan, J.-F., and Smirnov, O. M.Parallelisa- tion of the wide-band wide-field spectral deconvolution framework ddfacet on distributed memory hpc system. InADASS(2020)
work page 2020
-
[19]
J., Dulwich, F., Salvini, S., Adami, K
Mort, B. J., Dulwich, F., Salvini, S., Adami, K. Z., and Jones, M. E.Oskar: Sim- ulating digital beamforming for the ska aperture array. In2010 IEEE International Symposium on Phased Array Systems and Technology(2010), pp. 690–694
work page 2010
-
[20]
HGX H100 Product Carbon Footprint Summary
NVIDIA. HGX H100 Product Carbon Footprint Summary. https://images.nvidia. com/aem-dam/Solutions/documents/HGX-H100-PCF-Summary.pdf, 2023. Ac- cessed: December 2025
work page 2023
-
[21]
R., McKinley, B., Hurley-W alker, N., Briggs, F
Offringa, A. R., McKinley, B., Hurley-W alker, N., Briggs, F. H., W ayth, R. B., and Kaplan, D. L.Wsclean: an implementation of a fast, generic wide-field imager for radio astronomy.Monthly Notices of the Royal Astronomical Society 444, 1 (2014), 606–619
work page 2014
-
[22]
Portegies Zwart, S.The ecological impact of high-performance computing in astrophysics.Nature Astronomy 4, 9 (2020), 819–822
work page 2020
-
[23]
Pratley, L., Johnston-Hollitt, M., and McEwen, J. D.A fast and exact w-stacking and w-projection hybrid algorithm for wide-field interferometric imaging.The Astrophysical Journal 874, 2 (2019), 174
work page 2019
-
[24]
Reddi, V. J., Cheng, C., Kanter, D., Mattson, P., Schmuelling, G., Wu, C.-J., Anderson, B., Breughe, M., Charlebois, M., Chou, W., et al.Mlperf inference benchmark. In2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA)(2020), IEEE, pp. 446–459
work page 2020
-
[25]
InInternational Symposium on Field-Programmable Custom Computing Machines (FCCM)(May 2023), pp
Sarkar, R., and Hao, C.LightningSim: Fast and Accurate Trace-Based Simulation for High-Level Synthesis. InInternational Symposium on Field-Programmable Custom Computing Machines (FCCM)(May 2023), pp. 1–11
work page 2023
-
[26]
https://scitas-doc.epfl.ch/supercomputers/kuma/
SCITAS, E.Kuma cluster. https://scitas-doc.epfl.ch/supercomputers/kuma/. Accessed: 2025-11-27
work page 2025
-
[27]
astroCAMP: A Framework for Cross-Layer Co-Design of Radio Astronomy Imaging Pipelines
SEAMS Project. astroCAMP: A Framework for Cross-Layer Co-Design of Radio Astronomy Imaging Pipelines. https://github.com/SEAMS-Project/astroCAMP,
-
[28]
Accessed: 2025-12-13
work page 2025
-
[29]
Environmental footprint | skao, 2024
SKAO Communications. Environmental footprint | skao, 2024. Details SKAO’s goals to measure, monitor, and minimise environmental impact and use renewable energy
work page 2024
-
[30]
Sustainability at the skao, 2024
SKAO Communications. Sustainability at the skao, 2024. Outlines SKAO’s commitment to sustainability and alignment with UN SDGs
work page 2024
-
[31]
Suriano, L., Rodriguez, A., Desnos, K., Pelcat, M., and de la Torre, E.Analysis of a heterogeneous multi-core, multi-hw-accelerator-based system designed using preesm and sdsoc. In2017 12th International Symposium on Reconfigurable Communication-centric Systems-on-Chip (ReCoSoC)(2017), pp. 1–7
work page 2017
-
[32]
Tasse, C., Hugo, B., Mirmont, M., Smirnov, O., Atemkeng, M., Bester, L., Hardcastle, M., Lakhoo, R., Perkins, S., and Shimwell, T.Ddfacet: Facet- based radio imaging package.Astrophysics Source Code Library(2023), ascl–2305
work page 2023
-
[33]
Tolley, E., Frasch, S., Orliac, E., Krishna, S., Bianco, M., Kashani, S., Hurley, P., Simeoni, M., and Kneib, J.-P.Bipp: An efficient hpc implementation of the bluebild algorithm for radio astronomy.Astronomy and Computing 51(2025), 100920
work page 2025
-
[34]
TOP500.org. Green500 list — november 2025. https://top500.org/lists/green500/ 2025/11/, 2025. Accessed: December 2025
work page 2025
-
[35]
V an der Tol, S., Veenboer, B., and Offringa, A. R.Image domain gridding: a fast method for convolutional resampling of visibilities.Astronomy & Astrophysics 616(2018), A27
work page 2018
-
[36]
V an der Tol, S., Veenboer, B., and Offringa, A. R.Image domain gridding: A fast method for convolutional resampling of visibilities.Astronomy & Astrophysics 616(2018), A27
work page 2018
-
[37]
W.Radio-astronomical imaging on graphics processors.Astronomy and Computing 32(2020), 100386
Veenboer, B., and Romein, J. W.Radio-astronomical imaging on graphics processors.Astronomy and Computing 32(2020), 100386
work page 2020
-
[38]
Wu, S., Xie, Y., W ang, F., Xu, Y., Deng, H., Mei, Y., Lü, Y.-H. C., Hodosán, G., and Zhu, Y.Performance comparison of source finders in imaging quality assessment for ska1-low.The Astronomical Journal 170, 6 (2025), 308
work page 2025
-
[39]
In2023 26th Euromicro Conference on Digital System Design (DSD)(2023), IEEE, pp
Zaourar, L., Chillet, A., and Philippe, J.-M.A-deca: an automated design space exploration approach for computing architectures to develop efficient high- performance many-core processors. In2023 26th Euromicro Conference on Digital System Design (DSD)(2023), IEEE, pp. 756–763
work page 2023
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.