Recognition: unknown
A public dataset of Ariel simulated observations for developing exoplanetary atmosphere data reduction pipelines
Pith reviewed 2026-05-07 13:12 UTC · model grok-4.3
The pith
A public dataset of simulated Ariel mission observations provides ground-truth benchmarks for developing and testing exoplanet atmosphere data reduction pipelines, including a neural-network baseline that exposes risks from dataset shift.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The authors produce a large-scale public dataset of Ariel-simulated transit observations that includes both the planetary atmospheric signals and the full range of instrumental and astrophysical noise expected from the mission. They pair this with a deep neural network baseline for detrending and show that the network's performance drops when the test data distribution diverges from the training distribution, illustrating the practical limits of purely data-driven methods on future Ariel spectra.
What carries the argument
The simulated dataset itself, generated by ExoSim2 for instrument effects and TauREx for atmospheric models, together with the provided deep neural network baseline for time-series reduction.
If this is right
- Algorithms developed and validated on the dataset can be applied directly to Ariel survey data with greater confidence in their robustness.
- The explicit demonstration of dataset-shift failure provides a concrete test that any new machine-learning detrending method must pass before deployment.
- The resource enables direct head-to-head comparison of classical and data-driven detrending techniques on identical inputs with known ground truth.
- Community-wide stress-testing on the dataset can identify which methods scale reliably to the full Ariel sample of roughly one thousand planets.
Where Pith is reading between the lines
- The same simulation framework and shift-testing approach could be adapted to benchmark pipelines for JWST or future missions such as Habitable Worlds Observatory.
- If real Ariel data exhibit stronger or different systematics than the current simulations, retraining the baseline network on a mixture of simulated and early flight data would be a natural next step.
- The dataset could also serve as a controlled testbed for studying how stellar variability or spot-crossing events interact with atmospheric retrievals.
Load-bearing premise
The simulated noise, systematics, and signal properties accurately reproduce what will appear in real Ariel observations.
What would settle it
Once Ariel flight data are available, compare the statistical distributions and residual noise properties of the real spectra against the simulated dataset; large, systematic mismatches would show that the simulations do not capture the actual instrument behavior.
Figures
read the original abstract
Detecting and characterising exoplanet atmospheres remains challenging because atmospheric signals can be comparable to residual noise and instrumental/astrophysical systematics. Spectral features span from a few ppm for small planets up to $\sim 10^3$ ppm for warm/hot giants, while high-quality JWST time-series spectroscopy typically reaches $\sim 10$--$50$ ppm (occasionally $\sim 100$--$200$ ppm in the presence of stellar variability or stronger systematics), making correlated noise across temporal and spectral dimensions a key limitation. With JWST delivering an increasing volume of high-precision transmission spectra, and Ariel set to extend this to a homogeneous survey of $\sim 10^3$ exoplanet atmospheres, robust benchmarking resources with known ground truth are essential to develop and validate data-driven (including ML-based) detrending approaches. As a major step towards this goal, we use ExoSim2 and TauREx to generate one of the most comprehensive public datasets based on the current payload design of the ESA Ariel mission, specifically intended to benchmark detrending algorithms. We also provide a deep neural network baseline for time-series reduction, and use it to highlight the limitations of ML based detrendng methods, i.e. the risks posed by dataset shift when observed distributions diverge from those of the training set, a scenario likely to arise in real observations. This dataset is featured in the Ariel Data Challenge 2024 on Kaggle and has been field-tested for robustness and simulation fidelity. By making these resources publicly available, we aim to support the community in developing, comparing, and stress-testing scalable and reliable methods for exoplanet transmission spectroscopy.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents a public dataset of simulated Ariel exoplanet observations generated with ExoSim2 and TauREx based on the current mission payload design. The dataset is intended to support development and benchmarking of detrending and data reduction pipelines for transmission spectroscopy. The authors also supply a deep neural network baseline for time-series reduction and use it to illustrate the performance degradation caused by dataset shift when test distributions differ from the training set.
Significance. If the simulations faithfully reproduce Ariel's expected noise properties and systematics, the dataset will serve as a key community resource for validating algorithms ahead of the mission's survey of approximately 1000 atmospheres. The DNN baseline provides a concrete, reproducible demonstration of dataset-shift risks that is directly relevant to real observations, where training and test distributions are unlikely to match perfectly. Public release through the Ariel Data Challenge 2024 on Kaggle increases the work's immediate utility for comparative testing of detrending methods.
minor comments (3)
- Abstract: 'detrendng' is a typographical error and should read 'detrending'.
- Abstract and introduction: the statement that the dataset is 'one of the most comprehensive' would benefit from a brief quantitative comparison (e.g., number of planets, spectral channels, or total simulated hours) to previously released Ariel or JWST simulation suites.
- The manuscript would be strengthened by an explicit statement of the dataset access URL, file formats, and any accompanying documentation or metadata files in the main text rather than only in supplementary material.
Simulated Author's Rebuttal
We thank the referee for their positive assessment of the manuscript, recognition of the dataset's utility for the Ariel mission community, and recommendation to accept. We are pleased that the work's relevance for benchmarking detrending pipelines and illustrating dataset-shift risks in ML-based methods was acknowledged.
Circularity Check
No significant circularity in derivation chain
full rationale
This is a data-release paper whose core contribution is the generation and public provision of simulated Ariel observations using the established external tools ExoSim2 and TauREx, together with a DNN baseline trained and evaluated inside the same simulation framework. No equations, fitted parameters, or predictions are presented that reduce to the inputs by construction. The dataset-shift demonstration is a controlled internal experiment within the simulated distributions and does not rely on self-definitional steps, load-bearing self-citations, or renaming of known results. All claims rest on the explicit provision of the dataset and code, which are externally verifiable and independent of any internal derivation chain.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
JWST-TST DREAMS: NIRSpec/PRISM Transmission Spectroscopy of the Habitable Zone Planet TRAPPIST-1 e. , keywords =. doi:10.3847/2041-8213/adf42e , archivePrefix =. 2509.05414 , primaryClass =
-
[2]
JWST NIRISS transmission spectroscopy of the super-Earth GJ 357b, a favourable target for atmospheric retention. , keywords =. doi:10.1093/mnras/staf894 , archivePrefix =. 2505.24462 , primaryClass =
-
[3]
A benchmark JWST near-infrared spectrum for the exoplanet WASP-39 b. Nature Astronomy , keywords =. doi:10.1038/s41550-024-02292-x , archivePrefix =. 2407.13893 , primaryClass =
-
[4]
2023, Monthly Notices of the Royal Astronomical Society, 524, 835, doi: 10.1093/mnras/stad1762
Awesome SOSS: transmission spectroscopy of WASP-96b with NIRISS/SOSS. , keywords =. doi:10.1093/mnras/stad1762 , archivePrefix =. 2305.17001 , primaryClass =
-
[5]
C., Weiner Mansfield, M., Cubillos, P
Horizontal and vertical exoplanet thermal structure from a JWST spectroscopic eclipse map. Nature Astronomy , keywords =. doi:10.1038/s41550-025-02666-9 , archivePrefix =. 2510.24708 , primaryClass =
-
[6]
2018, Experimental Astronomy, 46, 135, doi: 10.1007/s10686-018-9598-x
A chemical survey of exoplanets with ARIEL. Experimental Astronomy , keywords =. doi:10.1007/s10686-018-9598-x , adsurl =
-
[7]
Characterizing Transiting Exoplanet Atmospheres with JWST. , keywords =. doi:10.3847/0004-637X/817/1/17 , archivePrefix =. 1511.05528 , primaryClass =
-
[8]
MIRI-LRS Spectrum of a Cold Exoplanet around a White Dwarf: Water, Ammonia, and Methane Measurements
MIRI-LRS Spectrum of a Cold Exoplanet around a White Dwarf: Water, Ammonia, and Methane Measurements. , keywords =. doi:10.3847/2041-8213/adbd46 , archivePrefix =. 2503.04531 , primaryClass =
-
[9]
Integrating Light Curve and Atmospheric Modeling of Transiting Exoplanets. , keywords =. doi:10.3847/1538-3881/abaabc , archivePrefix =. 1811.04686 , primaryClass =
-
[10]
Toward Atmospheric Retrievals of Panchromatic Light Curves: EXPLOR-ing Generalized Inversion Techniques for Transiting Exoplanets with JWST and Ariel. , keywords =. doi:10.3847/1538-3881/ad3032 , archivePrefix =. 2403.02244 , primaryClass =
-
[11]
ExoMol line lists - LIV. Empirical line lists for AlH and AlD and experimental emission spectroscopy of AlD in A ^ 1 (v = 0, 1, 2). , keywords =. doi:10.1093/mnras/stad3802 , archivePrefix =. 2312.05958 , primaryClass =
-
[12]
RAS Techniques and Instruments , keywords =
Simple semiclassical model of pressure-broadened infrared/microwave linewidths in the temperature range 200-3000 K. RAS Techniques and Instruments , keywords =. doi:10.1093/rasti/rzac004 , adsurl =
-
[13]
S., Tennyson, J., Yurchenko, S
ExoMol molecular line lists - XIV. The rotation-vibration spectrum of hot SO _ 2. , keywords =. doi:10.1093/mnras/stw849 , archivePrefix =. 1603.04065 , primaryClass =
-
[14]
Experimental Astronomy , keywords =
De-jittering Ariel: An optimized algorithm. Experimental Astronomy , keywords =. doi:10.1007/s10686-025-09999-3 , archivePrefix =. 2504.12907 , primaryClass =
-
[15]
Spectroscopy of planetary atmospheres in our Galaxy. , keywords =. doi:10.1007/s00159-013-0063-6 , adsurl =
-
[16]
Journal of Astronomical Telescopes, Instruments, and Systems , keywords =
Modeling phase variations introduced by extreme broadband dichroics for astronomical photometry. Journal of Astronomical Telescopes, Instruments, and Systems , keywords =. doi:10.1117/1.JATIS.11.1.014003 , adsurl =
-
[17]
and Bocchieri, Andrea and Pascale, Enzo and Lorenzani, Andrea and Papageorgiou, Andreas , title =
Mugnai, Lorenzo V. and Bocchieri, Andrea and Pascale, Enzo and Lorenzani, Andrea and Papageorgiou, Andreas , title =. Experimental Astronomy , year =. doi:10.1007/s10686-024-09976-2 , url =
-
[18]
Advances in neural information processing systems , volume=
The marginal value of adaptive gradient methods in machine learning , author=. Advances in neural information processing systems , volume=
-
[19]
International conference on machine learning , pages=
Train faster, generalize better: Stability of stochastic gradient descent , author=. International conference on machine learning , pages=. 2016 , organization=
2016
-
[20]
Comparing transit spectroscopy pipelines at the catalogue level: evidence for systematic differences. , keywords =. doi:10.1093/mnras/stae1073 , archivePrefix =. 2402.15803 , primaryClass =
-
[21]
Exoplanet transit spectroscopy with JWST NIRSpec: diagnostics and homogeneous case study of WASP-39 b. , keywords =. doi:10.1093/mnras/stae1230 , archivePrefix =. 2405.06737 , primaryClass =
-
[22]
Early Release Science of the exoplanet WASP-39b with JWST NIRCam. , keywords =. doi:10.1038/s41586-022-05590-4 , archivePrefix =. 2211.10489 , primaryClass =
-
[23]
ExoMol molecular line lists XXX: a complete high-accuracy line list for water. , keywords =. doi:10.1093/mnras/sty1877 , archivePrefix =. 1807.04529 , primaryClass =
-
[24]
, year = 2010, month = oct, volume =
HITEMP, the high-temperature molecular spectroscopic database. , year = 2010, month = oct, volume =. doi:10.1016/j.jqsrt.2010.05.001 , adsurl =
-
[25]
A hybrid line list for CH _ 4 and hot methane continuum. , keywords =. doi:10.1051/0004-6361/201731026 , archivePrefix =. 1706.05724 , primaryClass =
-
[26]
ExoMol molecular line lists - XXXV. A rotation-vibration line list for hot ammonia. , keywords =. doi:10.1093/mnras/stz2778 , archivePrefix =. 1911.10369 , primaryClass =
-
[27]
L., Tennyson, J., & Yurchenko, S
ExoMol molecular line lists - XXXVII. Spectra of acetylene. , keywords =. doi:10.1093/mnras/staa229 , archivePrefix =. 2001.04550 , primaryClass =
-
[28]
” Rothman et al. (2010) Polyansky et al. (2018) “
ExoMol line lists - III. An improved hot rotation-vibration line list for HCN and HNC. , keywords =. doi:10.1093/mnras/stt2011 , archivePrefix =. 1311.1328 , primaryClass =
-
[29]
Rovibrational Line Lists for Nine Isotopologues of the CO Molecule in the X ^ 1 ^ + Ground Electronic State. , keywords =. doi:10.1088/0067-0049/216/1/15 , adsurl =
-
[30]
Azzam, Ala'a A. A. and Tennyson, Jonathan and Yurchenko, Sergei N. and Naumenko, Olga V. , title = ". Monthly Notices of the Royal Astronomical Society , volume =. 2016 , month =. doi:10.1093/mnras/stw1133 , url =
-
[31]
Early Release Science of the exoplanet WASP-39b with JWST NIRSpec PRISM. , keywords =. doi:10.1038/s41586-022-05677-y , archivePrefix =. 2211.10487 , primaryClass =
-
[32]
Early Release Science of the exoplanet WASP-39b with JWST NIRSpec G395H. , keywords =. doi:10.1038/s41586-022-05591-3 , archivePrefix =. 2211.10488 , primaryClass =
-
[33]
D., Radica, M., Welbanks, L., et al
Early Release Science of the exoplanet WASP-39b with JWST NIRISS. , keywords =. doi:10.1038/s41586-022-05674-1 , archivePrefix =. 2211.10493 , primaryClass =
-
[34]
2023, Monthly Notices of the Royal Astronomical Society, 524, 377, doi: 10.1093/mnras/stad1580
Exoplanet spectroscopy with JWST NIRISS: diagnostics and case studies. , keywords =. doi:10.1093/mnras/stad1580 , archivePrefix =. 2306.04676 , primaryClass =
-
[35]
Detection of an Atmosphere on a Rocky Exoplanet. , keywords =. doi:10.3847/1538-3881/abe879 , archivePrefix =. 2103.05657 , primaryClass =
-
[36]
The Featureless HST/WFC3 Transmission Spectrum of the Rocky Exoplanet GJ 1132b: No Evidence for a Cloud-free Primordial Atmosphere and Constraints on Starspot Contamination. , keywords =. doi:10.3847/1538-3881/ac75de , archivePrefix =. 2105.10487 , primaryClass =
-
[37]
ARES. V. No Evidence For Molecular Absorption in the HST WFC3 Spectrum of GJ 1132 b. , keywords =. doi:10.3847/1538-3881/abf3c3 , archivePrefix =. 2104.01873 , primaryClass =
-
[38]
NeurIPS 2024 Competition Track , year=
Ariel Data Challenge 2024: Extracting exoplanetary signals from the Ariel Space Telescope , author=. NeurIPS 2024 Competition Track , year=
2024
-
[39]
The Impact of Cosmic Rays on the Sensitivity of JWST/NIRSpec. , keywords =. doi:10.1088/1538-3873/ab2fd6 , archivePrefix =. 1907.04051 , primaryClass =
-
[40]
New evolutionary models for pre-main sequence and main sequence low-mass stars down to the hydrogen-burning limit , volume =
Baraffe, Isabelle and Homeier, Derek and Allard, France and Chabrier, Gilles , doi =. New evolutionary models for pre-main sequence and main sequence low-mass stars down to the hydrogen-burning limit , volume =. Astronomy
-
[41]
V., Pascale, E., Edwards, B., Papageorgiou, A., & Sarkar, S
ArielRad: the Ariel radiometric model. Experimental Astronomy , keywords =. doi:10.1007/s10686-020-09676-7 , archivePrefix =. 2009.07824 , primaryClass =
-
[42]
TauREx 3: A Fast, Dynamic, and Extendable Framework for Retrievals. , keywords =. doi:10.3847/1538-4357/ac0252 , archivePrefix =. 1912.07759 , primaryClass =
-
[43]
Mugnai and Enzo Pascale , title =
Andrea Bocchieri and Lorenzo V. Mugnai and Enzo Pascale , title =. Space Telescopes and Instrumentation 2024: Optical, Infrared, and Millimeter Wave , editor =. 2024 , doi =
2024
-
[44]
Observational Techniques With Transiting Exoplanetary Atmospheres. arXiv e-prints , keywords =. doi:10.48550/arXiv.1804.07357 , archivePrefix =. 1804.07357 , primaryClass =
-
[45]
Subpixel Response Measurement of Near‐Infrared Detectors
Subpixel Response Measurement of Near-Infrared Detectors. , keywords =. doi:10.1086/517620 , archivePrefix =. astro-ph/0611339 , primaryClass =
-
[46]
EPSC-DPS Joint Meeting 2019 , year = 2019, volume =
An Updated Study of Potential Targets for Ariel. EPSC-DPS Joint Meeting 2019 , year = 2019, volume =
2019
-
[47]
SIAM Journal on Imaging Sciences , volume=
Image denoising: The deep learning revolution and beyond—a survey paper , author=. SIAM Journal on Imaging Sciences , volume=. 2023 , publisher=
2023
-
[48]
Neurocomputing , volume=
A survey of deep learning approaches to image restoration , author=. Neurocomputing , volume=. 2022 , publisher=
2022
-
[49]
arXiv preprint arXiv:2309.00810 , year=
RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model , author=. arXiv preprint arXiv:2309.00810 , year=
-
[50]
IEEE Transactions on Artificial Intelligence , year=
Transformer-based generative adversarial networks in computer vision: A comprehensive survey , author=. IEEE Transactions on Artificial Intelligence , year=
-
[51]
arXiv preprint arXiv:2303.07909 , year=
Text-to-image diffusion model in generative ai: A survey , author=. arXiv preprint arXiv:2303.07909 , year=
-
[52]
Learning to denoise astronomical images with U-nets. , keywords =. doi:10.1093/mnras/staa3567 , archivePrefix =. 2011.07002 , primaryClass =
-
[53]
F., Valtchanov, I., Lieu, M., et al
Deep learning-based super-resolution and de-noising for XMM-newton images. , keywords =. doi:10.1093/mnras/stac2437 , archivePrefix =. 2205.01152 , primaryClass =
-
[54]
Deeper, Sharper, Faster: Application of Efficient Transformer to Galaxy Image Restoration. arXiv e-prints , keywords =. doi:10.48550/arXiv.2404.00102 , archivePrefix =. 2404.00102 , primaryClass =
-
[55]
The Astronomical Journal , keywords =
Random Forests Applied to High-precision Photometry Analysis with Spitzer IRAC. The Astronomical Journal , keywords =. doi:10.3847/1538-3881/aba11f , archivePrefix =. 2006.14095 , primaryClass =
-
[56]
The Astronomical Journal , keywords =
Repeatability and Accuracy of Exoplanet Eclipse Depths Measured with Post-cryogenic Spitzer. The Astronomical Journal , keywords =. doi:10.3847/0004-6256/152/2/44 , archivePrefix =. 1601.05101 , primaryClass =
-
[57]
RAS Techniques and Instruments , volume=
Lessons learned from the 1st Ariel Machine Learning Challenge: Correcting transiting exoplanet light curves for stellar spots , author=. RAS Techniques and Instruments , volume=. 2023 , publisher=
2023
-
[58]
The Astronomical Journal , keywords =
Detrending Exoplanetary Transit Light Curves with Long Short-term Memory Networks. The Astronomical Journal , keywords =. doi:10.3847/1538-3881/ab6aa7 , archivePrefix =. 2001.03370 , primaryClass =
-
[59]
Machine Learning for Astrophysics , year = 2022, month = jul, eid =
Don't Pay Attention to the Noise: Learning Self-supervised Representations of Light Curves with a Denoising Time Series Transformer. Machine Learning for Astrophysics , year = 2022, month = jul, eid =. doi:10.48550/arXiv.2207.02777 , archivePrefix =. 2207.02777 , primaryClass =
-
[60]
2016 , eprint=
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , author=. 2016 , eprint=
2016
-
[61]
and Changeat, Quentin and Morvan, Mario and Al-Refaie, Ahmed F
Yip, Kai Hou and Waldmann, Ingo P. and Changeat, Quentin and Morvan, Mario and Al-Refaie, Ahmed F. and Edwards, Billy and Nikolaou, Nikolaos and Tsiaras, Angelos and de Oliveira, Catarina Alves and Lagage, Pierre-Olivier and Jenner, Clare and Cho, James Y-K. and Thiyagalingam, Jeyan and Tinetti, Giovanna , year =. doi:10.48550/ARXIV.2206.14642 , abstract =
-
[62]
RAS Techniques and Instruments , author =. 2023 , pages =. doi:10.1093/rasti/rzad001 , abstract =
-
[63]
A Comprehensive Reanalysis of K2-18 b's JWST NIRISS+NIRSpec Transmission Spectrum. , keywords =. doi:10.3847/1538-3881/ae019a , archivePrefix =. 2501.18477 , primaryClass =
-
[64]
Research Notes of the American Astronomical Society , keywords =
Are There Spectral Features in the MIRI/LRS Transmission Spectrum of K2-18b?. Research Notes of the American Astronomical Society , keywords =. doi:10.3847/2515-5172/add881 , archivePrefix =. 2504.15916 , primaryClass =
-
[65]
and McGill, Peter and Tilke, Lana J
Challenges in the detection of gases in exoplanet atmospheres. Nature Astronomy , keywords =. doi:10.1038/s41550-025-02730-4 , archivePrefix =. 2504.21788 , primaryClass =
-
[66]
Accurate Empirical Radii and Masses of Planets and Their Host Stars with Gaia Parallaxes. , keywords =. doi:10.3847/1538-3881/aa5df3 , archivePrefix =. 1609.04389 , primaryClass =
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.