{"id":"18d8f984-7f07-4c59-aa25-d8c4af4cbf89","arxiv_id":"2501.12809","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"ExoSim 2 is a modular, publicly available rewrite of the Exoplanet Observation Simulator that produces time-resolved focal planes, jittered sub-exposures, and noisy non-destructive reads for Ariel-like observations.","lead":"This paper introduces ExoSim 2, an open-source Python simulator that models how exoplanet transits look through the optics, jitter, and detectors of space telescopes such as Ariel. It is aimed at mission planners and data analysts who need realistic synthetic observations to test pipelines before real data arrives.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central validation rests on an unpublished read-out simulator and an undescribed <1 ppm comparison; if sequential readout interacts with the jitter timeline, the simulated NDR noise is not established as realistic.","rationale":"The paper is a credible software and methods contribution: the public GitHub repository, automated tests, benchmark results, and the batman-based astronomical signal validation provide real, inspectable support for the simulator's existence and basic functionality. The central claim, however, is that ExoSim 2 produces realistic time-resolved synthetic observations whose per-pixel signals and noise match ArielRad within stated thresholds. The most load-bearing part of that claim is the end-to-end noise validation. That validation is supported by three pillars: the ArielRad consistency checks, the instantaneous-readout/jitter validation, and the astronomical signal validation. The astronomical signal validation is well-documented. The ArielRad consistency checks are described with specific thresholds (1%, 5%, 1 ppm), but the 1 ppm result is attributed to undisclosed independent pipelines and is not reproducible from the paper alone. The instantaneous-readout validation is the least inspectable: the decisive comparison with a sequential read-out simulator is deferred to a separate paper, with no numbers or figures. The restricted B2 payload configurations further limit independent reproduction. The reader's weakest assumption identified exactly this gap, and I agree. The correct verdict remains CONDITIONAL: the paper should not be rejected, because the code and most checks are public, but the headline quantitative validation should not be taken as fully established until the read-out comparison and the 1 ppm methodology are made available. My proposed test directly targets the approximation that, if wrong, would invalidate the realism of the NDR noise output.","tokens_in":22775,"tokens_out":5639,"duration_ms":57202,"concrete_test":"Request (or, if feasible, independently implement) a sequential-readout mode in ExoSim 2 that shifts the pointing jitter timeline per pixel row according to the detector readout order, with tread approximately 0.1 s. Rerun the Section 4 CDS observation (FGS1, HD 209458, 8 hr, same seed and jitter timeline) and compare the resulting NDR residual RMS to the instantaneous-readout run. If the difference exceeds the photon-noise floor (or the stated 1 ppm threshold), the instantaneous-readout approximation is not validated for Ariel.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's strongest quantitative claim—that ExoSim 2 noise estimates agree with ArielRad to <1 ppm for a 10 hr HD 209458-like observation (Abstract, Section 3.1)—and its justification for the instantaneous-readout approximation both depend on evidence not present in the manuscript. In Section 3.2, the authors state that a separate read-out simulator (to be published in a separate paper) was used to compare sequential and instantaneous readout across 'various realistic jitter timelines' and found differences 'negligible compared to the photon noise,' but no data, figure, or error budget is shown. In Section 3.1, the <1 ppm result is attributed to 'different independent data reduction pipelines' without specifying the pipelines, the metric, or the noise model. In Section 4, the detrending analysis is deferred to Bocchieri et al. (2024, in preparation). The B2-phase payload configurations that define the simulated Ariel channels are stored in a restricted-access repository (Data Availability). Each of these gaps individually would be acceptable for a software paper; together they form the sole support for the headline validation. If sequential readout and jitter timelines interact at the NDR cadence in a way the unpublished simulator does not capture—for example, if jitter power at the row-read time scale (~0.1 s) is non-negligible—the NDR noise generated by ExoSim 2 would be biased, and the <1 ppm agreement would not be a valid demonstration of realism. The paper's own Section 5 limitation (no slow pointing drift) correctly scopes the jitter model to fast jitter, but does not resolve the readout-timing question.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces ExoSim 2, a complete Python 3 rewrite of the Exoplanet Observation Simulator, designed for spectro-photometric time-series observations of transiting exoplanets and applied to the Ariel space mission. The authors describe the modular Task-based architecture, the Signal data-cube class, and a three-step workflow that generates focal planes, sub-exposures with pointing jitter, and non-destructive reads (NDRs) with detector noise. Validation is presented against the ArielRad radiometric model for photon conversion efficiency, saturation time, signal levels, and a claimed sub-ppm noise comparison; against an analytic Gaussian-blur/jitter model; and against batman-generated transit signals. The paper also reports computational benchmarks and an example use case comparing jitter and no-jitter light curves.","tokens_in":22985,"tokens_out":4293,"duration_ms":45809,"significance":"If the claims are substantiated, ExoSim 2 is a valuable community resource: it is open source, modular, user-extensible, memory-efficient for large time-series data cubes, and accompanied by automated regression tests and documentation. The analytical Gaussian-jitter test and the noise-free batman comparison are appropriate internal consistency checks, and the benchmark results on thread scaling are useful for users planning large simulations. The main external validation, however, is a same-heritage comparison with ArielRad, so the paper's quantitative claims need to be presented with enough detail to be reproduced and assessed.","major_comments":[{"comment":"The <1 ppm noise-difference claim is the strongest quantitative validation in the paper, but it is not supported by the manuscript as written. The text says that 'different independent data reduction pipelines' were developed and that differences between ExoSim 2 and ArielRad noise estimates are below 1 ppm for a 10 hr HD 209458-like observation, yet it does not define the noise metric (per-pixel, per spectral bin, or broadband), identify which channels and readout modes were used, specify how jitter, photon noise, and read noise were included, or describe what the pipelines did. Please provide a figure or table of the comparison, define the metric exactly, and state the noise model and reduction steps; otherwise this central claim cannot be reproduced or independently assessed.","section":"§3.1"},{"comment":"The instantaneous-readout approximation is load-bearing for the NDR simulation, but the only support is a statement that a separate, unpublished read-out simulator found sequential versus instantaneous readout differences 'negligible compared to the photon noise' across 'various realistic jitter timelines.' No jitter power spectra, readout cadences, quantitative differences, or thresholds are given. Because the detector row read time (~0.1 s) is close to the NDR sub-exposure timescale, the interaction of jitter power at ~0.1 s with sequential readout is precisely the regime that matters. Please include a representative comparison figure, describe the jitter timelines tested, and specify the criterion used to define 'negligible.'","section":"§3.2"},{"comment":"The B2-phase payload configurations that define the simulated Ariel channels are stored in a restricted-access repository, so the central validation cannot be reproduced by an external user despite the paper's stated goal of reproducibility. Given that this is a software paper whose validation depends on those configurations, I ask that the authors provide an anonymized or reduced version of the configuration used for the ArielRad and batman comparisons, or at least a machine-readable summary of the key optical and detector parameters, so that the validation is repeatable by the community.","section":"Data Availability"}],"minor_comments":[{"comment":"There is a missing space in 'The currentExoSim 2' at the beginning of the last paragraph of Section 2.1.","section":"§2.1"},{"comment":"There is a formatting error in 'float64 values8 (64 bits = 8 bytes)'; the '8' should be separated as '(64 bits = 8 bytes)'.","section":"§2.2"},{"comment":"The target name is written as 'HD 209858 b' in the first paragraph; this should be 'HD 209458 b'.","section":"§3.3"},{"comment":"The row for 40 threads appears twice in Table 1; the duplicate row should be removed.","section":"Table 1"},{"comment":"In Equation (5), the product symbol is rendered as 'Q' in the text; please use the proper ∏ notation.","section":"Equation (5)"},{"comment":"The paper correctly states in Section 5 that the high-frequency module cannot simulate slow pointing drift, but this limitation is important enough to be highlighted in Section 2.2 as well, since users setting up jitter simulations need to know the applicable regime before running the code.","section":"§5"}],"recommendation":"major_revision","confidential_remarks":"This is a solid software-paper submission with an open-source codebase and useful automated tests. My main concern is that the headline numerical claims, especially the <1 ppm noise comparison in §3.1 and the instantaneous-readout justification in §3.2, rest on unpublished or undescribed validation apparatus. These are fixable with added figures, defined metrics, and a clear description of the validation setup. The restricted data availability is a separate reproducibility concern that should be addressed for a journal audience. I do not see a fatal flaw, but the load-bearing validation details need to be provided before the paper can be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"ExoSim 2 is a genuine software contribution: a from-scratch Python rewrite with Task classes, chunked HDF5 storage, an oversampled-shift jitter algorithm, and an astronomical-signal injection module. The code is public, automated tests exist with good coverage, and the benchmarks show reasonable scaling. Validation is mostly consistency checking — against ArielRad, an analytic Gaussian blur, and batman — and for a simulator paper that is the right kind of evidence. I think the central claim that the code behaves as intended holds up.\n\nSoft spots are concentrated in the validation sections. The abstract's <1 ppm noise agreement with ArielRad is too thin: we are told 'different independent data reduction pipelines' were used, but not which, using what metric, or with what uncertainties. That should be pinned down. The instantaneous-readout approximation rests on a separate read-out simulator that is not in this paper and not yet published; the authors say the differences are negligible compared to photon noise, but show no numbers. The B2-phase payload configurations are in a restricted repository, so the exact figures are not reproducible. Section 5 is admirably candid about missing slow drift, persistence, 1/f, and crosstalk.\n\nThe stress-test concern about sequential readout interacting with jitter at the NDR cadence is plausible but not, on reading, a demonstrated flaw. The authors have at least looked at the question; the problem is that the evidence is invisible. That is a reporting gap, not a load-bearing error. I would not call the paper circular: comparing against ArielRad is a heritage check, and the agreement with the analytic Gaussian and batman gives some independence.\n\nBottom line: this is a paper for the exoplanet instrumentation community, and it deserves a serious referee. I would send it to review, with the expectation that the <1 ppm claim gets concrete detail, the read-out simulator results get shown or cited, and the restricted payload data gets a synthetic substitute. Once those are in place, it is a solid, citable software paper.","headline":"ExoSim 2 is a real, useful software contribution with public code and sensible consistency checks; the main gaps are a vague <1 ppm claim and an unpublished read-out simulator, both fixable.","tokens_in":23647,"tokens_out":2021,"would_cite":true,"duration_ms":19748,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper presents ExoSim 2, a modular, publicly available exoplanet observation simulator that matches the Ariel radiometric model's photon conversion efficiency within 1%, saturation time within 5%, and noise estimates below 1 ppm for a…","keywords":["exoplanet transit spectroscopy","observation simulator","Ariel mission","time-domain simulation","detector noise","pointing jitter","non-destructive reads","synthetic observations"],"falsifier":"Run the sequential pixel-readout simulator, which the paper says exists, on jitter timelines where the non-destructive-read cadence approaches the per-pixel read time and compare its output noise with ExoSim 2's instantaneous-readout NDRs: a difference exceeding the expected photon noise would show that the approximation fails in that regime. A second check is to inject a slow pointing drift, which the paper's high-frequency module cannot model, and see whether the resulting jitter noise changes the sub-ppm agreement.","tokens_in":22509,"feed_emoji":"🛰️","tokens_out":8556,"duration_ms":81097,"temperature":0.7,"pith_summary":"ExoSim 2 is a complete rewrite of the Exoplanet Observation Simulator, written in Python with a modular Task-based design, that produces time-resolved synthetic observations for transiting-exoplanet spectroscopy from space, ground, or sub-orbital platforms. The paper's central claim is that the simulator reproduces the expected Ariel payload performance: photon conversion efficiency agrees with ArielRad to within 1%, saturation time matches within the 5% validation threshold, and total noise estimates agree below 1 ppm for a 10-hour observation of a star like HD 209458 with jitter, photon, and read noise. The design goal is that each component can be replaced by a user-defined function, so the same workflow can be adapted to instruments other than Ariel. A jitter test case shows that including pointing jitter inflates the RMS of simulated transit light curves beyond photon and detector noise alone, which is exactly the kind of systematic that data-reduction pipelines need to remove.","feed_headline":"ExoSim 2 simulates Ariel data with noise match below 1 ppm","feed_subtitle":"Open-source, modular simulator validated against Ariel's radiometric model, ready for pipeline prototyping.","key_machinery":"The central mechanism is the oversampled focal plane used as a library of pre-shifted images. During focal plane creation, ExoSim 2 convolves the illuminated field with an intra-pixel response function to produce osf×osf shifted versions of each frame, one per sub-pixel step. The sub-exposure block then simulates pointing jitter by picking the appropriate pre-shifted focal plane for each jitter position inside a sub-exposure and averaging, which avoids recomputing the PSF convolution at every time step. That jitter handling is carried by the chunked Signal and HDF5 data-cube structure, which keeps multi-gigabyte simulations tractable. The NDR block is the other load-bearing piece: it accumulates sub-exposures into ramps, adds detector noise, merges readout groups, and converts floating-point counts to integer ADU, producing data in the same format a real detector would output.","core_discovery":"On the paper's own terms, ExoSim 2 is a validated end-to-end, time-domain simulator rather than a static radiometric model. Its focal-plane block produces three oversampled images from the target source, background point sources, and foreground diffuse light; the sub-exposure block adds astronomical time variability and pointing jitter; and the NDR block accumulates sub-exposures into non-destructive reads with dark current, shot noise, and read noise, merges groups in the MULTIACCUM style, and converts counts to ADU. The validation claims are quantitative: automated tests keep the photon conversion efficiency difference from ArielRad under 1%, saturation time is consistent within the 5% threshold used in the ArielRad paper, and a 10-hour HD 209458-like simulation including jitter, photon, and read noise differs from ArielRad's noise estimate by below 1 ppm. The instantaneous-readout approximation is defended by a separate pixel-sequential read-out simulator, to be published elsewhere, that reportedly finds negligible differences against photon noise for realistic jitter timelines. The paper also states a limitation: the high-frequency module cannot simulate slow pointing drift.","pith_inferences":["A natural stress test for the instantaneous-readout assumption is to push NDR cadence down toward the per-pixel read time; the forthcoming sequential read-out simulator should be checked there, since the current paper only reports the regime where jitter timescales are very different.","Because the paper's noise-agreement claim excludes slow pointing drift by design, the 1 ppm agreement describes a jitter-only error budget; adding a drift model could widen the gap.","The astronomical-signal module decouples the target light curve from the instrument model, so the same machinery should be able to simulate other time-domain phenomena, such as eclipsing binaries or microlensing events, by swapping in a different light-curve task.","The stored jitter timelines and random seeds make the simulations reproducible, so once real Ariel data exist, a direct comparison of predicted versus observed jitter noise would test the whole chain."],"forward_implications":["Researchers can prototype and test Ariel data-reduction and detrending pipelines on realistic synthetic observations before the mission launches.","An instrument team can replace individual tasks such as PSF, background, noise, or light curve and reuse the same three-step workflow for other missions such as EXCITE or JWST.","The three independent execution steps mean a user can generate a focal plane once and then explore many readout strategies, jitter realizations, or noise seeds without a full re-run.","Automated validation tests comparing ExoSim 2 to ArielRad protect future code releases from silently drifting away from the validated radiometric agreement.","The jitter test case gives a concrete estimate of how much pointing jitter inflates transit light-curve residuals, setting a target for detrending techniques."],"supporting_citations":[{"why":"ArielRad is the radiometric baseline; the paper's headline validation numbers (1% efficiency, 5% saturation, sub-ppm noise) are comparisons against it.","marker":"[14]"},{"why":"The original ExoSim provides the predecessor architecture, source handling, and background treatment that ExoSim 2 rewrites and extends.","marker":"[1]"},{"why":"Defines the up-the-ramp MULTIACCUM readout and detector noise conventions that the NDR block emulates when merging reads.","marker":"[30]"},{"why":"Supplies the batman transit light-curve model used as the ground truth in the astronomical-signal validation.","marker":"[33]"},{"why":"Provides the default intra-pixel response function that yields the pre-shifted oversampled focal planes used by the jitter algorithm.","marker":"[28]"},{"why":"Supplies the synthetic stellar SED grid from which the HD 209458 target spectrum is taken.","marker":"[23]"},{"why":"Provides the aberrated, defocused PSFs used to build the FGS channel focal planes.","marker":"[24]"}],"fun_headline_variants":["Open-source ExoSim 2 matches Ariel noise within 1 ppm","ExoSim 2: validated Python simulator for Ariel data","Modular ExoSim 2 predicts Ariel noise to sub-ppm","Time-domain ExoSim 2 validated against ArielRad","ExoSim 2: next-gen simulator for exoplanet transits"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that reading out all detector pixels at the same instant, at each read of the exposure ramp, produces the same jitter-induced noise as the real sequential pixel-by-pixel readout; the paper rests this on a separate, not-yet-published read-out simulator.","fun_headline_variants_meta":{"raw":{"variants":["Open-source ExoSim 2 matches Ariel noise within 1 ppm","ExoSim 2: validated Python simulator for Ariel data","Modular ExoSim 2 predicts Ariel noise to sub-ppm","Time-domain ExoSim 2 validated against ArielRad","ExoSim 2: next-gen simulator for exoplanet transits"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000194,"raw_usage":{"total_tokens":1428,"prompt_tokens":1097,"completion_tokens":331,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":713,"completion_tokens_details":{"reasoning_tokens":239}},"tokens_in":713,"tokens_out":331,"duration_ms":3358,"temperature":1.0,"reasoning_tokens":239,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T16:45:44.790092+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the sequential pixel-readout simulator, which the paper says exists, on jitter timelines where the non-destructive-read cadence approaches the per-pixel read time and compare its output noise with ExoSim 2's instantaneous-readout NDRs: a difference exceeding the expected photon noise would show that the approximation fails in that regime. A second check is to inject a slow pointing drift, which the paper's high-frequency module cannot model, and see whether the resulting jitter noise changes the sub-ppm agreement.","supporting_citations":[{"cited_title":"ArielRad: the Ariel Radiometric Model","cited_arxiv_id":"2009.07824","evidence_quote":"ArielRad is the radiometric baseline; the paper's headline validation numbers (1% efficiency, 5% saturation, sub-ppm noise) are comparisons against it."},{"cited_title":"https://doi.org/10.1007/ s10686-020-09690-9","cited_arxiv_id":null,"evidence_quote":"The original ExoSim provides the predecessor architecture, source handling, and background treatment that ExoSim 2 rewrites and extends."},{"cited_title":"Detectors for the James Webb Space Telescope Near-Infrared Spectrograph I: Readout Mode, Noise Model, and Calibration Considerations","cited_arxiv_id":"0706.2344","evidence_quote":"Defines the up-the-ramp MULTIACCUM readout and detector noise conventions that the NDR block emulates when merging reads."},{"cited_title":"Sub-Pixel Response Measurement of Near-Infrared Sensors","cited_arxiv_id":"astro-ph/0611339","evidence_quote":"Provides the default intra-pixel response function that yields the pre-shifted oversampled focal planes used by the jitter algorithm."},{"cited_title":"Astronomy & Astrophysics 577, 42 (2015) https://doi.org/10","cited_arxiv_id":null,"evidence_quote":"Supplies the synthetic stellar SED grid from which the HD 209458 target spectrum is taken."},{"cited_title":"In: Coyle, L.E., Matsuura, S., Perrin, M.D","cited_arxiv_id":null,"evidence_quote":"Provides the aberrated, defocused PSFs used to build the FGS channel focal planes."}],"review_version":1}