{"id":"264c9994-ca42-45c3-914f-c19b5f36b69e","arxiv_id":"2411.15089","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Methane on Earth-like exoplanets is detectable in reflected light only at Archean-like abundances, and its detectability drops sharply as water vapor abundance rises.","lead":"This paper simulates how well a future space telescope could detect methane in the reflected light of Earth-like exoplanets, using newly built spectral grids and Bayesian retrievals. It finds that methane is only detectable at high Archean-like abundances, and that water vapor can mask it, which should shape observing plans for the Habitable Worlds Observatory.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Archean CH4 'detectable at all SNRs' overstates the discrete SNR grid actually tested; no SNR below 3 and no bandpass below 20% was simulated, so the universal claim is unsupported.","rationale":"The reader's verdict is CONDITIONAL and lists model fidelity as the weakest assumption. I agree that fidelity to real Earth spectra is a major limitation, but it is a broad, well-known caveat that applies to nearly all retrieval simulation studies. The more precise and actionable gap is the unsupported universal quantifier in the central claim: 'detectable at all SNRs and bandpass widths.' The actual data cover a sparse, discrete SNR grid (the text mentions SNRs of 3, 5, 9, etc.) and only three bandpass widths. This is an internal-consistency problem because the conclusion exceeds the experimental design. The proposed check is cheap and decisive. The reader's rationale does note the overgeneralization of 'all SNRs' but does not elevate it to the weakest assumption; my read sharpens this into the primary load-bearing concern. The verdict remains CONDITIONAL: the abstract and conclusions need qualification, and the code/data should be public to allow the check.","tokens_in":16169,"tokens_out":9397,"duration_ms":83307,"concrete_test":"Use the released KEN grids and retrieval code (Zenodo DOI 10.5281/zenodo.13760695) to re-run the Archean CH4 (7.07e-3 VMR) case at SNR=1 and SNR=2 with a 20% bandpass centered at 1.1 µm, and at a 10% bandpass width (if the grids support it). Compute the log-Bayes factor for each run. If any yields lnB < 5, the headline 'all SNRs and bandpass widths' must be revised to 'all tested SNRs (>=3) and bandpass widths (20-40%).' If the code does not support these configurations, report that the claim is not tested and qualify the abstract accordingly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract and conclusions claim that an Archean-Earth CH4 level (7.07e-3 VMR) would be 'detectable at all SNRs and bandpass widths.' In Section 3.2, the reported required SNRs are discrete values (SNRs of 9, 5, and 3 for the three highest abundances), and the heatmaps in Figures 6, 8, and 9 show SNR axes over a finite set (apparently 3 to 20). No retrieval was run at SNR below 3, and the only bandpass widths considered are 20%, 30%, and 40%. The future-work paragraph explicitly states that a 10% bandpass will be added later, confirming that 'all bandpass widths' is not covered by the current data. Since the log-Bayes factor must decrease toward zero as SNR approaches zero, the literal claim 'at all SNRs' is false for untested low-SNR and narrow-bandpass regimes. This matters because the paper's stated purpose is to inform HWO science requirements; a reader could set requirements using SNR values or bandpasses outside the validated domain. This is an internal-consistency issue, not a model-fidelity issue, and it directly concerns the headline claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces the KEN grids, a set of six-parameter geometric albedo grids for 0.2–2 μm generated with the Planetary Spectrum Generator (PSG) through the new Gridder pipeline, and applies them to a Bayesian detectability study of CH4 in reflected light from an Earth-like exoplanet. The authors validate the grids against PSG-simulated truth, then compute log-Bayes factors from PSGnest retrievals over 25 bandpasses between 0.8 and 1.5 μm, with 20%, 30%, and 40% widths at R=140 and R=70. They report that a modern-Earth CH4 abundance is undetectable up to SNR=20, that an Archean-Earth abundance is detectable at all SNRs and bandpass widths tested, and that CH4 detectability degrades as H2O abundance increases because of overlapping absorption features.","tokens_in":16367,"tokens_out":5754,"duration_ms":52391,"significance":"If the quantitative claims survive, the paper provides a useful, publicly available grid infrastructure and a concrete, evidence-based framework for setting HWO observing requirements in the 0.8–1.5 μm region. The interpolation-error validation and the use of log-Bayes factor comparisons are genuine strengths, and the qualitative conclusion that H2O can mask CH4 at low CH4 abundance is an important caution for biosignature surveys. The magnitude of the contribution is incremental within the BARBIE series, but the release of KEN grids and the explicit degeneracy analysis make it a valuable community resource.","major_comments":[{"comment":"The claim that an Archean CH4 level is 'detectable at all SNRs and bandpass widths' is not supported by the data. The retrievals are run at discrete SNR values (3, 5, 6, 9, 13, 20) and only at 20%, 30%, and 40% bandpass widths; no retrieval is performed at SNR<3, and the paper defers a 10% bandpass to future work. Since the log-Bayes factor must approach zero as SNR approaches zero, the literal 'all SNRs' claim is false and could mislead requirements setting. Please rephrase to 'all tested SNRs (≥3) and bandpass widths (20–40%)' and add a caveat about the untested low-SNR and narrow-bandpass regime. The related statement that modern CH4 is undetectable 'at any SNR ≤20' should likewise be restricted to the discrete SNR values tested.","section":"Abstract; §3.2; §5"},{"comment":"The required-SNR values are reported as exact integers (e.g., SNR of 9, 5, and 3) without propagated or sampling uncertainties. The location of a strong-detection threshold depends on the stochasticity of the evidence estimate and the spacing of the SNR grid; differences such as SNR=5 versus SNR=9 could be within that noise. Please provide estimates of evidence uncertainty from repeated retrievals or an explicit statement that the quoted SNRs are grid-resolution lower/upper bounds, not precise requirements.","section":"§3.2, Figures 8–9"},{"comment":"The validation is internal: both the 'true' spectra and the retrieval forward models are generated with PSG, so the exercise demonstrates grid interpolation accuracy and retrieval self-consistency but not the fidelity of the PSG atmosphere model (constant VMRs, fixed temperature profile, simple cloud fraction, N2-only background). The absolute SNR requirements inherit any systematic errors in PSG's opacities, cloud treatment, or missing physics (e.g., hazes, non-constant VMR). Please state this limitation explicitly and, if possible, compare one or two end-to-end cases against an independent radiative transfer model or observed Earthshine spectra.","section":"§2.1; §3.1"}],"minor_comments":[{"comment":"In the first paragraph, 'drastically effected' should be 'drastically affected'.","section":"Introduction"},{"comment":"The abstract contains a typo: 'H2o' should be 'H2O'.","section":"Abstract"},{"comment":"The text after Figure 8 contains 'sim4', 'sim6', and 'sim8' where the approximation symbol '∼' is missing.","section":"§3.2"},{"comment":"The sentence 'any molecule at such low abundances does effect spectral change' should likely read 'does not affect spectral change'.","section":"§3.1"},{"comment":"The caption for panel (d) contains '1.47 8µm' with an erroneous space; it should be '1.478 µm'.","section":"Figure 9"},{"comment":"The sentence 'This confirms thatd H2O are degenerate' contains a typo; it should read 'This confirms that H2O and CH4 are degenerate'.","section":"§4"},{"comment":"The table header 'T able 1' has a spurious space.","section":"Table 1"},{"comment":"Gridder is cited as 'Himes et al. in prep'; if it is not yet publicly available, please provide a repository or archival reference to support reproducibility.","section":"§2.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid incremental contribution to the BARBIE series and the grid-validation work is commendable. The main obstacle to acceptance is the overstated universal claim in the abstract and conclusions; this is fixable by rewording and by adding a limitations paragraph on the discrete SNR grid and internal PSG validation. I do not see a load-bearing error that would require rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid, well-scoped simulation study that gives HWO planners a genuinely useful quantitative map of how CH4 detectability depends on both CH4 and H2O abundance, wavelength, and bandpass. The KEN grids are a real resource. The main caveat is that the abstract's 'detectable at all SNRs and bandpass widths' is looser than the actual grid: SNRs were tested down to 3 (not 0), and bandpasses were 20/30/40%, not a continuous range. The paper's own future-work section says a 10% bandpass will be added later, which confirms the claim outruns the data.\n\nWhat's new: the KEN grids (0.2–2 µm, with CH4, CO2, CO, SO2, N2O added) and the Gridder package. The validation is more thorough than the earlier S23 work: 4800 UltraNest retrievals plus interpolation-error tests, and the comparison between interpolated and directly computed spectra is convincing. The central result—that H2O masks CH4 at low-to-moderate CH4 abundances and shifts required SNR by factors of ~2—is clearly demonstrated with heatmaps and spectral overlays. The paper is honest about the gravity parameter's poor constraint at low values and about the lack of fully consistent chemistry in the forward models.\n\nSoft spots, in proportion. The biggest is the overstatement above. It is a wording problem, not a fatal flaw, but it matters because the paper's purpose is to set HWO exposure-time requirements; a reader could carry the 'all SNRs' claim into a requirements document. Second, the whole analysis is PSG-generated truth retrieved with PSG, so model errors—opacity, clouds, vertical mixing-ratio profiles—are invisible. The authors do not discuss this explicitly. That is standard for this kind of study, but it puts a floor on how literally we should read the absolute SNR values. Third, required SNRs are reported as integers without uncertainties; since they come from a finite grid of Bayes factors, that is a minor presentation issue. Fourth, the KEN grids and Gridder are announced but not yet public; a Zenodo DOI is given, so this is likely temporary.\n\nFor whom: anyone doing HWO science-yield simulations or planning the coronagraph bandpasses, and retrieval developers who want a ready-made grid. It deserves a serious referee; the science case is important and the analysis is careful. I would send it to review and ask for a qualified abstract, a statement about PSG self-consistency, and a note on the discrete SNR grid.","headline":"A useful, careful simulation study mapping CH4/H2O detectability for HWO, but the abstract's 'all SNRs and bandpass widths' overstates what was actually computed.","tokens_in":16934,"tokens_out":2946,"would_cite":true,"duration_ms":29357,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a modern-Earth level of methane is undetectable in reflected light at any signal-to-noise ratio up to 20, while an Archean-Earth level is detectable at all tested SNRs and bandpass widths.","keywords":["exoplanet atmospheres","methane detectability","geometric albedo","Bayesian retrieval","nested sampling","Habitable Worlds Observatory","reflected light","biosignatures"],"falsifier":"An independent test would repeat the same abundance-versus-SNR grid retrievals using a radiative-transfer model that includes non-constant vertical mixing ratios, scattering hazes, and realistic cloud phase functions; if the SNR required for strong CH4 detection at 1.1 µm shifts by more than a factor of two in the high-H2O regime, the reported inverse H2O–CH4 correlation is an artifact of the model setup.","tokens_in":15969,"feed_emoji":"🔭","tokens_out":5084,"duration_ms":43379,"temperature":0.7,"pith_summary":"This paper asks whether methane, a key biosignature candidate, could be seen in the reflected light of an Earth-like exoplanet by a future space coronagraph, and what observing strategy would find it. Using new spectral grids called KEN, the authors run Bayesian retrievals over 25 bandpasses between 0.8 and 1.5 microns and six CH4 abundance levels spanning modern Earth to the Archean. They find that modern-Earth methane (1.65e-6 VMR) is undetectable at any tested SNR up to 20, while Archean-level methane (7.07e-3 VMR) is detectable at all tested SNRs and bandpass widths. They also find that water vapor masks methane: as H2O abundance rises, the SNR needed to detect CH4 rises, and conversely H2O detectability depends on CH4 abundance. These results matter because they set concrete signal-to-noise and wavelength requirements for the Habitable Worlds Observatory's search for life on Earth twins.","feed_headline":"Ancient methane is visible; modern methane is not","feed_subtitle":"Water vapor raises the signal-to-noise needed to spot methane on Earth-like worlds.","key_machinery":"The load-bearing object is the KEN grid set: precomputed geometric albedo spectra generated with the Planetary Spectrum Generator (PSG) over 0.2–2 µm, each grid spanning six parameters (surface pressure, surface albedo, gravity, cloud fraction, and the grid's three molecular constituents; the background gas is N2). Retrievals interpolate from these grids rather than running radiative transfer in real time, then use nested sampling (PSGnest) and the log-Bayes factor between models with and without a molecule to declare a detection strong (lnB ≥ 5), weak, or absent. The KEN grids isolate overlapping spectral features: CH4 is placed in the same grid as H2O, CO2, and N2O so that confusion between species is captured, while orthogonal molecules are separated into different grids.","core_discovery":"The central discovery is that methane detectability in the 0.8–1.5 µm reflected-light spectrum of an Earth twin is controlled jointly by CH4 abundance, H2O abundance, bandpass width, and wavelength, with H2O absorption overlapping and masking CH4 features. Specifically, the authors report that a modern-Earth methane level of 1.65×10−6 VMR produces no detection at any SNR ≤ 20, whereas an Archean-Earth level of 7.07×10−3 VMR is strongly detected at every tested SNR and bandpass width. The required SNR for a strong CH4 detection falls steeply with abundance: 9 at 4.15×10−4 VMR, 5 at 1.65×10−3 VMR, and 3 at 7.07×10−3 VMR, with the best bandpass centers near 1.05 µm and 0.9 µm. Removing H2O from the fiducial spectrum lowers the detectable CH4 abundance by more than an order of magnitude, demonstrating that the reported correlation is a masking effect and not an instrument artifact.","pith_inferences":["We infer that a null CH4 detection by HWO will be ambiguous: it could mean no biogenic methane, a modern-Earth-like abundance, or a methane-rich atmosphere hidden by water vapor, so abundance-agnostic biosignature claims will need H2O priors.","The same KEN grid architecture could be applied to the CH4–CO2 pair in the near-infrared, where CO2 features also overlap H2O, to predict similar confusion for carbon-bearing biosignatures.","One testable extension is to run the retrievals on real Earthshine spectra (sunlight reflected by Earth) binned to the same bandpasses, which would check whether the PSG-based masking correlation survives realistic atmospheric complexity."],"forward_implications":["Observational requirements for HWO should be set by Archean-Earth methane levels if CH4 is a target; modern-Earth levels cannot serve as a design driver for the 0.8–1.5 µm range.","A CH4 detection at low to moderate abundance is not interpretable without simultaneously constraining H2O, since water vapor can suppress the CH4 signal below detection.","The 1.1 µm region is the best compromise for detecting both CH4 and H2O; wavelengths beyond about 1.3 µm are poor for both molecules at the tested SNRs.","Required SNR for strong CH4 detection scales from about 9 at 4.15×10−4 VMR to 3 at 7.07×10−3 VMR, giving a concrete exposure-time target for mission planning."],"supporting_citations":[{"why":"Supplies the grid-based Bayesian retrieval method that the KEN grids extend and validate.","marker":"Susemiehl et al. 2023"},{"why":"BARBIE1 applies the same method and defines the fiducial Earth-twin setup used here.","marker":"Latouf et al. 2023"},{"why":"BARBIE2 continues the methodology and provides the detection thresholds and prior usage.","marker":"Latouf et al. 2024"},{"why":"Provides the modern-Earth model spectrum adopted as the fiducial data in this study.","marker":"Feng et al. 2018"},{"why":"Defines the log-Bayes factor thresholds that separate unconstrained, weak, and strong detections.","marker":"Benneke & Seager 2013"},{"why":"Describes the Planetary Spectrum Generator radiative transfer model used to build the KEN grids.","marker":"Villanueva et al. 2018"},{"why":"Documents the PSGnest retrieval implementation and grid interpolation used for the retrievals.","marker":"Villanueva et al. 2022"},{"why":"Supplies geological CH4 abundance estimates for different epochs of Earth's history.","marker":"Kasting 2005"},{"why":"Provides additional Earth-epoch CH4 abundance values interpolated for the study.","marker":"Kaltenegger et al. 2007"}],"fun_headline_variants":["Water vapor masks methane signals on Earth-like worlds","Methane detectability hinges on water abundance","To find methane, check water first","Modern methane hidden by water; ancient methane visible"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The analysis presupposes that PSG's radiative-transfer spectra, computed with constant gas mixing ratios, a single cloud fraction, a fixed temperature profile, and N2 as the only background gas, are accurate enough to represent real Earth-like exoplanets between 0.8 and 1.5 µm, and that the detectability metric computed on PSG-simulated data reflects detectability on real observations.","fun_headline_variants_meta":{"raw":{"variants":["Water vapor masks methane signals on Earth-like worlds","Methane detectability hinges on water abundance","To find methane, check water first","Modern methane hidden by water; ancient methane visible"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000301,"raw_usage":{"total_tokens":1795,"prompt_tokens":1064,"completion_tokens":731,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":680,"completion_tokens_details":{"reasoning_tokens":675}},"tokens_in":680,"tokens_out":731,"duration_ms":7734,"temperature":1.0,"reasoning_tokens":675,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:30:25.516280+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"An independent test would repeat the same abundance-versus-SNR grid retrievals using a radiative-transfer model that includes non-constant vertical mixing ratios, scattering hazes, and realistic cloud phase functions; if the SNR required for strong CH4 detection at 1.1 µm shifts by more than a factor of two in the high-H2O regime, the reported inverse H2O–CH4 correlation is an artifact of the model setup.","supporting_citations":[{"cited_title":"L., Liuzzi, G., Faggi, S., et al","cited_arxiv_id":null,"evidence_quote":"Documents the PSGnest retrieval implementation and grid interpolation used for the retrievals."}],"review_version":1}