REVIEW 4 major objections 5 minor 1 cited by
A space-based direct imaging survey with 25-30 habitable-zone exoEarths can recover only the strongest albedo step, while weaker trends need 80-90.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
An HWO-like survey needs about 30 Earth-sized planets to recover a strong albedo step at the habitable zone, but 80-90 for weaker trends.
T0 review reviewed 2026-08-04 challenge →
load-bearing objection A careful simulation with a useful qualitative answer, but the headline numbers rest on a KDE that the paper itself admits is sensitive—worth refereeing with requests for robustness bounds. the 4 major comments →
Bioverse: Assessing the Ability of Direct Imaging Surveys to Empirically Constrain the Habitable Zone via Trends in Albedo
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
The paper's central claim is that statistical power to detect an albedo-instellation trend grows sharply with the strength of the trend, not just with sample size. Using forward-modeled distributions of the directly observable quantity beta = C(L/Lsun)^{-1}/S_eff (a normalized contrast proportional to geometric albedo times squared radius times phase function), the authors draw random samples of N_EEC exoEarths and compute Bayes factors between a step-function trend model and a no-trend null model via kernel density estimates. For a strong trend (Delta A = 0.4), a 95% true-positive rate at Bayes factor K>10 needs about 30-35 exoEarths, close to the Decadal Survey target. For Delta A = 0.3, t
What carries the argument
The central object is beta, a normalized planet-star contrast that is the closest directly observable proxy for albedo when radius, phase, and albedo are degenerate: beta = A_g (R_p/1AU)^2 Phi(alpha). The paper injects a step-function albedo model A_g = A0 - Delta A inside the habitable zone and A0 outside (for rocky planets R_p < 1.4 R_Earth), forward-models detectability with an exposure-time calculator, builds smooth survey distributions with kernel density estimates, and then uses the Bayes factor from Equation 6, the product of per-point KDE likelihoods, to measure how often random N_EEC samples separate trend from null at K=3 and K=10 thresholds.
Load-bearing premise
The entire statistical-power calculation assumes that the simulated survey's forward-modeled distribution, including the injected step-function albedo, the assumed occurrence rates, exozodi level, noise, and KDE likelihood, is the true distribution a real survey would produce; if the real planetary population or measurement behavior differs, the required sample sizes will be larger.
What would settle it
Repeat the sample-size analysis with a different KDE bandwidth or a different occurrence-rate model; if the number of exoEarths needed for 95% power shifts by more than a factor of two, the headline numbers are not robust. Alternatively, apply the same Bayes-factor test to the real normalized-contrast data from the first ~30-exoEarth survey; if the trend is not recovered at K>10 in at least ~90% of repeated realizations, the claim fails.
If this is right
- If correct, the Decadal Survey's 25-exoEarth requirement is sufficient only for the strongest plausible albedo step; weaker but arguably more realistic trends would demand a larger telescope or a longer survey.
- Mission trade studies can use 'exoEarths needed to recover a population trend' as a design metric alongside raw EEC yield.
- Because the survey distribution is insensitive to 6-10 m telescope diameter and coronagraph depth, the key decision for this science case is yield, not the exact shape of the detection bias.
- Reducing measurement noise (contrast and semi-major axis uncertainty) by 10x improves statistical power only slightly, so chasing precision per target is a poor substitute for sample size.
- A survey that only characterizes habitable-zone planets would lose the comparison sample of rocky planets outside the HZ, which is essential for trend detection.
Where Pith is reading between the lines
- The reported sample sizes are likely optimistic: they treat the synthetic forward model as the true data-generating process, and Appendix B shows the true-positive rate is sensitive to KDE bandwidth; a real survey with different occurrence rates, exozodi, or phase-function behavior would probably need more exoEarths.
- The same Bayes-factor machinery could be applied to other HZ tests the authors list (water-vapor fraction, CO2 dependence on instellation), turning each into a sample-size requirement before the mission is built.
- If high-resolution spectroscopy becomes cheap enough, direct albedo estimates from retrievals would break the beta degeneracy; the paper leaves open whether the reduced sample size of well-characterized planets would outweigh the gain, which a follow-up simulation could settle.
- The false-positive rate increases with sample size and trend strength in a way the authors do not fully explain; if that variance growth is real, future significance thresholds may need to be calibrated per N_EEC rather than fixed at K=10.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper adapts the Bioverse statistical framework to simulate direct-imaging surveys for HWO-like architectures, injecting a toy step-function albedo trend (lower albedo inside the conservative Kopparapu HZ, A0=0.7 outside, ΔA up to 0.4). It forward-models the survey distribution of normalized contrast β versus instellation S_eff, then draws samples with a specified number of exoEarth candidates (EECs) and computes Bayes factors from KDE likelihoods (Eq. 6) to estimate true and false positive rates. The main findings are that a strong ΔA=0.4 trend requires roughly 30-35 EECs for 95% power at K>10, weaker trends (ΔA=0.3) require 80-90 EECs, and the Decadal Survey's 25-EEC target suffices only for the strongest trend. Appendices examine sensitivity to telescope design, KDE bandwidth, and simulation count.
Significance. If the quantitative estimates were robust, the paper would provide a useful, falsifiable framework for HWO trade studies and a caution that the 25-EEC target is marginal for population-level albedo trends. Strengths include the explicit and publicly available code fork, the Monte Carlo power-calculation approach, the convergence analysis in Appendix C, and the unusually honest sensitivity tests in Appendices A and B that expose where the numbers are fragile. The qualitative conclusion that weaker trends require much larger samples is likely robust; however, the headline sample-size numbers are not yet established because the central statistic depends on KDE bandwidth choices that are not fully quantified.
major comments (4)
- [§2.6, Eq. (6); Appendix B] The Bayes factor is computed as a product of KDE densities, with no marginalization over the KDE bandwidth or over the trend parameters (A0, ΔA, HZ boundaries). The statement 'there is no parameter to marginalize over' is therefore misleading. Appendix B and Figure 10 show that a 50% reduction in bandwidth noticeably lowers the true positive rate and may asymptote below 100% power, directly affecting the required N_EEC values in §3.2–§3.3. Please quantify this sensitivity: report required N_EEC for each bandwidth in Figure 10, validate the KDE with held-out samples or cross-validation, and give confidence intervals on the headline sample sizes.
- [§3.3, Figure 6] The false positive rate is reported to increase with trend strength and sample size in parts of the parameter space, and the authors state that 'the exact reason for the increase in the variance of the Bayes factor distribution remains unclear.' A hypothesis test whose false positive rate grows with sample size is not well calibrated; since the FPR is used to argue that the survey is power-limited rather than false-positive-limited, this behavior must be explained. Please diagnose the cause (e.g., heavy tails of the log-Bayes-factor distribution, KDE edge effects) and report numerical FPR values with uncertainties, or restrict the conclusions to the calibrated regime.
- [Title/Abstract; §2.2; §3.3] The simulations inject a step-function albedo dip at the known Kopparapu HZ boundaries with fixed A0 and ΔA. The analysis measures the power to detect that known-shape dip, not the ability to infer the HZ boundary location or the trend shape. Claims of 'empirically constrain the Habitable Zone' therefore overstate what is computed. Please clarify this distinction in the abstract and discussion, or add a version of the test in which the HZ boundaries (and ideally ΔA) are free parameters and are marginalized over.
- [§2.5; §4] The reported power is computed under the assumption that the Bioverse forward model is the true data-generating process. Any mismatch between the assumed planet distribution function, exozodi level, noise model, and reality will degrade the real-world detection power. The paper acknowledges this qualitatively in §4 but does not quantify it. At least for the headline number (30–35 EECs for ΔA=0.4), please state explicitly that this is an upper bound under the adopted model and provide a sensitivity test on the most uncertain inputs (e.g., exozodi level, η⊕, PDF shape).
minor comments (5)
- [Title] Typo: 'T rends' should be 'Trends'.
- [§4.3] Duplicate word: 'would would' should be 'would'.
- [§3.1 vs §3.3] The text uses 10,000 simulations for the N_EEC grid and 1,000 for the ΔA–N_EEC grid. Please state this clearly in both sections, since the FPR contours in Figure 6 may be noisy at 1,000 simulations.
- [Figure 6] The description of the bottom-left corner is confusing; it is unclear whether high FPR occurs at low ΔA/low N or low ΔA/high N. Annotate the figure or revise the text to make the behavior unambiguous.
- [Appendix A] The 'insensitivity' claim is tested only for telescope diameter and minimum contrast. Exozodi level, which is a known major determinant of yield and detectability, is not varied in the survey-distribution comparison; please note this limitation explicitly.
Circularity Check
No significant circularity: the paper is a self-contained forward-model power study, not a prediction derived from its own inputs by construction.
full rationale
The paper's derivation chain is an injection-recovery simulation. It specifies a toy albedo step function (Eq. 4), forward-models a direct imaging survey through Bioverse, builds survey distributions by simulating 1000 universes, and then measures how often a Bayes-factor test (Eq. 6) classifies random samples correctly. The test data and the KDE likelihoods are both generated from the same forward model, but that is the correct design for a power calculation: it calibrates the best-case statistical separability of the two hypotheses under the model assumptions. The sample-size numbers (e.g., ~30–35 EECs for 95% power at ΔA=0.4) are computed outputs of this Monte Carlo exercise, not fitted parameters renamed as predictions. The paper's external inputs (SAG 13 occurrence rates, Kopparapu HZ boundaries, HOSTS exozodi, ExEP uncertainties) are all cited to independent sources; self-citations to Bioverse and the HPIC provide the framework and target list but are not the load-bearing justification for the recovery rates. Appendix B is an explicit robustness check showing that the true-positive rate depends on KDE bandwidth; this is a statistical modeling caveat, and the paper flags it, but it does not reduce the central claim to its own inputs by definition. Therefore no circular step can be exhibited.
Axiom & Free-Parameter Ledger
free parameters (6)
- A0 (albedo outside HZ) =
0.7
- DeltaA (albedo step strength in HZ) =
0.4 strong; 0.3 and smaller explored
- KDE bandwidth =
0.05 on preprocessed axes
- Exozodiacal dust level =
3 zodi
- Detection SNR threshold =
SNR = 7
- Semi-major-axis uncertainty =
5%
axioms (7)
- domain assumption Lambertian phase function for all planets.
- domain assumption Kopparapu et al. (2013/2014) habitable zone boundaries and the step-function albedo model (Eq. 4) capture the true albedo-instellation relation.
- domain assumption SAG 13 planet occurrence rates with eta_earth = 0.24 and spectral-type dependence describe the real planet population.
- domain assumption The survey distribution built from 1000 forward simulations is a sufficient and unbiased model of the observable planet population.
- ad hoc to paper The Bayes factor can be computed as a product of KDE densities without marginalizing over population parameters or model complexity.
- domain assumption Measurement noise levels: 3% luminosity, 14% contrast, 5% semi-major axis, 3 zodi exozodiacal light.
- domain assumption Equal prior probabilities for the Trend and Null models.
Cite this review
Pith. "Pith review of Bioverse: Assessing the Ability of Direct Imaging Surveys to Empirically Constrain the Habitable Zone via Trends in Albedo." pith.science (2026). https://pith.science/paper/K2WV7VDH
@misc{pith2026250907297,
author = {Pith},
title = {Pith review of: Bioverse: Assessing the Ability of Direct Imaging Surveys to Empirically Constrain the Habitable Zone via Trends in Albedo},
year = {2026},
howpublished = {\url{https://pith.science/paper/K2WV7VDH}},
note = {Machine review of arXiv:2509.07297}
}
read the original abstract
Will future direct imaging missions such as NASA's upcoming Habitable Worlds Observatory (HWO) be able to understand Earth-sized planets as a population? In this study, we simulate the ability of space-based coronagraphy missions to uncover trends in planetary albedo as a function of instellation, and potentially constrain the boundaries of the habitable zone. We adapt the Bioverse statistical comparative planetology framework to simulate the scientific output of possible designs for HWO. With this tool, we generate a synthetic planetary population with injected population-level trends in albedo and simulate the observability of planets. We then determine the statistical power to which these trends can be recovered as a function of the strength of the injected trend and the sample size of Earth-sized planets in the habitable zone (exoEarths). The strongest trends in albedo require a sample size of roughly 25-30 exoEarths to recover with high confidence. However, for weaker albedo trends, the required number of planets increases rapidly. If a mission is designed to meet the Decadal Survey's requirement of 25 exoEarths, it would be able to recover very strong trends in albedo associated with the habitable zone, but would struggle to confidently detect weaker trends. We explore multiple strategies to increase one's ability to recover weak trends, such as reducing the uncertainties in observables, incorporating additional observables such as planet colors, and obtaining direct constraints on planetary albedo from full spectral retrievals.
Figures
Forward citations
Cited by 1 Pith paper
-
Not Earth-like Yet Temperate? More Generic Climate Feedback Configurations Still Allow Temperate Climates in Habitable Zone Exo-Earth Candidates
An idealized model with an extra generalized feedback produces chaotic and runaway climates and predicts that strong positive fourth feedbacks reduce long-term temperate habitability of Earth-like exoplanets.
Reference graph
Works this paper leans on
-
[1]
2025, AJ, 169, 125, doi: 10.3847/1538-3881/ada384
Affholder, A., Mazevet, S., Sauterey, B., Apai, D., & Ferri` ere, R. 2025, AJ, 169, 125, doi: 10.3847/1538-3881/ada384
-
[2]
Exploring Other Worlds: Science Questions for Future Direct Imaging Missions (EXOPAG SAG15 Report)
Apai, D., Cowan, N., Kopparapu, R., et al. 2017, arXiv e-prints, arXiv:1708.02821, doi: 10.48550/arXiv.1708.02821
work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.1708.02821 2017
- [3]
-
[4]
Bean, J. L., Abbot, D. S., & Kempton, E. M.-R. 2017, The Astrophysical Journal Letters, 841, L24
work page 2017
-
[5]
2019, The Astronomical Journal, 159, 3 —
Bixel, A., & Apai, D. 2019, The Astronomical Journal, 159, 3 —. 2021, The Astronomical Journal, 161, 228
work page 2019
-
[6]
Bryson, S., Kunimoto, M., Kopparapu, R. K., et al. 2021, AJ, 161, 36, doi: 10.3847/1538-3881/abc418
-
[7]
Cahoy, K. L., Marley, M. S., & Fortney, J. J. 2010, The Astrophysical Journal, 724, 189
work page 2010
-
[8]
Checlair, J. H., Abbot, D. S., Webber, R. J., et al. 2019, arXiv preprint arXiv:1903.05211
work page internal anchor Pith review Pith/arXiv arXiv 2019
-
[9]
A., McFadden, L., Robinson, T., et al
Crow, C. A., McFadden, L., Robinson, T., et al. 2011, The Astrophysical Journal, 729, 130
work page 2011
-
[10]
2022, AJ, 163, 299, doi: 10.3847/1538-3881/ac6b97
Damiano, M., & Hu, R. 2022, AJ, 163, 299, doi: 10.3847/1538-3881/ac6b97
-
[11]
2020, The Astronomical Journal, 159, 177
Ertel, S., Defr` ere, D., Hinz, P., et al. 2020, The Astronomical Journal, 159, 177
work page 2020
-
[12]
S., Seager, S., Mennesson, B., et al
Gaudi, B. S., Seager, S., Mennesson, B., et al. 2020, arXiv preprint arXiv:2001.06683
Pith/arXiv arXiv 2020
-
[13]
2023, AJ, 165, 267, doi: 10.3847/1538-3881/acd1ec
Pascucci, I., & L´ opez-Morales, M. 2023, AJ, 165, 267, doi: 10.3847/1538-3881/acd1ec
-
[14]
Hardegree-Ullman, K. K., Apai, D., Haffert, S. Y., et al. 2025, The Astronomical Journal, 169, 171
work page 2025
-
[15]
1998, The theory of probability (OuP Oxford)
Jeffreys, H. 1998, The theory of probability (OuP Oxford)
work page 1998
-
[16]
F., Whitmire, D
Kasting, J. F., Whitmire, D. P., & Reynolds, R. T. 1993, Icarus, 101, 108
1993
-
[17]
Kopparapu, R. K. 2018, Handbook of Exoplanets, 58
work page 2018
-
[18]
Kopparapu, R. K., Ramirez, R. M., SchottelKotte, J., et al. 2014, ApJL, 787, L29, doi: 10.1088/2041-8205/787/2/L29
-
[19]
K., Ramirez, R., Kasting, J
Kopparapu, R. K., Ramirez, R., Kasting, J. F., et al. 2013, The Astrophysical Journal, 765, 131
2013
-
[20]
K., H´ ebrard, E., Belikov, R., et al
Kopparapu, R. K., H´ ebrard, E., Belikov, R., et al. 2018, The Astrophysical Journal, 856, 122
work page 2018
-
[21]
2012, The Astrophysical Journal, 747, 25
Madhusudhan, N., & Burrows, A. 2012, The Astrophysical Journal, 747, 25
2012
- [22]
-
[23]
Mamajek, E., & Stapelfeldt, K. 2023, NASA ExEP Mission Star List for the Habitable Worlds Observatory: Most Accessible Targets to Survey for Potentially Habitable Exoplanets, https://exoplanets.nasa.gov/internal resources/2645 NASA ExEP Target List HWO Documentation 2023.pdf
work page 2023
-
[24]
2019, in Techniques and instrumentation for detection of exoplanets IX, Vol
Morgan, R., Savransky, D., Turmon, M., et al. 2019, in Techniques and instrumentation for detection of exoplanets IX, Vol. 11117, SPIE, 1111701
work page 2019
-
[25]
Mulders, G. D., Pascucci, I., & Apai, D. 2015a, ApJ, 798, 112, doi: 10.1088/0004-637X/798/2/112 —. 2015b, ApJ, 814, 130, doi: 10.1088/0004-637X/814/2/130 National Academies of Sciences, E., & Medicine. 2021, Pathways to Discovery in Astronomy and Astrophysics for the 2020s (Washington, DC: The National Academies Press), doi: 10.17226/26141
-
[26]
Neil, A. R., & Rogers, L. A. 2020, ApJ, 891, 12, doi: 10.3847/1538-4357/ab6a92
-
[27]
Pascucci, I., Mulders, G. D., & Lopez, E. 2019, ApJL, 883, L15, doi: 10.3847/2041-8213/ab3dac
-
[28]
Rizzo, M. L. 2019, Statistical computing with R (Chapman and Hall/CRC)
work page 2019
-
[29]
Roberge, A., Rizzo, M. J., Lincowski, A. P., et al. 2017, Publications of the Astronomical Society of the Pacific, 129, 124401 23
work page 2017
- [30]
-
[31]
Salvador, A., Robinson, T. D., Fortney, J. J., & Marley, M. S. 2024, ApJL, 969, L22, doi: 10.3847/2041-8213/ad54c5
-
[32]
2025, The Astrophysical Journal, 987, 24
Schlecker, M., Apai, D., Affholder, A., et al. 2025, The Astrophysical Journal, 987, 24
work page 2025
-
[33]
2024, PSJ, 5, 3, doi: 10.3847/PSJ/acf57f
Schlecker, M., Apai, D., Lichtenberg, T., et al. 2024, PSJ, 5, 3, doi: 10.3847/PSJ/acf57f
-
[34]
Shields, A. L., Meadows, V. S., Bitz, C. M., et al. 2013, Astrobiology, 13, 715
work page 2013
-
[35]
C., Roberge, A., Mandell, A., et al
Stark, C. C., Roberge, A., Mandell, A., et al. 2015, The Astrophysical Journal, 808, 149
work page 2015
-
[36]
C., Roberge, A., Mandell, A., & Robinson, T
Stark, C. C., Roberge, A., Mandell, A., & Robinson, T. D. 2014, The Astrophysical Journal, 795, 122
work page 2014
-
[37]
Stark, C. C., Belikov, R., Bolcar, M. R., et al. 2019, Journal of Astronomical Telescopes, Instruments, and Systems, 5, 024009
work page 2019
-
[38]
R., Huber, D., & van Saders, J
Tayar, J., Claytor, Z. R., Huber, D., & van Saders, J. 2022, The Astrophysical Journal, 927, 31 The LUVOIR Team. 2019, arXiv e-prints, arXiv:1912.06219. https://arxiv.org/abs/1912.06219
Pith/arXiv arXiv 2022
-
[39]
Traub, W. A. 2003, in ESA Special Publication, Vol. 539, Earths: DAR WIN/TPF and the Search for Extrasolar Terrestrial Planets, ed. M. Fridlund, T. Henning, & H. Lacoste, 231–239
work page 2003
-
[40]
Traub, W. A., & Oppenheimer, B. R. 2010, in Exoplanets, ed. S. Seager, 111–156
work page 2010
-
[41]
Tuchow, N. W., Stark, C. C., & Mamajek, E. 2024, The Astronomical Journal, 167, 139
work page 2024
-
[42]
Turbet, M. 2020, arXiv e-prints, arXiv:2005.06512, doi: 10.48550/arXiv.2005.06512
work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2005.06512 2020
This paper was first reviewed by deepseek-v4-flash on August 4, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.