REVIEW 2 major objections 4 minor 1 cited by
ZTF SNe Ia DR2: Towards cosmology-grade ZTF supernova light curves using scene modeling photometry
T0 review · 2 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read A newly identified CCD readout artifact, the 'pocket effect,' makes ZTF supernova photometry brightness-dependent by up to 7%, ruling out both the DR2 and scene-modeling light curves for precision cosmology.
desk verdict Honest, useful status report: the pocket effect and the 'not cosmology-ready' conclusion hold up, but the 90 mmag DR2 calibration claim is weaker than the abstract suggests. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Two objects carry the argument. Scene modeling photometry (SMP) models each supernova's scene as a per-exposure PSF plus a single host-galaxy model shared across all exposures and convolved with a seeing-matching kernel; it is a maximum-likelihood estimator that can be applied unchanged to field stars, so an external calibration (here the Pan-STARRS catalog) propagates through the same flux estimator used for the supernova. The pocket effect is a readout artifact discovered during this work: during serial readout, a fraction of the charge can escape into a 'pocket' — possibly a trap or a drift-field defect — and return after tens of microseconds; the escape probability depends on how full th
What would settle it
Take one of the affected CCDs and measure the PSF-to-aperture magnitude ratio as a function of source flux while artificially raising the sky background above the ~6000 ADU threshold where the pocket effect is said to disappear; if the flux-dependent nonlinearity persists at high background, the charge-pocket mechanism is wrong. Conversely, if applying the pixel-level correction now under development removes both the flux-dependent PSF skewness and the 90 mmag SMP–DR2 distance offset, the paper's central causal chain is confirmed.
Extended reading notes
Core claim
The central claim: the ZTF SN Ia DR2 sample — the largest low-redshift supernova dataset to date — cannot yet anchor precision cosmology because of a newly identified sensor effect. Using scene modeling photometry, which applies one common flux estimator to the supernova and to the field stars used for calibration, the authors reprocess all 3,628 supernovae. SMP and forced-photometry light curves agree on stretch and time of maximum, with only a 10 mmag color shift — so the DR2 population studies stand. But distance moduli differ by 90 mmag, and the likely cause is the pocket effect: a readout artifact in which a flux-dependent fraction of charge escapes and returns during transfer, distorti
Load-bearing premise
The claim that DR2's absolute calibration is off by 90 millimag assumes the scene-modeling calibration chain, anchored on Pan-STARRS star magnitudes, is the more accurate of the two pipelines — and the paper concedes its own unresolved nonlinearities could instead be the cause.
Editorial extensions
If this is right
- Every ZTF transient light curve taken since November 2019 carries a flux-dependent photometric error of up to ~7%, so any analysis reaching for percent-level accuracy must wait for the reprocessing.
- The pixel-level correction under development, combined with the demonstrated two-to-three-week reprocessing capability, makes a corrected 'DR2.5' release of the full 3,628-SN sample achievable.
- The consistency of stretch and peak time between the two pipelines, with only a 10 mmag color shift, means the DR2 papers' supernova-population conclusions are not overturned by the calibration problems.
- Reaching the 0.1% photometric accuracy target requires, in addition to the sensor correction, ZTF bandpass models accurate to roughly 3 Å; current models are off by 15–60 Å in parts of the focal plane.
- A CALSPEC-based calibration chain, replacing the current Pan-STARRS anchor, is the intended route to an absolute flux scale consistent across the g, r, and i bands.
Reading between the lines
- If the pocket effect is as universal as claimed, other surveys using CCDs with similar readout designs should look for the same signature — flux-dependent serial-direction PSF skewness that diminishes at sky backgrounds above ~6000 ADU — before trusting their own PSF photometry.
- The 90 mmag distance-modulus offset, if it stems from DR2's calibration rather than SMP's residual nonlinearities, would shift any low-redshift anchor built on DR2 by more than its statistical uncertainty; quantifying that shift on published dark-energy constraints is a natural follow-up the paper leaves implicit.
- A clean test the paper does not run: comparing both pipelines on supernovae observed before November 2019, where the pocket effect is stated to be negligible, would separate the pocket effect from any other calibration offset and test the 90 mmag attribution directly.
- The per-CCD bandpass differences between single- and dual-layer anti-reflective coatings imply that using a single average bandpass per filter — as the released photometry does — introduces position-dependent color biases; a spatially resolved bandpass model may be as important as the sensor correction for 0.1% cosmology.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents the current status of Scene Modeling Photometry (SMP) for the ZTF SN Ia DR2 sample. It describes the full pipeline (PSF training, astrometric transforms, photometric alignment, scene-model fit, PS1-anchored calibration), demonstrates its scalability on the 179 TB dataset, and reports a photometric repeatability of about 1% for field stars. The paper identifies a new CCD effect, the 'pocket effect', causing flux-dependent PSF distortions and photometric nonlinearities up to 7%, and finds that current bandpass models are not accurate enough for 0.1% cosmology. Comparing SMP with the DR2 forced-photometry light curves, it reports consistent x1 and tmax, a 10 mmag color shift, and a 90 mmag distance-modulus offset, interpreted as evidence that the DR2 absolute calibration is inadequate. The paper concludes that neither the DR2 nor the current SMP light curves are suitable for precise cosmological analyses, and outlines plans for a pixel-level pocket-effect correction, a CALSPEC-based calibration chain, and improved bandpass models.
Significance. The paper makes two contributions. First, it demonstrates a scalable SMP pipeline that processes the full DR2 dataset in two to three weeks, producing light curves for the largest SN Ia sample to date; this is a substantial technical achievement. Second, it provides direct, falsifiable empirical evidence for a new sensor effect ('pocket effect'): flux-dependent PSF skewness (Fig. 13), astrometric residuals (Fig. 2), and PSF-to-aperture flux ratios (Fig. 14) are all measured against external Gaia and PS1 references. This part of the paper is solid and will be important for ZTF and similar surveys. The conclusion that neither DR2 nor the current SMP internal release is suitable for 0.1% cosmology is strongly supported by the repeatability floor (~1%), the pocket-effect nonlinearities (up to 7%), and the bandpass model discrepancies. The weaker link is the interpretation of the 90 mmag SMP-DR2 offset as a DR2 calibration failure; the data do not identify which pipeline is biased. If the pocket-effect correction and CALSPEC chain are completed as described, the pipeline has clear potential to deliver cosmology-grade light curves.
major comments (2)
- [§5, Fig. 12, §7] The headline offset of 90 mmag in distance modulus is presented as evidence that 'the absolute calibration of the DR2 pipeline is not known with the precision needed' (abstract and §5). This attribution is not established. The SMP zero-point is anchored to PS1 through SMP photometry of field stars (Eq. 5, §3.9); those same field-star fluxes are subject to the pocket-effect nonlinearities quantified in §6.1. The paper's own §7 states 'we cannot rule out the possibility that the large nonlinearities identified in this study contribute to this discrepancy.' The data therefore support a 90 mmag discrepancy between two pipelines, not a verdict on which pipeline is biased. I request either (a) an independent check of the SMP absolute scale (e.g., comparison of SMP star fluxes to aperture photometry on low-sky frames or to CALSPEC standards), or (b) a rewording of the abstract and §5 to present
- [§6.3, Fig. 15, §7 item 3] The statement in the conclusion that 'current models are precise at the 3 nm level' conflicts with the measurements shown in Fig. 15. There, the SNCosmo bandpass models are found to be inaccurate by ~30 Å in g and ~15 Å in r, and the improved bench-based model deviates by ~60 Å in g from the stellar color terms. A 60 Å error is 6 nm, not 3 nm, and even the 30 Å SNCosmo error is at the upper edge of what the text calls the 3 nm level. The requirement for dark-energy measurements is ~3 Å in mean wavelength; the actual model errors are an order of magnitude larger. The conclusion should quote the measured model errors (Fig. 15) rather than the focal-plane non-uniformity from §2.2, so that the severity of this limitation is stated accurately.
minor comments (4)
- [Abstract and §4.2] The sample size is inconsistent: the abstract and §2.3 state 3628 SNe Ia, while §4.2 says '3582 SNe (out of 3826)'. Please reconcile the numbers and clarify which total is correct.
- [Throughout] Typos and grammar: 'This assess the robustness' should be 'This assesses the robustness'; 'the the ZTF SN Ia DR2 release' has a duplicated article; 'altough' → 'although'; 'directely' → 'directly'; 'sufficient' → 'sufficient'. A careful proofread is needed.
- [§3.9, Eq. (5)] The variable 'col' is not defined at first use. It should be stated that col is the PS1 g-i color (or whichever color is used in the calibration). Also, the term δzp(x) is not specified; if it is a position-dependent term, its functional form should be given.
- [Fig. 12] The Hubble-diagram residuals appear to show a redshift-dependent trend (black binned points), which the text only describes as 'a slight variation in redshift'. Please quantify this trend and state whether it affects the interpretation of the 90 mmag offset.
Circularity Check
No significant circularity: the central claims rest on empirical comparisons between independent reductions, anchored to external references, with explicit caveats rather than constructed equivalences.
full rationale
The paper's main quantitative results—the pocket-effect nonlinearities and the 90 mmag offset between SMP and DR2—are measured comparisons, not quantities defined in terms of one another. SMP fluxes are obtained by a separate maximum-likelihood scene-modeling fit from the pixels, and the calibration is anchored to the external PS1 catalog through field-star SMP light curves. The DR2 comparison is an empirical difference between two independent photometric reductions. The paper explicitly concedes that the SMP chain itself may suffer from the same nonlinearities, so the attribution of the 90 mmag offset to DR2 is a caveated interpretation, not a circular derivation. References to Rigault et al. (2025a) and Smith et al. (in prep.) are data/product citations, while the scene-modeling method is credited to Astier et al. (2013); none of these supply an unverified premise that the paper's own equations assume. The conclusion that neither dataset is cosmology-ready follows from independently identified sensor effects and bandpass inaccuracies. The skeptic's concern about which pipeline is correct is a calibration-accuracy risk, not a circularity.
Assumptions & free parameters
free parameters (3)
- Color transformation coefficient alpha (per band, Eq. 5) =
g: ~6 nm blue, r: ~14 nm red, i: ~15 nm red relative to PS1 bandpasses
- SALT2.4 light-curve parameters x1, c, tmax (per SN) =
not tabulated; used for consistency and Hubble diagram comparison
- Per-exposure photometric zero points zpi (global alignment m_ij = m_j + zpi) =
uncertainties < 0.8 mmag (Fig. 3)
assumptions (4)
- domain assumption The scene-modeling estimator of Astier et al. (2013) is statistically optimal and unbiased with sky-noise-only pixel weights and a fixed position per exposure.
- domain assumption PS1 catalog photometry is an adequate relative flux reference for calibrating the SMP light curves.
- domain assumption The flux-dependent PSF residuals are a property of the sensor pixels, not artifacts of the PSF model or weight scheme.
- ad hoc to paper The pocket mechanism (charge escape into a reservoir during readout, delayed return) explains the observed effect.
invented entities (1)
-
Pocket effect (charge pocket or drift-field defect)
independent evidence
Cite this review
Pith. "Pith review of ZTF SNe Ia DR2: Towards cosmology-grade ZTF supernova light curves using scene modeling photometry." pith.science (2026). https://pith.science/paper/KBJ6KCO4
@misc{pith2026250904073,
author = {Pith},
title = {Pith review of: ZTF SNe Ia DR2: Towards cosmology-grade ZTF supernova light curves using scene modeling photometry},
year = {2026},
howpublished = {\url{https://pith.science/paper/KBJ6KCO4}},
note = {Machine review of arXiv:2509.04073}
}
read the original abstract
The Zwicky Transient Facility (ZTF) is conducting a wide-field survey of the northern sky in three optical bands and the collaboration cosmology working group has released 3628 spectroscopically confirmed Type Ia supernovae (SNe Ia) discovered during its first 2.5 years of operation. This "ZTF SN Ia DR2" sample is the largest SN Ia dataset to date. Fully exploiting this dataset to improve understanding of the properties of dark energy requires a photometric accuracy of O(0.1%). This can be achieved using Scene Modeling Photometry (SMP), which is optimal to extract a transient signal (SN) from a complex background (its host), while ensuring a common flux estimator with nearby stars used as calibration reference. In this paper, we present the status of the SMP development and use it to assess the precision and accuracy of the ZTF SN Ia DR2 force photometry light curves. We reach a repeatability of the star observations better than 1%. However, we have identified a new sensor effect, dubbed "pocket-effect", which distorts the Point Spread Function (PSF) in a flux-dependent manner leading to non-linearities in the photometry of a few percent. Correcting for this effect requires time- and sensor-dependent corrections to be applied at the pixel level, which is currently under development. This effects affects all light curve releases to date -- both from forced photometry and scene modelling preventing ZTF SN Ia DR2 to be used for accurate cosmological inference. Comparing the SMP and forced photometry measurements, we find that stretch and color estimated from both processings are consistent, aside from a 10 mmag shift in color. This assess the robustness of results presented as part of the the ZTF SN Ia DR2 release. The absolute calibration however shifts by 90 mmag. A reprocessing of the full ZTF SN Ia DR2 dataset using the SMP method is currently in progress.
Figures
Figures from the paper (10 more)
Forward citations
Cited by 1 Pith paper
-
ZTF-SEDm Type Ia supernova sample for Twins Embedding spectrophotometric standardisation
Applying the Twins Embedding 'Read Between The Lines' color standardization to 688 ZTF Type Ia supernovae yields 0.153 mag Hubble residual scatter and near-zero host steps, competitive with SALT photometric standardization.
Reference graph
Works this paper leans on
-
[1]
2025, Astronomy and Astrophysics, 694, A3, publisher: EDP ADS Bibcode: 2025A&A...694A...3A
Amenouche, M., Rosnet, P., Smith, M., et al. 2025, Astronomy and Astrophysics, 694, A3, publisher: EDP ADS Bibcode: 2025A&A...694A...3A
work page 2025
-
[2]
2014, Journal of Instrumentation, 9, C03048
Antilogus, P., Astier, P., Doherty, P., Guyonnet, A., & Regnault, N. 2014, Journal of Instrumentation, 9, C03048
2014
-
[3]
2013, Astronomy and Astrophysics, 557, A55
Astier, P., El Hage, P., Guy, J., et al. 2013, Astronomy and Astrophysics, 557, A55
work page 2013
-
[4]
2006, Astronomy and Astrophysics, 447, 31
Astier, P., Guy, J., Regnault, N., et al. 2006, Astronomy and Astrophysics, 447, 31
work page 2006
- [5]
- [6]
-
[7]
Bertin, E. & Arnouts, S. 1996, Astronomy and Astrophysics Supplement Series, 117, 393
work page 1996
-
[8]
2014, Astronomy and Astrophysics, 568, A22
Betoule, M., Kessler, R., Guy, J., et al. 2014, Astronomy and Astrophysics, 568, A22
work page 2014
Show all 39 references
-
[9]
D., Walters, R., et al
Blagorodnova, N., Neill, J. D., Walters, R., et al. 2018, Publications of the As- tronomical Society of the Pacific, 130, 035003
2018
-
[10]
C., Gordon, K
Bohlin, R. C., Gordon, K. D., & Tremblay, P. E. 2014, Publications of the Astro- nomical Society of the Pacific, 126, 711
2014
-
[11]
C., Hubeny, I., & Rauch, T
Bohlin, R. C., Hubeny, I., & Rauch, T. 2020, The Astronomical Journal, 160, 21
2020
-
[12]
2022, The Astrophysical Journal, 938, 110 Dask Development Team
Brout, D., Scolnic, D., Popovic, B., et al. 2022, The Astrophysical Journal, 938, 110 Dask Development Team. 2016, Dask: Library for dynamic task scheduling
2022
-
[13]
M., Riddle, R., et al
Dekany, R., Smith, R. M., Riddle, R., et al. 2020, Publications of the Astronom- ical Society of the Pacific, 132, 038001 DES Collaboration, Abbott, T. M. C., Acevedo, M., et al. 2024, The Dark Energy Survey: Cosmology Results With ~1500 New High-redshift Type Ia Super- novae ...
2020
-
[14]
J., Kulkarni, S
Graham, M. J., Kulkarni, S. R., Bellm, E. C., et al. 2019, Publications of the As- tronomical Society of the Pacific, 131, 078001, publisher: The Astronomical Society of the Pacific
2019
-
[15]
2007, Astronomy and Astrophysics, 466, 11
Guy, J., Astier, P., Baumont, S., et al. 2007, Astronomy and Astrophysics, 466, 11
2007
-
[16]
2010, Astronomy and Astrophysics, 523, A7
Guy, J., Sullivan, M., Conley, A., et al. 2010, Astronomy and Astrophysics, 523, A7
2010
-
[17]
2015, Astron- omy and Astrophysics, 575, A41
Guyonnet, A., Astier, P., Antilogus, P., Regnault, N., & Doherty, P. 2015, Astron- omy and Astrophysics, 575, A41
2015
-
[18]
A., Marriner, J., Kessler, R., et al
Holtzman, J. A., Marriner, J., Kessler, R., et al. 2008, The Astronomical Journal, 136, 2306
2008
-
[19]
2008, The Astrophysical Journal, 686, 749 Article number, page 15 of 16 A&A proofs: manuscript no
Kowalski, M., Rubin, D., Aldering, G., et al. 2008, The Astrophysical Journal, 686, 749 Article number, page 15 of 16 A&A proofs: manuscript no. aanda
2008
-
[20]
Lezmy, J., Copin, Y ., Rigault, M., Smith, M., & Neill, J. D. 2022, Astronomy and Astrophysics, 668, A43 LSST Science Collaboration, Abell, P. A., Allison, J., et al. 2009, aDS Bibcode: 2009arXiv0912.0201L
2022 arXiv
-
[21]
A., Schlafly, E
Magnier, E. A., Schlafly, E. F., Finkbeiner, D. P., et al. 2020, ApJS, 251, 6
2020
-
[22]
J., Laher, R
Masci, F. J., Laher, R. R., Rusholme, B., et al. 2019, Publications of the Astro- nomical Society of the Pacific, 131, 018003
2019
-
[23]
2024, arXiv e-prints, arXiv:2407.01650
Neveu, J., Kuhn, D., Souverin, T., & LEMAITRE collaboration. 2024, arXiv e-prints, arXiv:2407.01650
2024 arXiv
-
[24]
T., Bellm, E
Patterson, M. T., Bellm, E. C., Rusholme, B., et al. 2018, Publications of the As- tronomical Society of the Pacific, 131, 018001, publisher: The Astronomical Society of the Pacific
2018
-
[25]
A., Fremling, C., Sollerman, J., et al
Perley, D. A., Fremling, C., Sollerman, J., et al. 2020, The Astrophysical Journal, 904, 35, publisher: The American Astronomical Society
2020
-
[26]
1999, The Astrophysical Jour- nal, 517, 565
Perlmutter, S., Aldering, G., Goldhaber, G., et al. 1999, The Astrophysical Jour- nal, 517, 565
1999
-
[27]
J., et al
Rest, A., Scolnic, D., Foley, R. J., et al. 2014, The Astrophysical Journal, 795, 44
2014
-
[28]
G., Filippenko, A
Riess, A. G., Filippenko, A. V ., Challis, P., et al. 1998, The Astronomical Journal, 116, 1009
1998
-
[29]
D., Blagorodnova, N., et al
Rigault, M., Neill, J. D., Blagorodnova, N., et al. 2019, Astronomy and Astro- physics, 627, A115
2019
-
[30]
2023, Union Through UNITY: Cos- mology with 2,000 SNe Using a Unified Bayesian Framework
Rubin, D., Aldering, G., Betoule, M., et al. 2023, Union Through UNITY: Cos- mology with 2,000 SNe Using a Unified Bayesian Framework
2023
-
[31]
V ., Kowalski, M., et al
Rubin, D., Linder, E. V ., Kowalski, M., et al. 2009, The Astrophysical Journal, 695, 391
2009
-
[32]
Schlafly, E. F. & Finkbeiner, D. P. 2011, The Astrophysical Journal, 737, 103, publisher: IOP ADS Bibcode: 2011ApJ...737..103S
2011
-
[33]
P., Suntzeff, N
Schmidt, B. P., Suntzeff, N. B., Phillips, M. M., et al. 1998, The Astrophysical Journal, 507, 46
1998
-
[34]
2015, aDS Bibcode: 2015arXiv150303757S
Spergel, D., Gehrels, N., Baltay, C., et al. 2015, aDS Bibcode: 2015arXiv150303757S
2015
-
[35]
Stetson, P. B. 1987, Publications of the Astronomical Society of the Pacific, 99, 191
1987
-
[36]
2011, The Astrophysical Journal, 737, 102
Sullivan, M., Guy, J., Conley, A., et al. 2011, The Astrophysical Journal, 737, 102
2011
-
[37]
2012, The Astrophysical Journal, 746, 85
Suzuki, N., Rubin, D., Lidman, C., et al. 2012, The Astrophysical Journal, 746, 85
2012
-
[38]
E., et al
Taylor, G., Lidman, C., Tucker, B. E., et al. 2021, Monthly Notices of the Royal Astronomical Society, 504, 4111
2021
-
[39]
O., & Gal-Yam, A
Zackay, B., Ofek, E. O., & Gal-Yam, A. 2016, The Astrophysical Journal, 830, 27, publisher: The American Astronomical Society Article number, page 16 of 16
2016
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.