{"id":"1efb1c8b-2ea8-42e4-8883-477821950336","arxiv_id":"2509.04073","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"ZTF's scene modeling photometry, compared against its forced photometry release, shows a new CCD pocket effect that adds up to 7% flux-dependent nonlinearities and makes the DR2 supernova light curves unusable for precision cosmology until reprocessed.","lead":"The ZTF collaboration's scene modeling pipeline shows that its 3,628 supernova light curves cannot yet be used for precision cosmology: a newly identified CCD readout effect, the pocket effect, distorts star images in a brightness-dependent way, adding up to 7% nonlinearities. The same effect likely contaminates the published DR2 light curves, so the team is reprocessing the full dataset with pixel-level corrections.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 90 mmag DR2-vs-SMP offset is attributed to DR2 calibration only by assuming SMP is the accurate chain; the paper's own caveat (§6, §7) that SMP nonlinearities may contribute means the strongest quantitative claim lacks an independent anchor.","rationale":"The paper is an honest status report whose central negative conclusion—that neither the current DR2 fluxes nor the current SMP measurements support precise cosmological analysis—is supported by multiple independent pieces of evidence: direct PSF-skewness measurements that are flux-dependent (§6.1), PSF-to-aperture nonlinearities up to 7% (§6.2), ~1% repeatability limits, and the paper's own explicit limitation list (§6, §7). The concern I identify does not undermine that broader conclusion. It targets a narrower but headline-level claim: that the 90 mmag SMP-DR2 offset specifically indicates a failure of DR2 absolute calibration. For that attribution, the SMP calibration chain must be trusted at the ~90 mmag level, but the same pocket effect corrupts the SMP field-star fluxes used in the calibration anchor, and the paper itself says it cannot rule out that SMP nonlinearities contribute. This is a genuine soft spot, but the reader's CONDITIONAL verdict already accounts for it by not treating the 90 mmag number as fully established. Therefore I do not recommend changing the verdict. The proposed test would settle the attribution by anchoring SMP to an independent absolute reference; until that is done, the safest public statement is that the 90 mmag offset shows at least one of the two pipelines is miscalibrated, not that DR2 is the guilty one.","tokens_in":22094,"tokens_out":3468,"duration_ms":36112,"concrete_test":"Select ~100 SNe with both SMP and DR2 light curves and recalibrate the SMP magnitudes against CALSPEC (or the in-prep ZTF standard-star catalog / GAIA spectrophotometry) instead of PS1. Then recompute the SMP-minus-DR2 distance-modulus offset with quoted uncertainties. If SMP-vs-external is ~0 and DR2-vs-external remains ~90 mmag, the DR2 absolute-calibration claim is confirmed. If SMP-vs-external is itself ~90 mmag (or similar in magnitude to the SMP-DR2 offset), then the offset cannot be attributed to DR2, and the §7 conclusion should be rephrased to reflect that the comparison alone does not identify which pipeline is wrong.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative result—'absolute calibration shifts by 90 mmag' (abstract, §5/Fig. 12)—only counts as evidence against DR2 if the SMP calibration chain is the more accurate of the two. The SMP chain is anchored to PS1 through field-star SMP fluxes, but the same pocket effect that distorts PSFs (up to 7%, §6.2) enters the SMP star-light-curve fluxes used for that anchor. The paper concedes in §6 and §7 that 'we cannot rule out the possibility that the large nonlinearities identified in this study contribute to this discrepancy.' If the SMP absolute calibration is itself biased by a comparable amount, the 90 mmag offset mainly diagnoses SMP, not DR2, and the specific statement that 'the absolute calibration of the DR2 pipeline is not known with the precision needed' is not established by this comparison. The broader conclusion that neither dataset is cosmology-ready survives, because the pocket-effect nonlinearities and 3 nm bandpass errors independently preclude 0.1% accuracy; but the headline claim of a 90 mmag DR2 absolute-calibration failure rests on an unverified assumption about which pipeline is right.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents the current status of Scene Modeling Photometry (SMP) for the ZTF SN Ia DR2 sample. It describes the full pipeline (PSF training, astrometric transforms, photometric alignment, scene-model fit, PS1-anchored calibration), demonstrates its scalability on the 179 TB dataset, and reports a photometric repeatability of about 1% for field stars. The paper identifies a new CCD effect, the 'pocket effect', causing flux-dependent PSF distortions and photometric nonlinearities up to 7%, and finds that current bandpass models are not accurate enough for 0.1% cosmology. Comparing SMP with the DR2 forced-photometry light curves, it reports consistent x1 and tmax, a 10 mmag color shift, and a 90 mmag distance-modulus offset, interpreted as evidence that the DR2 absolute calibration is inadequate. The paper concludes that neither the DR2 nor the current SMP light curves are suitable for precise cosmological analyses, and outlines plans for a pixel-level pocket-effect correction, a CALSPEC-based calibration chain, and improved bandpass models.","tokens_in":22367,"tokens_out":7324,"duration_ms":66114,"significance":"The paper makes two contributions. First, it demonstrates a scalable SMP pipeline that processes the full DR2 dataset in two to three weeks, producing light curves for the largest SN Ia sample to date; this is a substantial technical achievement. Second, it provides direct, falsifiable empirical evidence for a new sensor effect ('pocket effect'): flux-dependent PSF skewness (Fig. 13), astrometric residuals (Fig. 2), and PSF-to-aperture flux ratios (Fig. 14) are all measured against external Gaia and PS1 references. This part of the paper is solid and will be important for ZTF and similar surveys. The conclusion that neither DR2 nor the current SMP internal release is suitable for 0.1% cosmology is strongly supported by the repeatability floor (~1%), the pocket-effect nonlinearities (up to 7%), and the bandpass model discrepancies. The weaker link is the interpretation of the 90 mmag SMP-DR2 offset as a DR2 calibration failure; the data do not identify which pipeline is biased. If the pocket-effect correction and CALSPEC chain are completed as described, the pipeline has clear potential to deliver cosmology-grade light curves.","major_comments":[{"comment":"The headline offset of 90 mmag in distance modulus is presented as evidence that 'the absolute calibration of the DR2 pipeline is not known with the precision needed' (abstract and §5). This attribution is not established. The SMP zero-point is anchored to PS1 through SMP photometry of field stars (Eq. 5, §3.9); those same field-star fluxes are subject to the pocket-effect nonlinearities quantified in §6.1. The paper's own §7 states 'we cannot rule out the possibility that the large nonlinearities identified in this study contribute to this discrepancy.' The data therefore support a 90 mmag discrepancy between two pipelines, not a verdict on which pipeline is biased. I request either (a) an independent check of the SMP absolute scale (e.g., comparison of SMP star fluxes to aperture photometry on low-sky frames or to CALSPEC standards), or (b) a rewording of the abstract and §5 to present","section":"§5, Fig. 12, §7"},{"comment":"The statement in the conclusion that 'current models are precise at the 3 nm level' conflicts with the measurements shown in Fig. 15. There, the SNCosmo bandpass models are found to be inaccurate by ~30 Å in g and ~15 Å in r, and the improved bench-based model deviates by ~60 Å in g from the stellar color terms. A 60 Å error is 6 nm, not 3 nm, and even the 30 Å SNCosmo error is at the upper edge of what the text calls the 3 nm level. The requirement for dark-energy measurements is ~3 Å in mean wavelength; the actual model errors are an order of magnitude larger. The conclusion should quote the measured model errors (Fig. 15) rather than the focal-plane non-uniformity from §2.2, so that the severity of this limitation is stated accurately.","section":"§6.3, Fig. 15, §7 item 3"}],"minor_comments":[{"comment":"The sample size is inconsistent: the abstract and §2.3 state 3628 SNe Ia, while §4.2 says '3582 SNe (out of 3826)'. Please reconcile the numbers and clarify which total is correct.","section":"Abstract and §4.2"},{"comment":"Typos and grammar: 'This assess the robustness' should be 'This assesses the robustness'; 'the the ZTF SN Ia DR2 release' has a duplicated article; 'altough' → 'although'; 'directely' → 'directly'; 'sufficient' → 'sufficient'. A careful proofread is needed.","section":"Throughout"},{"comment":"The variable 'col' is not defined at first use. It should be stated that col is the PS1 g-i color (or whichever color is used in the calibration). Also, the term δzp(x) is not specified; if it is a position-dependent term, its functional form should be given.","section":"§3.9, Eq. (5)"},{"comment":"The Hubble-diagram residuals appear to show a redshift-dependent trend (black binned points), which the text only describes as 'a slight variation in redshift'. Please quantify this trend and state whether it affects the interpretation of the 90 mmag offset.","section":"Fig. 12"}],"recommendation":"major_revision","confidential_remarks":"The paper is an important status report for a major dataset. The empirical characterization of the pocket effect is solid and deserves publication. The main risk to the published record is the over-strong attribution of the 90 mmag offset to DR2 calibration; the authors should either provide an independent anchor or soften the abstract. Also check the sample-size inconsistency (3628 vs 3826). The paper is appropriate for the journal's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Thanks for the report. Here's my take.\n\nThe paper is a genuinely useful status report. The one new thing that matters is the pocket effect—a readout artifact that makes the PSF shape flux-dependent on some ZTF sensors, with photometric nonlinearities up to 7% on the worst chips. The evidence is direct: astrometric residuals in the serial direction that drift with star flux, PSF third-moment skewness that changes with magnitude, and PSF-to-aperture ratios that curve by 1-7%. The pipeline is also a real engineering achievement: 179 TB of images processed in two to three weeks, star repeatability below 1%, and a common flux estimator for SNe and field stars. The authors are explicit that neither DR2 forced photometry nor their own SMP light curves are usable for precision cosmology yet, and that negative conclusion is well supported.\n\nThe weakest part is the 90 mmag absolute calibration offset. It is a cross-pipeline comparison, not an external calibration. The paper's abstract says the offset 'suggests the absolute calibration of the DR2 pipeline is not known' to the needed precision, but that only follows if SMP is the accurate chain. The paper itself says it cannot rule out that the SMP nonlinearities contribute to the discrepancy. So the specific claim about DR2's calibration being wrong by 90 mmag is not established. What is established is a large pipeline-to-pipeline disagreement, which, combined with the pocket effect and the ~3 nm bandpass errors, is enough to keep the sample out of cosmology. The authors should soften the causal language.\n\nMinor: the 10 mmag color shift and 90 mmag distance shift are quoted without uncertainties. The statement that the effect affects all light curve releases is an inference, though a reasonable one. Neither changes my verdict. The circularity burden is low: the comparisons are empirical and anchored to Gaia and PS1.\n\nBottom line: this deserves a serious referee. Anyone using ZTF DR2, and anyone building survey photometry pipelines, should read it. I'd send it to review, with a request to clarify what the 90 mmag offset does and does not demonstrate.","headline":"Honest, useful status report: the pocket effect and the 'not cosmology-ready' conclusion hold up, but the 90 mmag DR2 calibration claim is weaker than the abstract suggests.","tokens_in":23124,"tokens_out":3372,"would_cite":true,"duration_ms":32368,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A newly identified CCD readout artifact, the 'pocket effect,' makes ZTF supernova photometry brightness-dependent by up to 7%, ruling out both the DR2 and scene-modeling light curves for precision cosmology.","keywords":["type Ia supernovae","scene modeling photometry","Zwicky Transient Facility","photometric calibration","CCD sensor effects","pocket effect","Hubble diagram","dark energy"],"falsifier":"Take one of the affected CCDs and measure the PSF-to-aperture magnitude ratio as a function of source flux while artificially raising the sky background above the ~6000 ADU threshold where the pocket effect is said to disappear; if the flux-dependent nonlinearity persists at high background, the charge-pocket mechanism is wrong. Conversely, if applying the pixel-level correction now under development removes both the flux-dependent PSF skewness and the 90 mmag SMP–DR2 distance offset, the paper's central causal chain is confirmed.","tokens_in":21920,"feed_emoji":"🔭","tokens_out":12751,"duration_ms":104864,"temperature":0.7,"pith_summary":"The paper's goal is a photometry pipeline accurate enough to use the 3,628 Type Ia supernovae of the ZTF SN Ia DR2 sample — the largest such dataset to date — to test dark energy at the few-tenths-of-a-percent level. It shows that scene modeling photometry (SMP), which fits the supernova and its host galaxy jointly and applies the exact same flux estimator to calibration stars, can process the full 179-terabyte dataset in about two weeks and reach better than 1% repeatability. But it also uncovers a sensor artifact it names the pocket effect: a readout flaw that distorts the point-spread function in a brightness-dependent way, causing photometric nonlinearities of up to 7% in every ZTF transient light curve taken since November 2019 — including both the released DR2 forced photometry and the new SMP light curves. Comparing the two pipelines, stretch and peak time agree and color shifts by only 10 mmag, but distances differ by 90 mmag, and the paper concludes that neither data product can yet anchor a cosmological analysis until the effect is corrected at the pixel level.","feed_headline":"Pocket effect skews supernova brightness by up to 7 percent","feed_subtitle":"The flaw hits every ZTF transient light curve since 2019, so the largest SN Ia sample must be reprocessed.","key_machinery":"Two objects carry the argument. Scene modeling photometry (SMP) models each supernova's scene as a per-exposure PSF plus a single host-galaxy model shared across all exposures and convolved with a seeing-matching kernel; it is a maximum-likelihood estimator that can be applied unchanged to field stars, so an external calibration (here the Pan-STARRS catalog) propagates through the same flux estimator used for the supernova. The pocket effect is a readout artifact discovered during this work: during serial readout, a fraction of the charge can escape into a 'pocket' — possibly a trap or a drift-field defect — and return after tens of microseconds; the escape probability depends on how full th","core_discovery":"The central claim: the ZTF SN Ia DR2 sample — the largest low-redshift supernova dataset to date — cannot yet anchor precision cosmology because of a newly identified sensor effect. Using scene modeling photometry, which applies one common flux estimator to the supernova and to the field stars used for calibration, the authors reprocess all 3,628 supernovae. SMP and forced-photometry light curves agree on stretch and time of maximum, with only a 10 mmag color shift — so the DR2 population studies stand. But distance moduli differ by 90 mmag, and the likely cause is the pocket effect: a readout artifact in which a flux-dependent fraction of charge escapes and returns during transfer, distorti","pith_inferences":["If the pocket effect is as universal as claimed, other surveys using CCDs with similar readout designs should look for the same signature — flux-dependent serial-direction PSF skewness that diminishes at sky backgrounds above ~6000 ADU — before trusting their own PSF photometry.","The 90 mmag distance-modulus offset, if it stems from DR2's calibration rather than SMP's residual nonlinearities, would shift any low-redshift anchor built on DR2 by more than its statistical uncertainty; quantifying that shift on published dark-energy constraints is a natural follow-up the paper leaves implicit.","A clean test the paper does not run: comparing both pipelines on supernovae observed before November 2019, where the pocket effect is stated to be negligible, would separate the pocket effect from any other calibration offset and test the 90 mmag attribution directly.","The per-CCD bandpass differences between single- and dual-layer anti-reflective coatings imply that using a single average bandpass per filter — as the released photometry does — introduces position-dependent color biases; a spatially resolved bandpass model may be as important as the sensor correction for 0.1% cosmology."],"forward_implications":["Every ZTF transient light curve taken since November 2019 carries a flux-dependent photometric error of up to ~7%, so any analysis reaching for percent-level accuracy must wait for the reprocessing.","The pixel-level correction under development, combined with the demonstrated two-to-three-week reprocessing capability, makes a corrected 'DR2.5' release of the full 3,628-SN sample achievable.","The consistency of stretch and peak time between the two pipelines, with only a 10 mmag color shift, means the DR2 papers' supernova-population conclusions are not overturned by the calibration problems.","Reaching the 0.1% photometric accuracy target requires, in addition to the sensor correction, ZTF bandpass models accurate to roughly 3 Å; current models are off by 15–60 Å in parts of the focal plane.","A CALSPEC-based calibration chain, replacing the current Pan-STARRS anchor, is the intended route to an absolute flux scale consistent across the g, r, and i bands."],"supporting_citations":[{"why":"Supplies the scene-modeling method (a joint PSF+galaxy maximum-likelihood fit) on which the entire SMP pipeline is built.","marker":"Astier et al. 2013"},{"why":"Releases the 3,628-supernova ZTF SN Ia DR2 sample whose forced-photometry light curves are reanalyzed and compared here.","marker":"Rigault et al. 2025a"},{"why":"Provides the DR2 forced-photometry light curves and their reported ~30 mmag calibration uncertainty, the comparison baseline for SMP.","marker":"Smith et al. (in prep.)"},{"why":"Supplies the detrended ZTF science frames (and masks, WCS) that form the input data for the SMP pipeline.","marker":"Masci et al. 2019"},{"why":"The ZOGY difference-imaging algorithm underlying the DR2 forced-photometry measurements that SMP is checked against.","marker":"Zackay et al. 2016"},{"why":"The Pan-STARRS catalog used as the current calibration anchor for the SMP light curves.","marker":"Magnier et al. 2020"},{"why":"The SALT2 light-curve model used to fit the SN data and extract stretch, color, and peak time from both pipelines.","marker":"Guy et al. 2007"},{"why":"Supplies SALT2 version 4 and the alpha/beta/M_B values used in the Tripp relation to build the Hubble diagram.","marker":"Betoule et al. 2014"},{"why":"Established the brighter-fatter effect, the closely related sensor nonlinearity that the new pocket-effect diagnosis is set against.","marker":"Antilogus et al. 2014; Guyonnet et al. 2015"}],"fun_headline_variants":["Pocket effect skews every ZTF supernova from 2019 onward","Largest supernova sample unusable for cosmology due to pocket effect","New sensor defect forces redo of ZTF supernova photometry","Pocket effect causes 7% brightness errors in 3,628 SNe","Reprocessing needed: pocket effect hits 3,628 Type Ia supernovae"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The claim that DR2's absolute calibration is off by 90 millimag assumes the scene-modeling calibration chain, anchored on Pan-STARRS star magnitudes, is the more accurate of the two pipelines — and the paper concedes its own unresolved nonlinearities could instead be the cause.","fun_headline_variants_meta":{"raw":{"variants":["Pocket effect skews every ZTF supernova from 2019 onward","Largest supernova sample unusable for cosmology due to pocket effect","New sensor defect forces redo of ZTF supernova photometry","Pocket effect causes 7% brightness errors in 3,628 SNe","Reprocessing needed: pocket effect hits 3,628 Type Ia supernovae"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000294,"raw_usage":{"total_tokens":1633,"prompt_tokens":918,"completion_tokens":715,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":662,"completion_tokens_details":{"reasoning_tokens":615}},"tokens_in":662,"tokens_out":715,"duration_ms":6665,"temperature":1.0,"reasoning_tokens":615,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T10:25:38.330852+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take one of the affected CCDs and measure the PSF-to-aperture magnitude ratio as a function of source flux while artificially raising the sky background above the ~6000 ADU threshold where the pocket effect is said to disappear; if the flux-dependent nonlinearity persists at high background, the charge-pocket mechanism is wrong. Conversely, if applying the pixel-level correction now under development removes both the flux-dependent PSF skewness and the 90 mmag SMP–DR2 distance offset, the paper's central causal chain is confirmed.","supporting_citations":[{"cited_title":"2013, Astronomy and Astrophysics, 557, A55","cited_arxiv_id":null,"evidence_quote":"Supplies the scene-modeling method (a joint PSF+galaxy maximum-likelihood fit) on which the entire SMP pipeline is built."},{"cited_title":"J., Laher, R","cited_arxiv_id":null,"evidence_quote":"Supplies the detrended ZTF science frames (and masks, WCS) that form the input data for the SMP pipeline."},{"cited_title":"O., & Gal-Yam, A","cited_arxiv_id":null,"evidence_quote":"The ZOGY difference-imaging algorithm underlying the DR2 forced-photometry measurements that SMP is checked against."},{"cited_title":"2007, Astronomy and Astrophysics, 466, 11","cited_arxiv_id":null,"evidence_quote":"The SALT2 light-curve model used to fit the SN data and extract stretch, color, and peak time from both pipelines."},{"cited_title":"2014, Astronomy and Astrophysics, 568, A22","cited_arxiv_id":null,"evidence_quote":"Supplies SALT2 version 4 and the alpha/beta/M_B values used in the Tripp relation to build the Hubble diagram."}],"review_version":1}