{"id":"32c09662-d0ee-4519-be7c-5f78eb88f8f3","arxiv_id":"2505.04686","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A reanalysis of ZTF volume-limited Type Ia supernovae finds color-dependent, continuous host-galaxy trends that favor dust-driven intrinsic scatter, with the new continuous model Host2D preferred over the standard mass-step model at 4 sigma.","lead":"This paper reanalyzes 796 nearby supernovae from the Zwicky Transient Facility and finds that the remaining brightness scatter depends on supernova color and host galaxy mass in a continuous, dust-like way, not a simple two-population jump. The authors build a new model, Host2D, which they say explains the data 4 sigma better than the standard 'mass-step' model.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 4.0σ/3.5σ claims rest on chi-square tail probabilities that are not calibrated for non-nested models or an in-sample fixed σ_int; a null simulation is needed.","rationale":"I read this paper as a reanalysis whose central quantitative claim is the significance of the continuous, color-dependent Host2D model over the discrete mass-step model. The direction of the offset trends and the color-dependent scatter difference are supported by nonparametric tests and by the public code, so this is not a challenge to the basic phenomenology. The load-bearing weak point is the significance calibration: Eq. 7 is applied to non-nested models, the spline knots and host split points are optimized in-sample, and σ_int is fixed in-sample to make the base reduced chi-square close to unity. This is essentially the same weakness the reader identified, though I put at least as much weight on the non-nested/in-sample optimization issue as on the exact value of σ_int. The concrete null simulation is a single, feasible check that would settle whether the 4.0σ improvement is real or an artifact of uncalibrated model comparison. I therefore recommend no adjustment to the reader's conditional verdict: acceptance should require either this calibration or a more modest, clearly labeled claim.","tokens_in":19450,"tokens_out":7024,"duration_ms":74618,"concrete_test":"Run a null-model simulation: generate 796 SNe from the best-fit mass-step model #12 using the ZTF covariance matrices and σ_int = 0.17; apply the same cuts and SALT2mu fitting; for each realization fit both the mass-step model and Host2D with free knot positions, recording the Δχ². If the observed Δχ² = 36.8 (nominal 4.0σ) occurs at <1% frequency, the claim survives; if it occurs at >5% frequency, the significance is inflated by non-nested/in-sample optimization. Repeat the exercise with σ_int = 0.15 and 0.20 to bracket the in-sample scale choice.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim is the significance of Host2D over the mass-step model (Sec. 4.5, Table 1). These significances come from converting Δχ² into σ via Eq. 7, a chi-square tail probability with Δν degrees of freedom. That conversion is valid only if (1) the denominator in Eq. 6 is known exactly and (2) the models being compared are nested with identifiable parameters. Neither condition is established. σ_int is fixed at 0.17 mag because it makes the 2-parameter Tripp reduced chi-square close to unity (Sec. 2); the same data set is used to set the scale of the Δχ² being tested, so the reported significances are conditional on an in-sample choice. Moreover, the mass-step model (a discontinuous host-property shift) and Host2D (Eq. 10, a continuous bilinear spline with knot positions c0,h0 optimized) are non-nested; Δν = 9 is not the effective number of degrees of freedom, and the host split points 9.5 and 10.3 are selected from the residual profile (Sec. 3.2). The nonparametric scatter difference (3.4σ) and public code are real supporting evidence, but the headline '4.0σ improvement' and 'strongest evidence for continuity' are not robust until the significance is calibrated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reanalyzes the ZTF DR2 volume-limited Type Ia supernova sample (z<0.06) with the SALT2mu fitting path and literature-consistent cuts, with the goal of explaining why the ZTF collaboration's G25a analysis did not see evidence for the dust-based, color-dependent 'alligator-mouth' residual pattern. The authors report a color-dependent host-mass separation (0.18±0.09 mag increase from blue to red), a larger scatter for red SNe than blue SNe at 3.4σ, and a new continuous model, Host2D, which they claim improves over the mass-step model at 4.0σ and over a color-dependent step model at 3.4σ. They attribute the difference from G25a to binning, split-point choices, and simultaneous fitting of the step parameter γ rather than to a difference in the sample.","tokens_in":19704,"tokens_out":6977,"duration_ms":70433,"significance":"If the central claims hold, the paper is significant: it would reconcile the large ZTF volume-limited sample with the dust-based trends seen in DES, Pantheon+, and other samples, and it would strengthen the case that a simple discrete mass step is an insufficient standardization correction. The paper has real strengths: the Brown-Forsythe scatter comparison and the blue/red separation statistic are model-independent diagnostics; the analysis is transparently documented; and the Host2D code is public. The main risk is that the headline significance claims rest on an in-sample calibration of the intrinsic scatter and on non-nested model comparisons that have not been calibrated by simulations or cross-validation, so the parametric evidence for the specific quantitative claims is currently weaker than the abstract suggests.","major_comments":[{"comment":"The headline 4.0σ improvement (model #14 vs #12) and the related 3.3–3.4σ claims are obtained by converting Δχ² to a p-value with Δν degrees of freedom, but the preconditions for that conversion are not met. The intrinsic scatter σ_int is fixed at 0.17 mag in Sec. 2 so that the reduced chi-square of the two-parameter Tripp residuals is close to unity, meaning the scale of the Δχ² being tested is calibrated on the same data used for the test. The mass-step and Host2D models are non-nested, Host2D's knot locations and spline coefficients are optimized on the residuals being compared, and Δν=9 is not the effective parameter count. In addition, the Host2D rows in Table 1 keep α=0.141 and β=3.006 fixed at the Tripp-only values, while the step-model rows refit α and β, so the Δχ² mixes the spline gain with the α,β difference. The manuscript explicitly states that no full-scale simulations are performed (Sec. 1, final paragraph). The 4.0σ and 'strongest evidence to date (3.5σ)' claims are therefore uncalibrated until a null-simulation or cross-validation study fixes the false-positive rate.","section":"Sec. 4, Eq. (7) and Table 1"},{"comment":"The low- and high-mass split points (9.5 and 10.3 dex) are selected from the observed residual profile of the same sample, and the 0.18±0.09 mag increase in the low/high separation is then used as evidence for the alligator-mouth shape. Because the split points are post hoc, the separation statistic and its uncertainty are in-sample quantities and cannot be assigned the implied statistical weight. I ask for a validation that either fixes the splits a priori based on independent criteria or propagates the split-selection step (for example through a bootstrap that re-selects the split points), and for the abstract and conclusions to report the separation with that caveat.","section":"Sec. 3.2 and Fig. 6"},{"comment":"Host2D is built by fitting an 11-parameter spline to the rolling-median plus Monte-Carlo-smoothed residual map computed from the same data, and Table 1 then compares the resulting χ² against models evaluated on the raw residuals. The smoothing and bin-level fitting reduce the effective fluctuation level without contributing to Δν, so the reported improvements over BS21-lite (for example the 3.4σ quoted for model #14 versus #13) can be optimistic. Please repeat the comparison with cross-validation (fit on a training subset and evaluate χ² on a held-out subset) or on null realizations generated from the Tripp-only model.","section":"Sec. 4.4 and Fig. 8"}],"minor_comments":[{"comment":"The abstract reports the 'strongest evidence to date (3.5σ)' for continuity, but Sec. 4.4 quotes 3.4σ for the Host2D versus BS21-lite comparison; please reconcile these numbers.","section":"Abstract"},{"comment":"There are two typographical errors: 'in rediction' should be 'reduction' and 'prehaps' should be 'perhaps'.","section":"Sec. 4.1 and Sec. 4.4"},{"comment":"The text calls σ²µ the 'uncertainty in the distance modulus,' but the expression includes the intrinsic scatter term σ²int; please clarify that this is a total error budget and state how σ_int enters the bootstrap procedure described in Sec. 3.1.","section":"Eq. (3)"},{"comment":"The sentence introducing the p-value contains a drafting artifact: 'The significance is measured by the p-value3 may' should be completed or the stray 'may' removed.","section":"Sec. 4.1 and Eq. (7)"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper deserves a serious referee, but the headline '4.0 sigma' needs to be taken with a grain of salt. The real contribution is the careful reanalysis of the ZTF volume-limited sample and the demonstration that analysis choices—binning, split points, and whether you fit the step parameter simultaneously—can change the conclusion about the mass-step and color dependence. The nonparametric diagnostics are the strongest part: the Brown-Forsythe test shows red SNe have ~33% larger scatter at 3.4 sigma, and the blue/red separation difference of 0.18±0.09 mag is a clean, model-independent statement. That is a solid piece of evidence that the residual structure is color-dependent.\n\nThe new Host2D model is a continuous bivariate spline over color and host mass with 11 parameters. The paper argues it beats the mass-step model by 4.0 sigma and the BS21-lite model by 3.4 sigma. Those numbers are computed from chi-square differences converted to p-values using a chi-square tail with Δν degrees of freedom. The problem is that the significance is only legitimate if the intrinsic scatter sigma_int is known and the models are nested. Neither is true: sigma_int is fixed at 0.17 mag to make the reduced chi-square of the Tripp model close to unity, and Host2D is not nested with the step model, and the knot locations are optimized on the same data. So the 4.0 sigma is an in-sample, conditional statement. The authors should have done a null simulation or cross-validation to calibrate the significance. They mention they don't do full simulations, but that is exactly what this claim needs.\n\nThe split points at 9.5 and 10.3 are also post-hoc, chosen from the residual profile. That's not fatal—the nonparametric test and the separation difference don't depend on those cuts—but it does mean the binned 'alligator mouth' figure is partly constructed. The qualitative result, that the trend is continuous rather than a sharp step, is probably right, but the quantitative significance is not robust.\n\nThe citation pattern looks fair, and the code is public. The authors also explicitly state their limitations, which is good. Overall, this is a useful paper for anyone working on SN Ia standardization or H0. With a sensitivity analysis of sigma_int and a null simulation test, the significance claims would be much more convincing. I'd send it to peer review with major revisions.","headline":"Solid nonparametric evidence for color-dependent residual scatter, but the headline significance claims are not calibrated and need revision.","tokens_in":20302,"tokens_out":2593,"would_cite":true,"duration_ms":24467,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ZTF's volume-limited SNe Ia show a continuous, color-dependent host correction is preferred over the discrete mass-step model.","keywords":["Type Ia supernovae","Hubble residuals","mass step","host-galaxy properties","dust extinction","ZTF DR2","continuous scatter model"],"falsifier":"Refit the same 796 supernovae with $\\sigma_{\\mathrm{int}}$ treated as a free parameter or marginalized out and recompute $\\Delta\\chi^2$ between Tripp+step and Host2D; if the preference drops below about $2\\sigma$, the continuous-model claim weakens. Alternatively, run the split-point selection on simulated data generated from a dust-free step model and check whether the 9.5 and 10.3 dex splits still produce a $0.18$ mag alligator-mouth profile.","tokens_in":19182,"feed_emoji":"🔭","tokens_out":5877,"duration_ms":54889,"temperature":0.7,"pith_summary":"This paper reanalyzes the 945 Type Ia supernovae in the volume-limited ZTF DR2 sample with the same fitting conventions used for earlier large samples, and argues that the data support a dust-driven, color-dependent explanation of residual scatter rather than a discrete progenitor-step picture. The authors report that the host-mass “step” depends on SN color — red supernovae show a larger host-dependent residual than blue ones by $0.18 \\pm 0.09$ mag — and that red supernovae have roughly $\\sim 33\\%$ larger residual scatter than blue ones at more than $3\\sigma$ significance. Their new correction model, Host2D, treats the residual as a continuous surface over SN color and host mass; it improves the fit over the standard mass-step model by $4.0\\sigma$, which they call the strongest evidence to date that this relation is continuous rather than step-like. If correct, the standard mass-step correction is insufficient and cosmology analyses should adopt a continuous, color-dependent host correction.","feed_headline":"ZTF supernovae favor continuous host-dust correction over mass-step","feed_subtitle":"Reanalysis of 945 volume-limited SNe Ia reports a 4.0σ preference for a color-dependent, continuous host-galaxy correction.","key_machinery":"The load-bearing object is Host2D, an 11-parameter correction map built as a bivariate spline with a single knot: in each of the four regions around the knot, the residual is a linear function of SALT2 color $c$ and host property $h$, with the two regions connected continuously. Its best-fit knot lands near $c_0 \\approx 0.04$ and host mass $\\approx 10.03$, matching the point where the alligator-mouth divergence begins. The paper also relies on rolling-median binning with Monte Carlo bootstrapping to reveal the continuous profile, a three-way host-mass split (at $9.5$ and $10.3$ dex) to expose the intermediate population, and a fixed intrinsic scatter $\\sigma_{\\mathrm{int}} = 0.17$ mag used in the $\\chi^2$ comparisons of models.","core_discovery":"The central claim is that the ZTF DR2 volume-limited sample, analyzed in the same way as other large SNe Ia samples, reproduces the “alligator-mouth” pattern predicted by dust-scatter models: the host-galaxy dependence of Hubble residuals nearly vanishes for blue supernovae near $c \\approx -0.1$ (separation $0.05 \\pm 0.06$ mag) and opens up to $0.23 \\pm 0.07$ mag for red supernovae near $c \\approx 0.1$, a color-dependent increase of $0.18 \\pm 0.09$ mag. The paper further claims that red supernovae show $\\sim 33\\%$ larger residual scatter than blue supernovae ($3.4\\sigma$), that a color-dependent model beats a simple step model at $\\sim 3\\sigma$, and that the relationship between SN color, host-galaxy properties, and luminosity is continuous rather than characterized by a discrete step ($3.5\\sigma$). Its new Host2D model improves the fit over the mass-step model by $4.0\\sigma$, and the paper traces the earlier ZTF conclusion to differences in binning, split locations, and fitting method rather than to the data itself.","pith_inferences":["If Host2D’s significance depends on the fixed $\\sigma_{\\mathrm{int}} = 0.17$ mag, then marginalizing over $\\sigma_{\\mathrm{int}}$ or allowing it to be color-dependent could move the reported $4.0\\sigma$ preference; this test is not run in the paper.","The host-mass split points at 9.5 and 10.3 dex are read off the same residuals used to claim continuity, so a fully unbiased validation would preselect those splits (e.g., on a simulated sample) before measuring the $0.18$ mag color dependence.","A dust interpretation also predicts a correlation between residual scatter and host extinction or reddening; checking whether redder-host supernovae concentrate at the high-scatter end would extend the dust reading beyond mean offsets."],"forward_implications":["The standard mass-step correction, which applies a single offset split at one host-mass value, underfits the data; a color-dependent surface is needed.","Cosmology analyses should expect redder supernovae in massive hosts to carry larger correction uncertainties, since residual scatter grows with color.","The difference between the earlier ZTF conclusion and this paper’s conclusion is attributable to analysis methodology, not to a difference in the sample.","A natural next step is extending Host2D into stretch space (a “Host3D” model), because stretch-dependent residual trends survive all corrections tested here."],"supporting_citations":[{"why":"The earlier ZTF analysis whose conclusion this paper re-examines; its fitting choices, split points, and gamma definition are tested directly.","marker":"G25a"},{"why":"Supplies the dust-scatter hypothesis and the alligator-mouth model that Host2D extends into a continuous surface.","marker":"BS21"},{"why":"The ZTF DR2 data release that provides the light curves and host-galaxy properties used in the analysis.","marker":"Rigault et al. (2025)"},{"why":"The SALT2mu fitting method used to compute Hubble residuals without dependence on a cosmological model.","marker":"Marriner et al. (2011)"},{"why":"The SALT2 light-curve model used to refit the ZTF supernovae.","marker":"Betoule et al. (2014b)"},{"why":"Prior DES analysis reporting an alligator-mouth profile and a smaller gamma value, providing the literature comparison point.","marker":"Vincenzi et al. (2024)"},{"why":"A previous continuous residual-surface analysis on a volume-complete DES sample, similar in spirit to Host2D.","marker":"Wiseman et al. (2022)"}],"fun_headline_variants":["ZTF reanalysis: continuous dust model beats mass-step by 4σ","Supernova color-host relation is continuous, not a simple step","ZTF data support dust-dependent scatter, not progenitor bimodality","4σ evidence for continuous dust correction in Type Ia supernovae","Volume-limited ZTF sample resolves SN color-host dust interplay"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported significances rest on fixing the intrinsic scatter at $\\sigma_{\\mathrm{int}} = 0.17$ mag in the $\\chi^2$ statistic; if the true intrinsic scatter differs or depends on color, the $4.0\\sigma$ and $3.5\\sigma$ preferences change.","fun_headline_variants_meta":{"raw":{"variants":["ZTF reanalysis: continuous dust model beats mass-step by 4σ","Supernova color-host relation is continuous, not a simple step","ZTF data support dust-dependent scatter, not progenitor bimodality","4σ evidence for continuous dust correction in Type Ia supernovae","Volume-limited ZTF sample resolves SN color-host dust interplay"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0007,"raw_usage":{"total_tokens":3301,"prompt_tokens":1225,"completion_tokens":2076,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":841,"completion_tokens_details":{"reasoning_tokens":1986}},"tokens_in":841,"tokens_out":2076,"duration_ms":13713,"temperature":1.0,"reasoning_tokens":1986,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:23:14.675751+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Refit the same 796 supernovae with $\\sigma_{\\mathrm{int}}$ treated as a free parameter or marginalized out and recompute $\\Delta\\chi^2$ between Tripp+step and Host2D; if the preference drops below about $2\\sigma$, the continuous-model claim weakens. Alternatively, run the split-point selection on simulated data generated from a dust-free step model and check whether the 9.5 and 10.3 dex splits still produce a $0.18$ mag alligator-mouth profile.","supporting_citations":[],"review_version":1}