Pith. sign in

REVIEW 3 major objections 5 minor 13 references

Multi-night photometric repeatability of STDWeb: an empirical error budget from a 14-night campaign on a single field

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A drifting colour term, not bad errors, explains STDWeb's multi-night scatter.

desk verdict A genuinely useful, carefully quantified error budget for STDWeb whose central color-term diagnosis holds up, modulo the 'entire excess' overclaim and an unresolved in-sample fitting question. read the letter →

arxiv 2608.10017 v1 pith:3X5EHWPO submitted 2026-08-08 astro-ph.IM

Pierre-Yves Lechapelain This is my paper · ORCID
classification astro-ph.IM
keywords photometricrepeatabilitycolourtermtime-seriesphotometryerrorbudgetSTDWebdifferentialpro-amastronomyreference-catalogue
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper measures how well STDWeb's reported photometry repeats on the same field night after night, using 157 exposures taken on 14 nights and the light curves of 101 constant field stars. It finds that the pipeline's formal errors are accurate within a single night, but that naive use of its calibrated magnitude across nights gives a true single-epoch scatter roughly 1.5 to 1.7 times the formal error. The paper shows that this entire excess is a time-varying colour term: transforming magnitudes into the catalogue system with $m_{\rm sys} = {\rm mag\_calib} + c_t ({\rm BP}-{\rm RP})$ collapses the night-level offset to 3.4 mmag and restores $\chi_{\rm camp}\simeq1$. The correction improves absolute per-star repeatability from 43 to 9.5 mmag, and it works without an external colour catalogue because colours fitted from the light curves themselves give the same budget.

What carries the argument

The load-bearing object is the catalogue-system magnitude $m_{\rm sys}={\rm mag\_calib}+c_t({\rm BP}-{\rm RP})$, where $c_t$ is the per-frame colour-term coefficient that STDWeb already fits as part of its photometric solution and ${\rm BP}-{\rm RP}$ is the star's colour. The paper's argument is that ${\rm mag\_calib}$ is expressed in a time-varying instrumental pseudo-band, so making the transformation to the catalogue system explicit removes the otherwise unexplained night-level offsets. The supporting machinery is a closed variance decomposition $\sigma_{\rm camp}^2\simeq\sigma_{\rm within}^2+\sigma_{\rm night}^2$, which lets the paper isolate what the colour term contributes, plus a data-driven colour fit that allows the correction to run when no external colour catalogue is available.

What would settle it

Redo the multi-night budget with the per-epoch $c_t$ values randomly permuted across epochs while keeping each star's colour fixed: the explanation predicts that the night term and the $\chi_{\rm camp}$ excess must return, because the colour-term correction has been destroyed. If the improved $\chi_{\rm camp}$ persists under permutation, the colour term is not the actual cause and the collapse seen with the true $c_t$ is a fitting artifact.

Watch

Extended reading notes

Core claim

The paper's central claim is that the excess scatter of STDWeb magnitudes across nights—the factor by which the true single-epoch uncertainty exceeds the reported formal error—is the epoch-dependent colour term. STDWeb's calibrated magnitude ${\rm mag\_calib}$ lives in an instrumental pseudo-band whose relation to the catalogue system changes from epoch to epoch, as the fitted colour-term coefficient $c_t$ wanders between $-0.14$ and $-0.28$. Defining $m_{\rm sys}={\rm mag\_calib}+c_t({\rm BP}-{\rm RP})$ collapses the night-level offset from 8.6 to 3.4 mmag for the bright half of the ensemble, restores $\chi_{\rm camp}=1.0$–$1.1$, and improves the absolute per-star repeatability anchored to the reference catalogue from 43 to 9.5 mmag, while correlations between residuals and airmass or colour vanish. Repeating the budget with stellar colours fitted from the light curves as the slope of ${\rm mag\_calib}$ against $c_t$ gives an indistinguishable result, so the correction needs no external colour catalogue.

Load-bearing premise

The result stands or falls on the assumption that STDWeb's per-frame colour-term coefficient is a faithful description of the real, time-varying transformation between the instrumental band and the catalogue system, so that a linear colour correction removes the true chromatic systematics and does not merely re-fit the scatter.

Editorial extensions

If this is right

  • Within a single night, STDWeb's formal errors can be used as-is: the measured within-night scatter matches the reported formal error ($\chi_{\rm within}\simeq1$).
  • Across nights, users should analyse $m_{\rm sys}={\rm mag\_calib}+c_t({\rm BP}-{\rm RP})$ rather than the raw pipeline magnitude; doing so restores $\chi_{\rm camp}=1.0$–$1.1$ and makes formal errors usable across nights.
  • Targets with no colour information and too few epochs to fit one should have their formal errors inflated by a factor of roughly 1.5–2, or an 8 mmag night term added in quadrature.
  • A low-cost colour-CMOS system can deliver absolute photometry that repeats at 9.5 mmag per star across 14 nights, with a nightly zero point stable to 2.3 mmag, when the colour-term correction is applied.
  • Campaign-level meta-analyses of STDWeb products should consume ${\rm mag\_calib}$ and $c_t$ directly; re-deriving zero points outside the pipeline is an avoidable failure mode that produced spurious excursions of up to 0.7 mag in this campaign.
  • The error budget is self-contained: stellar colours fitted from the light curves reproduce catalogue ${\rm BP}-{\rm RP}$ colours to 0.027 mag and produce an identical budget, so the correction works without an external colour catalogue.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same qualitative mechanism—a time-varying colour term inflating multi-night scatter—should appear in any pipeline that calibrates against a broadband survey catalogue with a fitted colour term, so the pattern is likely to generalise even if the exact numbers are station-specific.
  • Editorial inference: because the colour correction is self-contained and matches catalogue colours so closely, the method could be used to inter-calibrate heterogeneous cameras within a network, effectively building a common colour system from photometry alone.
  • Editorial inference: a direct testable extension would be to repeat this campaign with a filter that more closely matches the catalogue passband; the paper's mechanism predicts the night term should shrink roughly in proportion to the reduction in colour-term variance.
  • Editorial inference: the cautionary result on re-deriving zero points suggests that archived light curves from other pipelines may contain similar time-varying pseudo-band terms, and re-analysing them with a colour-term transformation could reduce systematic scatter in existing transient and variability surveys.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper presents an empirical multi-night photometric error budget for the STDWeb pipeline, based on 157 G-band exposures of a single field over 14 nights, analysed through 101 constant field stars. The authors find that within-night single-epoch scatter matches the reported formal errors (chi_within ~ 1), while the naive campaign-level scatter is 11-12 mmag with chi_camp ~ 1.5-1.7. They attribute the entire excess to an epoch-dependent colour term: computing m_sys = mag_calib + c_t (BP-RP) with the pipeline's per-frame colour-term coefficient reduces the night term to 3.4 mmag, restores chi_camp ~ 1.0-1.1, and improves Gaia-anchored per-star repeatability from 43 to 9.5 mmag. Colours fitted from the light curves themselves reproduce Gaia BP-RP to 0.027 mag and yield the same budget. The paper includes a closed variance decomposition, practical recommendations, and a cautionary note on re-deriving zero points outside the pipeline.

Significance. The paper addresses a real need for pro-am networks: a validated, closed error budget for a widely used web pipeline. If the central claim holds, the practical message is strong and directly actionable: formal errors are trustworthy within a night, and applying the provided colour-term correction makes them trustworthy across nights. The variance budget closes internally, and the data-fitted colours are validated against Gaia (r=0.98, 0.027 mag). The paper is honest about its caveats (single field, single instrument, unbalanced exposure mix) and includes a cautionary tale about re-deriving zero points. However, the central 'entire excess' claim rests on a correction that is fitted to the same ensemble on which it is evaluated, so the significance of the physical interpretation is conditional on an out-of-sample demonstration.

major comments (3)
  1. [Sect. 2.3, Eq. (1), Sect. 4.3] The decisive test is in-sample: STDWeb's per-frame colour-term coefficient c_t is fitted on the same 101 ensemble stars and the same frames whose residuals are then used to compute the night term and chi_camp. Because the per-frame photometric solution includes a colour term, the least-squares fit forces the ensemble residuals in m_sys to have zero mean and zero linear colour dependence within each frame. The observed collapse of sigma_night from 8.6 to 3.4 mmag and chi_camp from 1.69 to 1.03 is therefore partly a property of the fitting procedure. I request an out-of-sample test: split the 101 stars into two disjoint sets, fit c_t on one set and compute the budget on the other, or repeat the analysis with an independent reduction pipeline. Without such a test, the claim that the entire excess is the epoch-dependent colour term is not fully established.
  2. [Sect. 3.3, Table 2] The data-driven colour fitting is partially circular. Fitting each star's colour as the slope of mag_calib versus c_t removes the linear c_t dependence from the residuals by construction, so the identical budget in the last column of Table 2 is not an independent confirmation of the correction. The Gaia-colour column is the meaningful external validation; the fitted-colour column demonstrates only that the method is self-contained. The text should clearly state that the Gaia-colour test is the one that validates the physical origin, and that the fitted-colour test is a practical alternative, not an independent check.
  3. [Sect. 2.3, Sect. 5.2, Table 2] chi_camp is computed as the ratio of the empirical scatter on m_sys to sigma_form, the formal error reported on mag_calib. After applying Eq. (1), the uncertainty in m_sys should include contributions from the uncertainty in c_t and in (BP-RP). The paper does not propagate these terms or argue that they are negligible. Because the central claim is that formal errors are valid across nights on m_sys, the authors should either propagate these uncertainties into the formal error or justify that they are small relative to the 8.1 mmag median formal error.
minor comments (5)
  1. [Sect. 3.2, Eq. (2)] The symbol c_k is used for the per-epoch ensemble offset while c_t denotes the colour-term coefficient; this is confusing. Suggest renaming the epoch offset to z_k or d_k.
  2. [Sect. 2.2] The field coordinates or a reference to a public chart would aid reproducibility.
  3. [Fig. 1] Please specify the bin width or smoothing scale of the running median of the formal error.
  4. [Table 1] The caption should state that the 16-84% range is the central-quantile range over stars and clarify how it is computed.
  5. [Sect. 5.1] The residual night term of ~3 mmag is attributed to 'second-order chromatic effects' among other causes; a simple quadratic colour-term test in the residuals would help quantify this.

Circularity Check

1 steps flagged · score 3.0 of 10

The Gaia-colour version of the correction is externally grounded, but the self-contained version fits each star's colour as the slope of mag_calib against c_t, which removes the linear c_t dependence by construction.

  1. self definitional [Sect. 3.3, Eq. (1), and Table 2 (last column)]
    "the colour of a constant star is simply (minus) the slope of its mag_calib light curve against the per-epoch coefficient c_t [3]. We implement this as an iteratively 3σ-clipped linear regression of m_i,k on c_t,k per star. ... Repeating the entire budget with colours fitted from the data (Sect. 3.3) instead of Gaia colours gives an indistinguishable result (last column of Table 2)."

    When stellar colour is fitted as the negative slope of mag_calib versus c_t, forming m_sys = mag_calib + c_t*(fitted colour) removes the best-fit linear dependence of mag_calib on c_t from each star's residuals by construction. The collapse of sigma_night from 8.6 to 3.2 mmag and chi_camp from 1.69 to 1.03 in the 'fitted colours' column is therefore guaranteed for the linear component, regardless of whether c_t represents a physical atmospheric/instrumental colour term or an in-sample nuisance absorbing flat-field, illumination, or PSF systematics. The independent Gaia cross-check (r=0.98, 0.027 mag scatter) provides real external support that the fitted colours are physical, but the budget improvement itself does not discriminate between these interpretations.

full rationale

The paper's central diagnostic with Gaia BP-RP colours is not circular: m_sys = mag_calib + c_t (BP-RP) uses an externally supplied colour per star and a per-frame coefficient c_t emitted by the pipeline, and the measured improvements (sigma_night 8.6 -> 3.4 mmag, chi_camp 1.69 -> 1.03, vanishing airmass and colour correlations) are empirical consequences of applying that catalogue-system transformation. The within-night validation (chi_within ~ 1, 120 s vs 30 s scatter ratio 1.9) is also self-contained and independent of the colour-term hypothesis. The main partial circularity is confined to the self-contained variant of Sect. 3.3: fitting each star's colour as the slope of mag_calib against c_t makes the linear c_t component of the scatter vanish by construction. This does not invalidate the paper, because the fitted colours reproduce Gaia BP-RP to 0.027 mag, providing independent grounding, and the Gaia-based analysis stands on its own. The absence of a true out-of-sample test of c_t (stars or frames not used in its fit) is a validity caveat rather than a demonstrated circularity, and the paper's caveats in Sect. 5.5 partially acknowledge the single-field, single-instrument scope. Overall, the central claim has substantial independent content; only the no-catalogue variant contains a by-construction component, so the circularity score is moderate rather than severe.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The analysis introduces no free parameters or invented entities. It relies on the pipeline's per-frame calibration outputs, Gaia reference photometry, and the assumptions that the field stars are constant and that the linear colour term is an adequate model of the instrumental-to-catalogue transformation.

assumptions (5)
  • domain assumption The 101 ensemble stars are constant at the mmag level over the campaign.
    Sect. 3.1: ensemble members are assumed constant; low-level intrinsic variability would inflate the upper percentiles of sigma_camp but not its median, so the central estimates are robust to this.
  • domain assumption The linear transformation m_sys = mag_calib + c_t (BP-RP) fully captures the time-dependent instrumental-to-catalogue transformation at each epoch.
    Eq. (1), Sect. 2.3 and 4.3: the entire correction rests on this linear colour-term model, with the residual 3.4 mmag night term attributed to unmodeled second-order effects.
  • domain assumption Gaia DR3 G and BP-RP photometry are accurate external references, and proper-motion propagation is valid.
    Sect. 2.2 and 3.1: matching and anchoring use Gaia; absolute repeatability is measured against Gaia G.
  • domain assumption STDWeb's per-frame c_t values, formal errors, and zero-point model are reliable outputs of the pipeline.
    Sect. 2.3: the analysis consumes mag_calib and c_t as primary observables; if the pipeline's c_t is biased, the colour-term attribution weakens.
  • standard math The variance decomposition sigma_camp^2 approximately equals sigma_within^2 plus sigma_night^2 (Eq. 4) assumes independence between within-night and night-to-night noise.
    Sect. 5.1: used to close the budget; the agreement with the direct measurement supports the approximation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi-night photometric repeatability of STDWeb: an empirical error budget from a 14-night campaign on a single field." pith.science (2026). https://pith.science/paper/3X5EHWPO

@misc{pith2026260810017,
  author       = {Pith},
  title        = {Pith review of: Multi-night photometric repeatability of STDWeb: an empirical error budget from a 14-night campaign on a single field},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3X5EHWPO}},
  note         = {Machine review of arXiv:2608.10017}
}
abstract

Web-based photometric pipelines such as STDWeb are routinely used by small observatories and professional-amateur networks across the full range of photometric science: variable stars, supernovae, exoplanet transits, transient follow-up. The formal magnitude uncertainty they report describes a single reduction of a single frame; campaign science instead requires the scatter of the same star, same field, night after night. We measure this empirically on the field of the Einstein Probe trigger EP-WXT 01709274151, observed on 14 nights in 2026 July: 157 Gaia-like G exposures reduced homogeneously with STDWeb, analysed through 101 constant field stars (10.5<G<14.5). The median formal error is 8.1 mmag (6.7 mmag for the bright half), and within a single night the empirical differential scatter matches it ($\chi_{\rm within}\simeq1$). Used naively - taking the pipeline's mag_calib, which lives in the instrumental pseudo-band - each star acquires a night-level offset of 7-9 mmag, the campaign scatter grows to 11-12 mmag, and the formal error is roughly half of the true one ($\chi_{\rm camp}=1.5$-$1.7$). We show that this entire excess is the epoch-dependent colour term: analysing the catalogue-system magnitude $m_{\rm sys}$ = mag_calib + colour term x (BP-RP) collapses the night term to 3.4 mmag, restores $\chi_{\rm camp}=1.0$-$1.1$, and improves the absolute Gaia-anchored per-star repeatability from 43 to 9.5 mmag, with the nightly zero-point peak-to-peak shrinking from 130 to 11 mmag and the airmass and colour correlations of the residuals vanishing. Stellar colours fitted from the light curves themselves reproduce Gaia BP-RP to 0.027 mag (r=0.98) and yield an identical budget: the correction needs no external colour catalogue. We provide the closed variance decomposition, practical recommendations, and a cautionary note on re-deriving zero points outside the pipeline.

Figures

Figures reproduced from arXiv: 2608.10017 by the authors.

Figure 1
Figure 1. Per-star campaign-level single-epoch scat [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Differential residuals δi,k of the 101 ensemble stars (on mag calib) with the 14 nights concatenated (inter-night gaps suppressed; vertical lines mark night boundaries). Black points: per-epoch ensemble median. The band stays flat at the ∼10 mmag level; the small star-level night offsets visible here constitute σnight and are removed by the colour term ( [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Left: fitted colour-term coefficient ct per epoch over the campaign; the nightly medians move between −0.14 and −0.28, so the instrumental-to-catalogue transformation is time-dependent. Right: stellar colours fitted from the light curves as the slope of mag calib versus ct, against Gaia BP −RP; the identity line is dashed. noise no longer hides it) from night to night, even though the ensemble correction removes the… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 12 canonical work pages

  1. [3]

    2026, private communication

    Karpov, S. 2026, private communication

    work page 2026

  2. [1]

    2025,STDweb: simple transient de- tection pipeline for the web, Acta Polytechnica, 65(1), 50–64 (arXiv:2411.16470)

    Karpov, S. 2025,STDweb: simple transient de- tection pipeline for the web, Acta Polytechnica, 65(1), 50–64 (arXiv:2411.16470)

    arXiv 2025

  3. [2]

    2021,STDPipe: Simple Transient Detection Pipeline, Astrophysics Source Code Li- brary, ascl:2112.006

    Karpov, S. 2021,STDPipe: Simple Transient Detection Pipeline, Astrophysics Source Code Li- brary, ascl:2112.006

    work page 2021

  4. [4]

    Gaia Collaboration 2023,Gaia Data Release 3: Summary of the content and survey properties, A&A, 674, A1

    work page 2023

  5. [5]

    Lechapelain, P.-Y. et al. 2026, GCN Circu- lar 45133: optical counterpart of EP-WXT trigger 01709274151 identified as the M dwarf PM J15587+2351E

    work page 2026

  6. [6]

    2006,The effect of red noise on planetary transit detection, MNRAS, 373, 231

    Pont, F., Zucker, S., & Queloz, D. 2006,The effect of red noise on planetary transit detection, MNRAS, 373, 231

    work page 2006

  7. [7]

    2005,Cor- recting systematic effects in a large set of photo- metric light curves, MNRAS, 356, 1466

    Tamuz, O., Mazeh, T., & Zucker, S. 2005,Cor- recting systematic effects in a large set of photo- metric light curves, MNRAS, 356, 1466

    work page 2005

  8. [8]

    Kov´ acs, G., Bakos, G., & Noyes, R. W. 2005,A trend filtering algorithm for wide-field variability surveys, MNRAS, 356, 557

    work page 2005

Show all 13 references
  1. [9]

    2022, MUPHOTEN: a MUlti-band PHOtometry Tool for TElescope Network, arXiv:2201.07565

    Duverne, P.-A., Antier, S., Basa, S., et al. 2022, MUPHOTEN: a MUlti-band PHOtometry Tool for TElescope Network, arXiv:2201.07565

    2022 arXiv

  2. [10]

    2022,GRANDMA observations of ZTF/Fink transients during summer 2021, MNRAS, 515, 6007

    Aivazyan, V., Almualla, M., Antier, S., et al. 2022,GRANDMA observations of ZTF/Fink transients during summer 2021, MNRAS, 515, 6007

    2022

  3. [11]

    2026,PhoPS: an au- tomated photometric pipeline for survey-era as- tronomy, arXiv:2607.27414

    Erece, O., & Kilic, Y. 2026,PhoPS: an au- tomated photometric pipeline for survey-era as- tronomy, arXiv:2607.27414

    2026 arXiv

  4. [12]

    A., et al

    Irving, Z. A., et al. 2026,phoptic — a Python package for reducing astronomical im- ages, arXiv:2607.14014

    2026 arXiv

  5. [13]

    1996,SExtractor: Soft- ware for source extraction, A&AS, 117, 393

    Bertin, E., & Arnouts, S. 1996,SExtractor: Soft- ware for source extraction, A&AS, 117, 393. 6

    1996

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.