{"id":"2d3e1f9a-eb6a-4173-b336-610a19f97b20","arxiv_id":"2607.27414","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"PhoPS is an open-source Python pipeline that automatically calibrates astrometry and photometry using on-the-fly Gaia DR3 reference indexes and a robust field-dependent zero-point model.","lead":"PhoPS is a new open-source Python pipeline that automatically calibrates astrometry and photometry of point sources using Gaia DR3 reference stars. It improves astrometric accuracy by 15% via epoch-propagated reference indexes and reveals magnitude-dependent uncertainties in its photometric error model.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Astrometric 15% gain may be inflated: the propagated/non-propagated residuals appear to be measured against different epoch reference frames, not a single common truth catalog.","rationale":"The reader's weakest_assumption focuses on the photometric radial zero-point model, which is a legitimate concern for the photometric calibration. However, the paper's headline quantitative claim is the astrometric improvement, and the most load-bearing condition for that claim is the consistency of the residual reference frame between the two compared solutions. The text is ambiguous but strongly suggests each solution was evaluated against its own reference epoch. If so, the two RMS values in Table 2 are not residuals relative to a common truth catalog, and their difference cannot be directly interpreted as an improvement in astrometric accuracy. The non-propagated solution is fitted to J2016 coordinates, so its residual reflects misfit to that catalog epoch, while the propagated solution is fitted to and evaluated against epoch-of-observation coordinates. Proper motion over the multi-year baseline shifts reference positions; the polynomial/SIP model can absorb only a smooth component of that shift, so the 15% improvement could largely represent the epoch mismatch rather than a genuine accuracy gain. This is a concrete, testable issue, and resolving it is essential before the astrometric claim can be accepted. The photometric validation concerns identified by the reader (post-hoc stability filter, small bright-star sample, SIP-order selection on the same dataset) remain valid secondary issues. Because the astrometric issue is fixable by reanalysis and the pipeline itself is open-source and documented, the conditional verdict is retained pending this check.","tokens_in":15199,"tokens_out":6273,"duration_ms":72029,"concrete_test":"Rerun the §3.2 comparison computing residuals for both the propagated and non-propagated WCS solutions against a single common reference: the epoch-propagated Gaia DR3 positions (the same file used in the propagated case). For the non-propagated WCS, propagate its J2016.0 output coordinates to the observation epoch using the same proper motions/parallax before differencing, or equivalently project the propagated reference positions through the inverse WCS into pixel space and compare with measured centroids. Recompute the 3σ-clipped N-weighted total RMS in Table 2. If the non-propagated total RMS drops from 0.284'' to near 0.241'', the claimed 15% improvement is largely an artifact of differing residual reference frames; if it remains near 0.284'' or larger, the claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (0.284'' to 0.241'' total RMS, 15.0% improvement, §3.2/Table 2) rests on a controlled comparison between epoch-propagated and J2016.0 Gaia indexes. The text says residuals were computed 'between sources detected in the observed frames and the corresponding Gaia DR3 positions under two conditions' — i.e., each solution appears to be evaluated against the same epoch reference used to build that solution. For the non-propagated case, the WCS is fit to J2016.0 positions and residuals are measured against that catalog epoch; for the propagated case, residuals are measured against epoch-of-observation positions. These are not the same reference quantity. Proper motion over the ~10-year baseline shifts reference-star positions, and the SIP polynomial can absorb only a smooth part of that shift. The non-propagated solution is therefore penalized by the epoch mismatch in its reference catalog, while the propagated solution is rewarded for agreeing with the same catalog it was calibrated to. A user's actual astrometric accuracy is the error relative to the true epoch-of-observation positions, which requires evaluating both solutions against the same propagated Gaia catalog. As written, the 15% number may conflate calibration-reference consistency with astrometric accuracy.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"PhoPS is an open-source Python pipeline for automated astrometric calibration and photometric reduction of point sources. Its two distinctive features are (i) dynamic on-the-fly generation of local Gaia DR3 astrometric index files propagated to the epoch of observation, avoiding pre-installed index collections, and (ii) a field-dependent photometric zero-point model based on RANSAC linear regression in radial distance from the detector centre. The paper validates these features using 840 TUG100 images of (201) Penelope: the propagated astrometric index is reported to reduce the clipped N-weighted total RMS residual from 0.284 arcsec to 0.241 arcsec (15.0%), and a population-level photometric uncertainty validation using 203 reference stars and 92,980 measurements yields normalized residuals with magnitude-dependent widths (sigma_z = 2.069, 1.129, 0.698 for the 10-13, 13-16, 16-18 G-mag bins). The authors conclude that PhoPS is a portable, robust tool suitable for asteroid light-curve analysis and stellar variability studies.","tokens_in":15508,"tokens_out":5726,"duration_ms":62309,"significance":"If the results hold, PhoPS provides a genuinely useful engineering contribution: dynamic, epoch-propagated index generation is a practical improvement over static Astrometry.net index collections, and the RANSAC-based radial zero-point model is a sensible lightweight alternative to per-star differential photometry. The astrometric comparison in §3.2 is a controlled A/B test that keeps all pipeline components identical except for epoch propagation, which is exactly the right experimental design for isolating that effect. The photometric validation is commendably honest about the magnitude-dependent behaviour of the reported uncertainties. However, two methodological issues in the validation are load-bearing: the astrometric residuals are computed against different reference epochs for the two solutions, and the photometric uncertainty validation applies a post-hoc stability filter that can bias the residual distributions. These issues do not necessarily invalidate the claims, but they must be addressed before the central results can be accepted as quantitative evidence.","major_comments":[{"comment":"The propagated and non-propagated residuals are measured against different reference quantities: the propagated solution is compared against epoch-of-observation Gaia positions, while the non-propagated solution is compared against native J2016.0 positions. Because proper motion separates these frames, the reported 15.0% improvement conflates calibration-reference consistency with astrometric accuracy. To support the claim of an accuracy gain, both WCS solutions should be evaluated against the same reference set (e.g., the propagated Gaia positions). Please recompute the non-propagated residuals against the propagated catalog, or otherwise perform a common-truth comparison.","section":"§3.2, Table 2"},{"comment":"The photometric uncertainty validation applies a 'stability filter' that removes stars with outlying ratios of robust scatter to mean reported uncertainty, evaluated separately within each magnitude bin. This filter is post-hoc and the threshold is not specified. Because the goal is to validate the uncertainty model, removing the most discrepant stars can bias the normalized residual widths toward unity and suppress evidence of underestimation. Report the exact filter criterion, and show the residual statistics with and without the filter, or with a pre-specified fixed threshold. This is especially important for the bright bin (N*=9), where sigma_z = 2.069 is already statistically fragile.","section":"§3.1, §3.3, Table 3"},{"comment":"The validation is restricted to RANSAC calibration inliers. Since the zero-point model is fitted to those same inliers, the validation is partly circular: the reported uncertainties reflect the scatter of stars that the model already deemed consistent. The paper should state how many stars/measurements were excluded by the inlier criterion and the stability filter, and discuss how the residual distributions would change if a held-out set of Gaia-matched stars (not used in calibration) were used. This would strengthen the claim that the uncertainty model is representative.","section":"§3.3, §4"}],"minor_comments":[{"comment":"The definition of 'N-weighted total RMS' is not given. Please define the weighting scheme (presumably number of measurements per bin) or state that the overall value is simply the RMS over all matched measurements after clipping.","section":"§3.2, Table 2"},{"comment":"The SIP polynomial order (2-6) was tested on the same dataset and order 5 was adopted because it minimized scatter. This introduces a selection on the validation data and may slightly inflate the apparent astrometric quality. Please state this clearly as a tunable parameter or provide a cross-validation or independent confirmation.","section":"§3.2"},{"comment":"The stability filter description would benefit from a precise definition of 'outlying ratios' (e.g., median absolute deviation cut or percentile). Currently it is not reproducible from the text.","section":"§3.1"},{"comment":"The abstract states 'Epoch propagation reduced the clipped N-weighted total RMS residual from 0.284 arcsec to 0.241 arcsec, a 15.0% improvement.' Given the reference-frame issue, the wording should be qualified as 'relative to the respective reference catalogue epochs' until a common-truth comparison is provided.","section":"Abstract"},{"comment":"The paper correctly acknowledges that the linear radial zero-point model may not capture asymmetric or higher-order structure. It would be useful to quantify the magnitude of the residual spatial pattern (e.g., the standard deviation of per-star residuals as a function of position) to indicate how much of the observed sigma_z could be due to model inadequacy.","section":"§4"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid software description with a plausible and useful contribution. The two main validation claims are, however, weakened by methodological issues: the astrometric comparison measures residuals against different reference epochs, and the photometric uncertainty validation uses a non-specified post-hoc filter. Both are fixable within the manuscript's scope, but the authors should be asked to provide a common-truth astrometric comparison and to report the photometric statistics with and without the stability filter. Given the paper's otherwise careful and honest presentation, major revision is appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a useful open-source pipeline, but the headline astrometric result—the 15% improvement—is not actually demonstrated. The two conditions are evaluated against different reference frames, so the comparison conflates calibration-reference consistency with real astrometric accuracy.\n\nThe genuinely new thing is the dynamic on-the-fly generation of Gaia DR3 index files propagated to the observation epoch. That is a meaningful convenience for small/medium observatories, and the RANSAC-based radial zero-point model is a sensible robust approach for field-dependent photometry. The paper is honest about the photometric uncertainty behavior, and the light-curve examples are fine.\n\nThe problem is in §3.2. For the propagated case, the WCS is fit to epoch-of-observation Gaia positions and the residuals are measured against those same positions. For the non-propagated case, the WCS is fit to J2016.0 positions and residuals are measured against J2016.0. Proper motion over the ~10-year baseline moves the reference stars. So the non-propagated solution carries an extra epoch-mismatch term that the propagated solution does not. The 15% gain may be mostly that mismatch, not an improvement in the accuracy a user would get on sky. The right test is to compute residuals for both solutions against the same propagated Gaia catalog. As written, the claim is unsupported.\n\nOther soft spots, in decreasing severity. The SIP order was chosen on the same dataset used for validation—minor but worth noting. The photometric uncertainty validation uses a stability filter that can bias the residual distribution, and the bright-star bin has only 9 stars; the authors acknowledge the small sample. The radial linear zero-point model is a real limitation for wide fields, but they flag it in the Discussion. The quantitative validation is from one telescope, so the instrument-independence claim rests on prior applications, not on controlled tests here.\n\nIf the authors redo the astrometric comparison against a common reference and the improvement survives, this is a solid, citable software paper. As it stands, the central quantitative claim should not be taken at face value. I'd send it to review, but with a clear request for a fair residual comparison.","headline":"Useful open-source pipeline, but the headline astrometric gain isn't proven: the two conditions are measured against different epoch references, so the 15% number conflates reference-frame consistency with real accuracy.","tokens_in":15962,"tokens_out":3602,"would_cite":false,"duration_ms":37641,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PhoPS builds epoch-propagated Gaia reference indexes on the fly and a radial detector zero-point model, claiming 15% better astrometry and stable light curves.","keywords":["photometric calibration","astrometric calibration","Gaia DR3","epoch propagation","RANSAC zero-point model","point-source photometry","asteroid light curves","time-series photometry"],"falsifier":"Take a wide-field image with strong asymmetric vignetting and a dense set of Gaia reference stars; fit the PhoPS radial linear model and a full two-dimensional zero-point map, and compare the spatial structure of the residuals. If the radial-model residuals show coherent two-dimensional patterns beyond the bootstrap uncertainty, the assumption fails. Similarly, collect more than nine bright reference stars (10≤G<13) and check whether sigma_z remains near 2; if it does, the bright-star uncertainty model is genuinely mis-scaled.","tokens_in":15094,"feed_emoji":"🔭","tokens_out":5885,"duration_ms":59724,"temperature":0.7,"pith_summary":"PhoPS is an automated, instrument-independent pipeline that calibrates both astrometry and photometry of point sources using the same Gaia DR3 reference catalogue. The paper's central claim is that building local astrometric indexes from Gaia positions propagated to the epoch of observation improves astrometric accuracy over using native catalogue-epoch positions; on 141,672 matched measurements from 840 images, the total clipped RMS residual drops from 0.284 to 0.241 arcsec, a 15.0% gain. For photometry, the paper claims a field-dependent zero-point model, a linear function of radial distance fitted by RANSAC, correctly absorbs large-scale spatial systematics such as vignetting, yielding stable light curves for both stars and asteroids. The validation also shows the uncertainty model is not globally consistent: normalised residuals are near zero on average but their width varies with brightness, with the brightest bin over-dispersed and the faintest bin under-dispersed.","feed_headline":"Epoch-shifted star references cut astrometric error by 15%","feed_subtitle":"A new pipeline propagates Gaia star positions to the observation date and maps detector-wide photometric drift.","key_machinery":"Two mechanisms carry the argument. First, dynamic Gaia DR3 index generation: the pipeline queries the Gaia archive for each field, propagates proper motions and parallaxes to the observation epoch, and creates temporary local astrometric indexes, so the reference frame matches the moment of observation rather than the catalogue epoch. Second, the field-dependent zero-point model f(r)=α+βr, a radial linear model fitted with RANSAC over matched reference stars; the resulting zero point is evaluated at each target's radial position, and bootstrap resampling of the RANSAC inliers quantifies the zero-point uncertainty that is added in quadrature to the formal photometric error.","core_discovery":"The authors introduce PhoPS, an open-source Python pipeline for fully automated photometric reduction with integrated astrometric calibration of point sources. Instead of relying on a pre-installed collection of static astrometric indexes, PhoPS queries Gaia DR3 for each field, propagates source positions from the catalogue epoch J2016.0 to the observation epoch using proper motions and parallaxes, and builds local astrometric indexes on the fly. This epoch-consistent reference, they argue, is what makes the astrometric improvement possible; the propagated solution beats the non-propagated one in every magnitude bin, with a clipped N-weighted total RMS residual of 0.241 arcsec versus 0.284 a","pith_inferences":["The 15% astrometric gain was measured on data with a limited time baseline from the Gaia epoch; a dataset collected several years further from J2016.0, or in a high-proper-motion field, should show a larger relative gain, making the dynamic-index strategy increasingly valuable for long-term surveys.","If the radial linear zero-point assumption is the bottleneck, replacing it with a two-dimensional polynomial or a pixel-grid model should reduce spatially structured residuals on wide-field images; this is directly testable with the same reference-star sample.","The magnitude-dependent sigma_z pattern suggests an extension in which the reported uncertainty includes a brightness-dependent term fitted from the validation residuals; a dedicated bright-star sample with more than nine stars would decide whether the bright-end under-estimate is real.","Because the pipeline leaves residual elongation in the astrometric error distribution unmodelled, combining the epoch-propagated indexes with an instrument-specific distortion correction could close part of the remaining 0.241 arcsec floor."],"forward_implications":["Adopting epoch-propagated Gaia references lowers astrometric RMS in all brightness bins, with the largest relative gain (22.2%) at intermediate magnitudes.","The pipeline can produce asteroid light curves and stellar time series from the same frames without manual reference-star selection; a reference field star is recovered at its catalogued magnitude to within one millimagnitude.","Because zero-point variations across frames do not appear as coherent features in calibrated light curves, the radial RANSAC model is sufficient to absorb short-term systematics like vignetting and tracking degradation.","Reported photometric uncertainties are not uniformly scaled: users should treat bright-star errors as underestimated and faint-star errors as conservative, and the paper argues against interpreting the full-sample variance as a single global error model.","No permanent astrometric index collection is required, and indexes are reused for repeated observations of the same field, supporting survey-scale time-series reductions."],"fun_headline_variants":["PhoPS pipeline cuts star-position error 15% via epoch-corrected Gaia","Epoch-propagated Gaia refs slash astrometric RMS by 15%","Automated PhoPS uses Gaia epoch shifts to trim astrometry error 15%","Open-source PhoPS improves photometry and astrometry via epoch-propagation","Epoch-aware Gaia indexes in PhoPS slash astrometric error by 15%"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the photometric zero point varies linearly with radial distance from the detector centre; if a real detector shows asymmetric or higher-order spatial structure, calibrated magnitudes inherit spatial systematics that the quoted uncertainties do not capture.","fun_headline_variants_meta":{"raw":{"variants":["PhoPS pipeline cuts star-position error 15% via epoch-corrected Gaia","Epoch-propagated Gaia refs slash astrometric RMS by 15%","Automated PhoPS uses Gaia epoch shifts to trim astrometry error 15%","Open-source PhoPS improves photometry and astrometry via epoch-propagation","Epoch-aware Gaia indexes in PhoPS slash astrometric error by 15%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001101,"raw_usage":{"total_tokens":4481,"prompt_tokens":848,"completion_tokens":3633,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":592,"completion_tokens_details":{"reasoning_tokens":3527}},"tokens_in":592,"tokens_out":3633,"duration_ms":23297,"temperature":1.0,"reasoning_tokens":3527,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T07:43:58.934803+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a wide-field image with strong asymmetric vignetting and a dense set of Gaia reference stars; fit the PhoPS radial linear model and a full two-dimensional zero-point map, and compare the spatial structure of the residuals. If the radial-model residuals show coherent two-dimensional patterns beyond the bootstrap uncertainty, the assumption fails. Similarly, collect more than nine bright reference stars (10≤G<13) and check whether sigma_z remains near 2; if it does, the bright-star uncertainty model is genuinely mis-scaled.","supporting_citations":[],"review_version":1}