{"id":"b80c664e-2e4b-4154-b549-d6af7ed97a81","arxiv_id":"2607.05658","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Adding GNSS-derived Zenith Wet Delay as an input to a fine-tuned Aurora weather model improves 6-hour precipitation skill, with gains concentrated in extreme events (ETS99 +8.8%).","lead":"This paper tests whether adding a satellite-navigation measurement of atmospheric moisture—the Zenith Wet Delay—to a large weather AI model improves 6-hour rain forecasts. On a held-out test period, rain forecasts improve most for extreme events, with the Equitable Threat Score rising about 9% at the 99th percentile.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Step 2's matched baseline is not input-matched: Model A adds ZWD as both input and auxiliary output/loss, so the reported gain may stem from multi-task regularization rather than the GNSS input signal.","rationale":"The reader's weakest_assumption identifies the ERA5-dependence of ZWDX and the lack of an ERA5-only control. That is a real attribution concern, but the input/output confound is more basic and more load-bearing: even if ZWDX were a pure, independent GNSS product, the current experimental design still could not attribute the precipitation gain to the ZWD input channel. Model A and Model B differ simultaneously in (i) the ZWD input and (ii) the ZWD auxiliary output/loss; the paper's phrase 'differ only in whether ZWD is included as an additional input and auxiliary target' (Methods, Figure 7) concedes the conflation. The baseline is therefore not matched in the sense required to isolate the input's contribution. The reported effects are plausibly explained by multi-task learning: predicting a physically related moisture variable at λ_ZWD=2 could regularize the shared encoder and improve precipitation skill independent of any new information at inference. The reader's proposed ERA5-only control would not resolve this confound, because a column-moisture input derived from ERA5 would still leave the auxiliary-output difference intact. The pseudo-replication concern (ten checkpoints from one run) is secondary; the effect is consistent across checkpoints and reproduced at two scales, but the input/output confound affects the interpretation of all those results. The paper is otherwise careful and transparent, and the fix is straightforward: a three-way ablation separating ZWD input from ZWD output/loss. I therefore keep the reader's CONDITIONAL verdict, but with an additional, more fundamental condition than the one stated.","tokens_in":23419,"tokens_out":7712,"duration_ms":84817,"concrete_test":"Train a third model B' identical to Model A (same pretrained init, data, step budget, optimizer, λ_ZWD=2, same ZWD output channel and loss) but with the ZWD input channel removed (masked to zero). Evaluate B' on the held-out test set. If B' reproduces A's ETS gains at the 75th–99th percentiles relative to the original baseline, the gain is due to the auxiliary target/loss rather than the ZWD input; if B' falls back to baseline, the input path is confirmed. A complementary run with ZWD input but no output/loss would further isolate the direction.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim requires that including ZWD as an input improves precipitation forecasts. But the Step 2 comparison (Methods, 'Experimental design'; Eq. 1) varies two things at once: Model A receives ZWD as an additional input and is trained with ZWD as an additional output/loss (λ_ZWD=2), while Model B has neither. The paper states the contrast 'isolates the contribution of GNSS-derived ZWD,' but it does not: any benefit from the extra prediction target—forcing the shared encoder to build moisture representations—is attributed to the input. The loss-weight sweep (Supp. Note 2) varies λ_ZWD but always keeps the ZWD output present in A and absent in B, so it cannot separate the two paths. Without a control that adds ZWD as an output without an input (or as input without output), the headline 8.8% ETS gain is consistent with an auxiliary-task effect, not with GNSS information entering through the surface encoder. This is the most load-bearing concern because it undermines the causal attribution even before considering whether ZWDX carries independent GNSS information.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports fine-tuning the Aurora weather foundation model with an additional surface variable, GNSS-derived Zenith Wet Delay (ZWD) from the ZWDX gridded product, and evaluates the effect on 6-hour accumulated precipitation forecasting. Precipitation is a new downstream task not seen in pretraining. The authors compare two models, A (with ZWD as input and auxiliary output/loss) and B (without ZWD), sharing initialization, optimizer, data, and step budget. They report that ZWD is learned at skill comparable to Aurora's pretrained variables; that including ZWD improves precipitation metrics, with gains growing with event severity (headline 8.8% ETS increase at the 99th percentile); that the precipitation power spectrum becomes closer to the target at all resolved scales; and that the benefit is larger for a smaller 110M-parameter variant. Results are evaluated on a held-out test set (April–December 2020) and are averaged over ten late-training checkpoints per model, with paired significance tests.","tokens_in":23726,"tokens_out":3953,"duration_ms":43180,"significance":"If the reported effect is genuinely attributable to the GNSS-derived information, the paper would be a valuable demonstration that an independent observational stream can improve a foundation-model weather forecast on a high-impact, tail-weighted target. The study has concrete strengths: public code and data links, a held-out test period, evaluation at two model scales, a checkpoint-ensemble analysis, and a generally careful discussion of limitations, including the ERA5 dependence of ZWDX and the non-conservative regridding of the precipitation target. However, the causal attribution of the precipitation gain to the GNSS input is currently underdetermined: the main comparison varies both the input and an auxiliary output/loss, and no control is provided against an ERA5-only column-moisture input. These are not mere presentation issues; they are load-bearing for the abstract's central claim.","major_comments":[{"comment":"Model A differs from Model B in two ways: A receives ZWD as an additional input and also predicts ZWD as an auxiliary output with loss weight lambda_ZWD=2, while B has neither. The text states that this contrast 'isolates the contribution of GNSS-derived ZWD,' but it cannot separate the input effect from the multi-task/auxiliary-target effect. A model that is asked to predict ZWD may develop better intermediate moisture representations even if ZWD is never used as an input. The loss-weight sweep in Supplementary Note 2 always keeps the ZWD output in A and absent in B, so it does not address this confound. A control with ZWD as output only (no input) or as input only (no auxiliary loss) is needed to support the causal claim.","section":"Methods, 'Experimental design'; Eq. (1)"},{"comment":"The manuscript acknowledges that ZWDX is an XGBoost function of ERA5 specific humidity, location, and time, trained on GNSS station labels, and that Aurora is pretrained on ERA5. Yet no control is run with an ERA5-only column-moisture field (e.g., ERA5 integrated water vapour or column-integrated specific humidity) in place of ZWDX. Without such a control, the reported precipitation gains could be obtained from any ERA5-derived moisture-column input, and the 'GNSS' attribution is not established. The Discussion's statement that the gains are a 'conservative lower bound' assumes, rather than demonstrates, that the GNSS labels contribute information beyond ERA5 humidity. An ablation replacing the ZWDX input with a purely ERA5-derived moisture column would directly address this.","section":"Methods, 'ZWDX global gridded ZWD product'; Discussion"},{"comment":"The ten checkpoints are sampled from a single training run per model at matched training steps. They are not independent observations; consecutive checkpoints along the same trajectory are autocorrelated. The paired t-test treats the ten paired differences as independent, which can underestimate the standard error and inflate the t-statistic when the differences are positively autocorrelated. The assertion that 'a paired t-test moreover remains valid under any pairing' refers to the A_i-versus-B_i pairing, not to independence across i. The reported p-values (e.g., Table 3, p=0.004 for ETS99) are therefore likely anti-conservative. A block-bootstrap or mixed-effects analysis, or at least an estimate of the effective sample size, should be reported.","section":"Methods, 'Checkpoint ensemble and paired significance testing'"},{"comment":"The three-way 110M ablation (surface-only, precipitation-only without ZWD, precipitation-plus-ZWD) is useful, but it still does not separate the input and auxiliary-output channels: the 'With ZWD' configuration adds both the ZWD input and the ZWD output/loss relative to the 'Without ZWD' configuration. The same control described in the first major comment is needed at this scale as well before the inverse-capacity-scaling claim can be cleanly interpreted as evidence about the ZWD input signal specifically.","section":"Results, 'The precipitation gain costs little...' / Supp. Note 3"}],"minor_comments":[{"comment":"The Log Spectral Distance is described in the text as a root-mean-square difference over wavenumbers, but the displayed equation is sqrt((log10 S_pred - log10 S_target)^2), which is just an absolute value. The averaging over wavenumbers should be made explicit in the equation.","section":"Methods, Eq. (2)"},{"comment":"The abstract states 'the first integration of GNSS-derived ZWD into Aurora' without the 'to our knowledge' qualifier used in the Introduction. Please align the wording, and note that the same applies to 'first integration of a geodetic observable into an atmospheric foundation model' in the Results/Discussion.","section":"Introduction / Abstract"},{"comment":"The caption says the rollout scores are 'averaged over ten initialisation dates spanning the held-out test set.' Please clarify whether these ten dates are the same as the ten checkpoints used in the paired tests; if not, the relationship between the two ensembles should be stated.","section":"Results, Fig. 3 caption"},{"comment":"The bilinear-versus-conservative regridding tradeoff is discussed clearly. A sentence noting that the absolute scores may depend on the regridding choice while the contrast is unaffected is already present and is helpful.","section":"Methods, 'Precipitation target and regridding'"},{"comment":"At the 75th percentile, lw=10 (0.692) falls below the 'No ZWD' baseline (0.695). The text says 'drops below it,' which is correct, but the main-text sentence 'only at lambda_ZWD=10 does skill fall back to near-baseline' should be reconciled with this below-baseline value for precision.","section":"Supplementary Note 2, Table S2"}],"recommendation":"major_revision","confidential_remarks":"The paper is promising and generally well written, with a clear protocol and useful supplementary material. The main issue is causal identification: the headline gain is attributed to the GNSS input, but the experimental contrast also changes the auxiliary loss, and the input itself is a fitted function of ERA5 humidity. These confounds are fixable with additional control experiments (ZWD as output only; ERA5-only moisture-column input). I would not require full retraining at both scales if the 110M scale can be used for the decisive controls, but the central attribution claim needs to be backed by at least one such experiment. The manuscript's own acknowledgment of the ZWDX dependence is a point in its favor, but the missing control remains load-bearing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know before you read it. The experiment is cleaner than most fine-tuning studies, and the improvement in precipitation skill, especially in the extreme tail, looks real rather than like noise. But the paper's central claim — that GNSS-derived ZWD causes the improvement — is not established by the design. The authors are honest about one of the two problems and silent on the other, so the paper deserves a serious referee but not acceptance as is.\n\nWhat is genuinely new: this is the first attempt to feed a GNSS-derived moisture product into a pretrained weather foundation model, and the authors do it with a matched training protocol, shared initialization, identical schedules, and ten matched checkpoints. That scaffolding is good. The cross-scale comparison (110M vs 1.3B) and the spectral analysis are thoughtful, and the case studies are the right way to show the gain is not just an aggregate artifact. If the result holds, it matters for operational forecasting.\n\nThe soft spots are load-bearing. First, ZWDX is an XGBoost model trained on ERA5 humidity with GNSS station labels. The input to Aurora is therefore a transformation of ERA5, not raw GNSS data. Without an ERA5-only column-moisture control — e.g., ERA5 integrated water vapor, or a simple column integral of specific humidity — you cannot attribute the gain to GNSS information. The paper's phrase \"GNSS observations therefore encode information\" overreaches. Second, and I think this is the more serious issue, the Step 2 contrast conflates input and output. Model A receives ZWD as an additional input and is trained with ZWD as an auxiliary output with weight λ_ZWD = 2. Model B has neither. So the comparison varies two things at once. The gain could come from the extra prediction target forcing the shared encoder to build moisture representations, not from the input signal. The loss-weight sweep does not fix this, because λ_ZWD is only changed while the ZWD output remains present. You need a control that adds ZWD as an output only, or as an input only, to separate the pathways. The Methods sentence claiming the contrast \"isolates the contribution\" is simply wrong.\n\nA smaller but real issue: the paired t-test across ten checkpoints from a single run is pseudo-replication. The checkpoints are serially correlated, so p-values like p=0.004 are not trustworthy. The authors acknowledge this and frame it as a robustness check, which is fair, but the main text should not present it as significance testing.\n\nWho should read this: anyone working on integrating non-reanalysis observations into ML weather models, and anyone designing ablations for foundation-model fine-tuning. It's a useful paper to argue about, and the authors' candor about limitations makes it worth engaging. Send it to review, but ask for the two missing controls or a reframed claim. If the authors cannot run the controls, they should say \"a ZWDX-derived moisture field as an additional input and auxiliary target improves precipitation skill\" rather than \"GNSS observations encode information.\"","headline":"The paper is careful and the empirical pattern is consistent, but the headline GNSS attribution rests on two confounds the authors disclose yet never control for: ZWDX is an ERA5-based proxy, and the Step 2 design adds ZWD as both input and auxiliary loss, so the gain could be multi-task regularization rather than the input signal.","tokens_in":24159,"tokens_out":3396,"would_cite":false,"duration_ms":35833,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding GNSS-derived column-moisture measurements to the Aurora weather foundation model improves 6-hour precipitation forecasts, with the largest gains — up to an 8.8% rise in Equitable Threat Score — at the 99th percentile of rain intensit","keywords":["GNSS","Zenith Wet Delay","precipitation forecasting","weather foundation model","Aurora","extreme precipitation","column water vapour","Equitable Threat Score"],"falsifier":"Train the 'With ZWD' and 'Without ZWD' models with the same protocol but replace ZWD with an ERA5-only integrated-water-vapour column (or with ZWDX values whose GNSS-labelled ordering has been destroyed, e.g., shuffled in time/location). If the 99th-percentile ETS gain vanishes or shrinks to noise, the reported benefit is not attributable to GNSS observations, only to adding any column-moisture variable. A second check: run the 110M model from at least 3 independent seeds and see whether the +50% ETS99 gain holds across seeds.","tokens_in":23328,"feed_emoji":"🛰️","tokens_out":6202,"duration_ms":62037,"temperature":0.7,"pith_summary":"This paper claims that Zenith Wet Delay (ZWD) — the delay GNSS signals accumulate from atmospheric water vapour, a direct measure of column moisture — can be added to a leading machine-learning weather foundation model (Aurora) through a simple extension of its surface encoder, and that doing so systematically improves fine-tuned 6-hour precipitation forecasts. The improvement grows with event severity: at the 99th percentile of observed precipitation, the Equitable Threat Score rises by 8.8% relative to an identically trained model without ZWD, and the predicted precipitation power spectrum becomes more realistic at synoptic and planetary scales. The gain is reproduced across training checkpoints, persists over short-to-intermediate lead times, and is larger for a smaller model, suggesting that the column-moisture signal is most valuable when the network has less capacity to reconstruct it internally. A sympathetic reader would care because heavy precipitation is precisely the forecast target where both numerical and machine-learning models are weakest, and ZWD is an all-weather, observation-derived quantity not currently used by these models.","feed_headline":"GNSS moisture data boost AI extreme-rain forecasts 8.8%","feed_subtitle":"Adding satellite-derived column water vapour to the Aurora model lifts skill most where rain is heaviest.","key_machinery":"The central object is Zenith Wet Delay (ZWD), the signal delay a GNSS satellite link accumulates from atmospheric water vapour, mapped to the zenith and separated from the hydrostatic delay; it is a direct, weather-independent measure of column water vapour. The carrying mechanism is the ZWDX gridded product (an XGBoost model trained on over 19,000 GNSS stations) whose 0.25-degree, 6-hourly ZWD fields are added to Aurora as a new surface variable through an extension of the linear surface encoder and decoder, leaving the Perceiver encoder, 3D Swin Transformer U-Net backbone, and decoder untouched. The experimental contrast is between two fine-tuned models that share initialisation, optimiser","core_discovery":"The paper's central claim is that GNSS-derived Zenith Wet Delay, delivered as a gridded product (ZWDX) and inserted into Aurora as a new surface variable, carries information that Aurora can exploit for precipitation forecasting. The authors fine-tune Aurora for 6-hour accumulated precipitation in two matched configurations that differ only in whether ZWD is included, and report that the ZWD model learns ZWD itself at parity with pretrained variables (R=0.998, relative MAE 1.92%), improves precipitation MAE by 0.7%, RMSE by 1.8%, MSE by 3.7%, and FSS at the 95th percentile by 1.5%, and raises ETS by +1.2%, +1.9%, +3.9%, and +8.8% at the 75th, 90th, 95th, and 99th percentiles. They further re","pith_inferences":["One testable extension the paper leaves implicit: replace ZWD with an ERA5-only integrated-water-vapour column (or with ZWDX fields whose GNSS-ordering is destroyed) and rerun the matched-pair contrast. If the 99th-percentile ETS gain vanishes, the benefit is about adding column moisture, not about GNSS observations specifically; if it survives, the GNSS labels genuinely matter.","If the attribution is real, feeding raw GNSS station delays directly rather than the XGBoost-smoothed product should increase the gain, especially in data-sparse regions — a prediction the paper gestures at in its discussion.","The same mechanism should transfer to other non-reanalysis observables (radar reflectivity, dense environmental sensor networks) given the encoder can ingest them, so the general principle is broader than ZWD.","The paper's checkpoint-pairing design samples within-trajectory variability, not seed-to-seed variability; a multi-seed repeat at the 110M scale would help quantify run-to-run robustness, which the authors explicitly note is prohibitively expensive at 1.3B."],"forward_implications":["If the claim holds, weather foundation models can ingest variables absent from reanalysis and improve on their hardest task: extreme precipitation.","Downstream users doing short-range (6h to ~3-day) precipitation forecasting from fine-tuned foundation models should expect the largest benefit in the heavy-precipitation tail, where ETS gains reach roughly 9% at the large scale and 50% at the small scale.","The result implies that the moisture column, not just its level-resolved humidity profile, is a useful conditioning variable for AI precipitation models; ZWD effectively hands the model the column integral directly.","Because the gain grows as model capacity shrinks, smaller or regional models — the typical operational setting — may benefit disproportionately from such observational inputs.","The spectral improvement at synoptic/planetary scales suggests ZWD helps correct the over-smoothing of precipitation fields, not merely their intensity calibration."],"fun_headline_variants":["GNSS water vapor data lifts AI extreme-rain skill 8.8%","Satellite moisture helps AI forecast worst downpours better","Aurora model gains edge on extreme rain via GNSS moisture","Adding GNSS wet delay to AI model boosts severe rain forecasts","AI weather model improves extreme rain 8.8% with GNSS data"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the ZWDX labels contain genuine GNSS moisture information beyond what ERA5 already supplies — the paper admits the product is XGBoost-predicted from ERA5 inputs and never tests an ERA5-only column-moisture baseline; a secondary premise is that ten checkpoints from one training run per model approximate run-to-run variability.","fun_headline_variants_meta":{"raw":{"variants":["GNSS water vapor data lifts AI extreme-rain skill 8.8%","Satellite moisture helps AI forecast worst downpours better","Aurora model gains edge on extreme rain via GNSS moisture","Adding GNSS wet delay to AI model boosts severe rain forecasts","AI weather model improves extreme rain 8.8% with GNSS data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00049,"raw_usage":{"total_tokens":2255,"prompt_tokens":759,"completion_tokens":1496,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":503,"completion_tokens_details":{"reasoning_tokens":1417}},"tokens_in":503,"tokens_out":1496,"duration_ms":11850,"temperature":1.0,"reasoning_tokens":1417,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T08:24:34.046282+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the 'With ZWD' and 'Without ZWD' models with the same protocol but replace ZWD with an ERA5-only integrated-water-vapour column (or with ZWDX values whose GNSS-labelled ordering has been destroyed, e.g., shuffled in time/location). If the 99th-percentile ETS gain vanishes or shrinks to noise, the reported benefit is not attributable to GNSS observations, only to adding any column-moisture variable. A second check: run the 110M model from at least 3 independent seeds and see whether the +50% ETS99 gain holds across seeds.","supporting_citations":[],"review_version":2}