{"id":"9de56221-7453-4492-94d7-cae58baa6053","arxiv_id":"2411.12640","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Leadsee-Precip is a deep learning diagnostic model that converts circulation fields into 6-hour precipitation with a loss weighted toward rare heavy rain, and reports strong hit scores for heavy rain in China.","lead":"The authors train a global deep learning model that produces 6-hour precipitation maps from atmospheric circulation fields, using satellite and radar rainfall as its training target. The model is meant to improve forecasts of heavy rain, where many AI weather models remain weak.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Max-pooled CMORPH target is the sub-grid maximum, not a 0.25° grid-cell accumulation, so the global TS/FSS and the claimed NWP/AI comparisons do not measure skill for grid-cell precipitation.","rationale":"The central claim requires both a valid target and a fair comparison. The max-pooling interpolation of CMORPH corrupts the target at the resolution used by the model, independent of any retrieval bias. This is the most load-bearing concern because it undermines the validity of the quantitative skill scores in Table 1, which are the only global evidence for the headline claim. The reader's weakest assumption identified CMORPH accuracy and mentioned max-pooling, but focused on retrieval bias; I sharpen the concern to the interpolation method itself, which is internal to the paper and not an external data limitation. The lack of quantitative baselines against AI/NWP models is also important, but the max-pooling issue is more fundamental: even a perfect comparison would be built on an unrepresentative target. The station-based evaluation offers some independent support, and the model could be correct in structure, which is why the verdict remains conditional rather than reject. The proposed test would settle whether the reported skill is an artifact of the target definition.","tokens_in":8576,"tokens_out":6118,"duration_ms":59593,"concrete_test":"Recompute the 0.25° CMORPH target using area-weighted mean pooling (or conservative remapping) from the native 8-km 6-hour accumulations. Retrain Leadsee-Precip on the mean-pooled target and re-evaluate on the same test split. Then compute TS/FSS for Leadsee, FuXi, and ECMWF HRES against both the max-pooled and mean-pooled targets. If Leadsee's advantage at ≥25 mm/6h shrinks or reverses under mean-pooling, the headline claim is an artifact of the max-pooled target. Also report the ratio of max-pooled to mean-pooled target values for heavy-rain grid cells; a large ratio confirms the bias.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.5.2 states that 6-hour CMORPH accumulations at native ~8 km resolution are interpolated to 0.25° by max-pooling. Max-pooling selects the highest 8-km value inside each 0.25° cell. For convective precipitation, a 0.25° cell contains substantial sub-grid variability; the max can be several times larger than the cell-average. Thus the 'ground truth' used for training and for Table 1 is not the areal average precipitation that a 0.25° model or NWP grid-box forecast represents. The model is effectively trained to predict sub-grid convective maxima. Any comparison against AI models or NWP forecasts using this target is biased because those systems predict grid-cell mean precipitation. This is not a generic 'CMORPH is imperfect' issue: even a perfect 8-km retrieval, when max-pooled, produces a systematically overintense target. The paper offers no justification for max-pooling; standard upscaling uses averaging or conservative remapping. Consequently, the reported TS/FSS values in Table 1 are not evidence for the claim that heavy precipitation is more consistent with observations than AI or NWP models. The station-based evaluation (Table 2) is independent, but it uses station point measurements against model grid values, and no baseline models are scored there either; the only model comparison is a single case (Fig. 5).","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Leadsee-Precip, a global deep learning diagnostic model that maps ERA5 circulation fields to 6-hour accumulated precipitation at 0.25° resolution, trained against CMORPH precipitation with a new information balance (IB) loss intended to emphasize rare heavy precipitation. A LoRA fine-tuned version at 5 km resolution is presented for China using the CROA dataset. The authors report TS and FSS scores on a global test set, station-based verification over China, and a single case study comparing against FuXi and ECMWF HRES. The central claim is that heavy precipitation from Leadsee-Precip is more consistent with observations than AI global weather models and competitive with global numerical weather prediction models.","tokens_in":8894,"tokens_out":3909,"duration_ms":41859,"significance":"If the headline claim were fully supported, Leadsee-Precip would be a practically useful diagnostic tool, particularly for post-processing circulation forecasts from AI weather models. The IB weighting scheme is a reasonable and clearly motivated approach to the long-tail precipitation regression problem, and the use of independent station data for evaluation is a genuine strength. The paper also acknowledges limitations candidly (e.g., no significant heavy-precipitation improvement from fine-tuning in Fig. 6). However, the manuscript currently lacks the systematic baseline comparisons needed to support its central comparative claim, and the max-pooled CMORPH target raises a representativeness issue that affects the interpretation of all global skill scores. The results may still be valid, but the evidence as presented is insufficient to establish the abstract's claim.","major_comments":[{"comment":"The max-pooling interpolation used to convert CMORPH from its native 8 km resolution to the 0.25° target grid produces a value equal to the largest 8-km precipitation within each target cell, not the cell-area mean. Because NWP and AI weather models predict grid-cell mean precipitation, the TS and FSS scores in Table 1 do not measure skill for the same quantity that these baseline models are designed to predict. Please either justify the choice of predicting sub-grid maxima (with physical or application-specific reasons) or repeat the training and evaluation using averaging or conservative remapping, and report the sensitivity of the headline scores to this choice.","section":"Section 3.5.2 and Table 1"},{"comment":"The abstract's claim that heavy precipitation from Leadsee-Precip is more consistent with observations than AI global weather models, and competitive with global NWP models, is not supported by any systematic evaluation in the manuscript. The only direct side-by-side comparison is the single North China case in Fig. 5. Please add a table reporting the same TS and FSS thresholds for baseline models (e.g., FuXi, GraphCast, ECMWF HRES) computed on the same test period and target grid, ideally with the same station-based and grid-based protocols, together with uncertainty estimates.","section":"Abstract and Section 4"},{"comment":"The information balance weights in Eq. (1) depend on the 92-bin partition used to estimate P(y_i) and on the temperature coefficient τ. The bin edges are not specified anywhere in the paper, and τ=2 is reported only as the result of ablation experiments that are not shown. Since these choices directly determine the loss function and therefore the trained model, please provide the bin boundaries and the frequency curve used to define P(y_i), and report the ablation over τ either in the main text or in a supplement.","section":"Section 3.2"},{"comment":"No uncertainty estimates are provided for the TS, FSS, or bias values. This is particularly important for the high-threshold rows with very few positive samples (e.g., TS=0.003 at 100 mm/6h in Table 1). Confidence intervals, bootstrap estimates, or at least the number of event days and grid points used would be needed to judge whether differences between models, or between Tables 2 and 3, are meaningful.","section":"Tables 1–3"},{"comment":"The text in Section 4.3 states that the fine-tuned model shows no significant improvement in heavy precipitation compared with the original model (Fig. 6), yet Table 3 and the conclusion highlight higher TS scores. Please report the per-threshold improvements explicitly and temper the conclusion accordingly; if the improvement is concentrated at lower thresholds or in overall pattern, this should be stated clearly so that readers do not over-read the headline result.","section":"Section 4.3 and Fig. 6"}],"minor_comments":[{"comment":"In the FSS formula, P and T are not explicitly defined as neighborhood-averaged fractions; please add that these are mean values over a 7×7 pixel window and clarify that the same window is used for all thresholds.","section":"Section 3.4, Eq. (3)"},{"comment":"The text lists the thresholds as 0.1, 1, 5, 10, 25, and 50 mm/6h, but Table 1 also includes 100 mm/6h; the list should be corrected to include all seven thresholds.","section":"Section 4.1"},{"comment":"There is a typo: 'Panle' should be 'Panel'. Additionally, the caption refers to 24-hour accumulated precipitation while the text discusses 25 mm/6h; please clarify which accumulation interval the thresholds refer to in the figure.","section":"Fig. 5 caption"},{"comment":"Please clarify how the hourly ERA5 data from 2013–2022 are aggregated to match the 6-hourly data from 1998–2012, and whether the 6-hourly data are instantaneous or accumulated values. The input consistency across the mixed-resolution training set affects the reproducibility of the model.","section":"Section 3.5.1"},{"comment":"Please specify whether the normalization in Eq. (1) is performed over the full training set or per batch. If the latter, the loss magnitude depends on the batch composition, which should be stated for reproducibility.","section":"Section 3.2, Eq. (1)"},{"comment":"Reference [25] appears unrelated to station precipitation data; please verify that this citation is appropriate and consider citing the original station dataset source instead.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The central comparative claim in the abstract is currently not substantiated by a baseline comparison table, and the max-pooled CMORPH target is a potentially serious representativeness issue. If the authors can add a fair baseline table (ideally on the same target definition) and address the target representation question, the paper could become a solid contribution. Also note that no code or data availability statement is provided, which limits reproducibility; I would encourage the editor to request one."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nMy take: the architecture and the information balance loss are a reasonable engineering contribution, but the target definition breaks the main comparison claims. The paper's real novelty is applying logit-adjustment-style weighting to regression for precipitation, plus the LoRA fine-tuning pipeline to 5 km over China. The independent station evaluation is a genuine positive; the station data were held out.\n\nThe soft spot is load-bearing. Section 3.5.2 says CMORPH 8-km data are max-pooled to 0.25 degrees. Max-pooling picks the wettest sub-grid pixel, so the target is the convective maximum, not the grid-cell accumulation. For convective events, that can be several times the areal average. The model is effectively trained to predict sub-grid maxima, and Table 1's TS and FSS are not measuring skill for grid-cell precipitation. Any comparison against AI or NWP models, which forecast cell means, is unfair. The abstract's claim of being 'more consistent with observations' and 'competitive' is backed only by a single qualitative case in Fig. 5; there is no baseline table anywhere. The station evaluation is independent but also lacks baseline models.\n\nOther gaps: tau is set to 2 after an unshown ablation; the 92-bin partition is not specified; hyperparameters for LoRA are not given; no code or weights. The paper's own Fig. 6 admits no significant heavy-precipitation gain from fine-tuning.\n\nThis is all fixable. If they switch to conservative remapping or averaging, add quantitative baselines, and release details, this could be a solid operational-style paper. The IB scheme is worth a look even now. I would not cite the headline result, but I would send it to a serious referee, because the core ideas deserve scrutiny and the data evaluation exists. The referee should require the target fix and baseline tables.\n\nBest.","headline":"A useful precipitation diagnostic architecture undermined by a target-definition choice that invalidates the headline comparisons.","tokens_in":9376,"tokens_out":2917,"would_cite":false,"duration_ms":30329,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A deep learning model converts atmospheric circulation into 6-hour precipitation maps whose heavy rainfall is more consistent with observations than other AI weather models and competitive with global numerical weather prediction models.","keywords":["heavy precipitation","information balance scheme","deep learning","precipitation diagnostic model","CMORPH","long-tail distribution","threat score","fractions skill score"],"falsifier":"Verify Leadsee-Precip's 6-hour, >25 mm precipitation against an independent gauge-based network (e.g., the roughly 10,470 Chinese stations the paper uses) on the same events, computing Threat Scores for Leadsee-Precip and the paper's global numerical baseline side by side. If the numerical model's Threat Score is higher, the central claim of competitive heavy-precipitation skill is falsified.","tokens_in":8399,"feed_emoji":"🌧️","tokens_out":13148,"duration_ms":114818,"temperature":0.7,"pith_summary":"The paper aims to close a known weak spot in deep learning weather forecasting: precipitation, especially heavy rain, which predictive models handle poorly because heavy events are rare and the data distribution is long-tailed. Leadsee-Precip is a global diagnostic model that takes a single time step of atmospheric circulation fields (five upper-air variables on 13 pressure levels plus four surface variables from the ERA5 reanalysis) and outputs a 6-hour accumulated precipitation field. Two design choices carry the argument: an information balance loss that weights each sample's error by the negative log frequency of its precipitation bin, so rare heavy rain errors count more, and training against the satellite-radar CMORPH product instead of ERA5 precipitation. On the global test set, the model reaches a Threat Score of 0.185 and a Fractions Skill Score of 0.570 at the 25 mm/6h threshold, and its heavy-rain fields line up better with observations than the native precipitation outputs of other AI weather models while staying competitive with global numerical weather prediction models.","feed_headline":"A deep learning model outdoes other AI weather models at heavy rain","feed_subtitle":"It scores TS 0.185 and FSS 0.570 at 25 mm/6h and can attach to any circulation model to issue rain forecasts.","key_machinery":"The central object is the information balance (IB) scheme, a loss-weighting rule that multiplies each sample's squared error by its normalized negative log frequency, $W_i = [-\\log P(y_i)]^\\tau / \\sum_i [-\\log P(y_i)]^\\tau$, where $P(y_i)$ is the empirical frequency of the precipitation bin containing the target value and $\\tau = 2$. The scheme converts the long-tail distribution of precipitation into a per-sample importance weight, making the optimizer more sensitive to rare heavy-rain errors than to the frequent zero and light-precipitation samples that dominate ordinary MSE training. The second load-bearing ingredient is the training target, the satellite-radar CMORPH precipitation aggregated to 6 hours and max-pooled to 0.25 degrees, which the paper uses in place of the ERA5 precipitation field because that reanalysis product is known to be biased.","core_discovery":"Leadsee-Precip is presented as a solution to the long-tail precipitation problem in deep learning weather models. The model uses an encoder-decoder architecture with separate 3D and 2D convolutions for upper-air and surface variables, a MogaNet (multi-order gated aggregation) bottleneck, and a shortcut connection to reconstruct 0.25-degree global precipitation. The information balance scheme computes a per-sample weight $W_i = [-\\log P(y_i)]^\\tau / \\sum_i [-\\log P(y_i)]^\\tau$ over 92 magnitude bins, multiplying the MSE loss so that rare, high-magnitude precipitation errors dominate training. With $\\tau = 2$ and CMORPH-derived 6-hour targets resampled by max-pooling, the model achieves TS 0.185 and FSS 0.570 for 6-hour precipitation exceeding 25 mm on a global test set from April to September 2022, and station-based evaluation over China shows TS 0.11 at the 25 mm/6h threshold. A LoRA (low-rank adaptation) fine-tuned version at 5 km resolution over China, trained on the CROA observational analysis, improves these station-based scores (TS 0.14 at 25 mm/6h) and better captures the location of heavy rainbands. The paper also shows the model can be driven by the circulation forecasts of an AI weather model (FuXi); the resulting precipitation fields are weaker than with reanalysis input but still capture heavy events in North China better than FuXi's native precipitation.","pith_inferences":["The information balance weighting is a generic recipe for any regression task with a long-tailed target distribution, such as extreme winds, air quality, or streamflow; the paper only applies it to precipitation.","Because the training targets are made by max-pooling high-resolution CMORPH rain rates into 0.25-degree cells, the heavy-rain labels are likely inflated relative to grid-cell averages, which may explain why the station-based Threat Score (0.11) is lower than the CMORPH-based one (0.185) at 25 mm/6h.","Coupling a diagnostic precipitation model to an autoregressive circulation forecast makes the precipitation skill inherit the circulation model's error growth; a stronger test would be to fine-tune on the circulation model's own forecast fields and verify over longer lead times, as the paper suggests but does not carry out.","The fine-tuning result, which raises TS from 0.11 to 0.14 at 25 mm/6h despite a 5 km output grid, suggests that resolution alone is not the bottleneck for heavy-rain skill; the training target's fidelity and the loss weighting are more likely levers."],"forward_implications":["When driven by the ERA5 circulation fields, Leadsee-Precip acts as a global diagnostic tool that reconstructs 6-hour precipitation at 0.25 degrees from a single atmospheric state.","Because it accepts circulation fields as input, the model can be attached to any global circulation model to issue precipitation forecasts, at the cost of weaker intensity when the input is a model's predicted rather than reanalyzed circulation.","Fine-tuning the upsampling branch on a 5 km regional observational analysis over China raises Threat Scores and lowers bias, and the same procedure can in principle be applied to other regions with high-quality analyses.","The reported skill at the 25 mm/6h threshold (TS 0.185, FSS 0.570) gives other AI weather models a concrete precipitation benchmark to beat, in addition to the usual circulation-variable metrics.","Replacing an AI weather model's native precipitation output with a separate diagnostic model trained on satellite-radar targets can improve heavy-rain realism without changing the circulation forecast core."],"supporting_citations":[{"why":"Supplies the ERA5 reanalysis circulation fields used as model input, with mixed 6-hour and 1-hour temporal resolution.","marker":"[1]"},{"why":"Pangu-Weather excludes precipitation, illustrating the gap in existing AI global weather models that this paper targets.","marker":"[5]"},{"why":"GraphCast is an AI weather model whose precipitation skill is not claimed, motivating the need for a dedicated diagnostic model.","marker":"[6]"},{"why":"FuXi is the AI circulation model used in the coupling experiment, and its native precipitation output serves as a comparison baseline.","marker":"[7]"},{"why":"Documents biases in the ERA5 precipitation field, justifying the switch to CMORPH as the training target.","marker":"[11]"},{"why":"Provides the logit adjustment loss idea that the information balance scheme generalizes from classification to regression.","marker":"[22]"},{"why":"Defines the Fractions Skill Score used to evaluate the spatial skill of the precipitation fields.","marker":"[23]"},{"why":"Supplies the satellite-radar CMORPH product from which the 6-hour 0.25-degree training targets are made.","marker":"[24]"}],"fun_headline_variants":["AI rain model uses rare-event weighting to beat rivals on heavy storms","Deep learning model improves heavy rain forecasts, works with any weather model","Information balance scheme helps AI forecast heavy rain more accurately","Global AI model with rare-event loss outperforms peers on extreme precipitation","Heavy rain forecasting boosted by AI model with balanced training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The CMORPH-derived 6-hour accumulated precipitation product is treated as accurate ground truth, including for heavy rain, after a max-pooling interpolation to 0.25 degrees; if CMORPH is biased for extreme rain, especially over land, the model learns the retrieval's biases and the reported TS and FSS do not measure skill for actual precipitation.","fun_headline_variants_meta":{"raw":{"variants":["AI rain model uses rare-event weighting to beat rivals on heavy storms","Deep learning model improves heavy rain forecasts, works with any weather model","Information balance scheme helps AI forecast heavy rain more accurately","Global AI model with rare-event loss outperforms peers on extreme precipitation","Heavy rain forecasting boosted by AI model with balanced training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000286,"raw_usage":{"total_tokens":1727,"prompt_tokens":1038,"completion_tokens":689,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":654,"completion_tokens_details":{"reasoning_tokens":603}},"tokens_in":654,"tokens_out":689,"duration_ms":7721,"temperature":1.0,"reasoning_tokens":603,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T17:19:23.278294+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Verify Leadsee-Precip's 6-hour, >25 mm precipitation against an independent gauge-based network (e.g., the roughly 10,470 Chinese stations the paper uses) on the same events, computing Threat Scores for Leadsee-Precip and the paper's global numerical baseline side by side. If the numerical model's Threat Score is higher, the central claim of competitive heavy-precipitation skill is falsified.","supporting_citations":[{"cited_title":"The era5 global reanalysis","cited_arxiv_id":null,"evidence_quote":"Supplies the ERA5 reanalysis circulation fields used as model input, with mixed 6-hour and 1-hour temporal resolution."},{"cited_title":"Fuxi: A cascade machine learning forecasting system for 15-day global weather forecast","cited_arxiv_id":null,"evidence_quote":"FuXi is the AI circulation model used in the coupling experiment, and its native precipitation output serves as a comparison baseline."},{"cited_title":"An evaluation of era5 precipitation for climate monitoring","cited_arxiv_id":null,"evidence_quote":"Documents biases in the ERA5 precipitation field, justifying the switch to CMORPH as the training target."},{"cited_title":"A long-term assessment of precipitation forecast skill using the fractions skill score","cited_arxiv_id":null,"evidence_quote":"Defines the Fractions Skill Score used to evaluate the spatial skill of the precipitation fields."},{"cited_title":"Cmorph: A method that produces global precipitation estimates from passive microwave and infrared data at high spatial and temporal resolution","cited_arxiv_id":null,"evidence_quote":"Supplies the satellite-radar CMORPH product from which the 6-hour 0.25-degree training targets are made."}],"review_version":1}