Pith. sign in

REVIEW 5 major objections 7 minor 48 references

Self-supervised Spatial-Temporal Learner for Precipitation Nowcasting

T0 review · 5 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Rain nowcasting accuracy jumps 18 percent with self-supervised model

desk verdict Worth a major revision: the pretraining ablation is real, but the headline outperformance claim is undercut by the paper's own skill scores and a 24x parameter mismatch. read the letter →

arxiv 2412.15917 v1 pith:XW57YX2O submitted 2024-12-20 cs.LG

Haotian Li , Arno Siebes , Siamak Mehrkanoon This is my paper · ORCID
classification cs.LG
keywords precipitationnowcastingself-supervisedlearningmaskedimagemodelingspatial-temporalforecastingconvolutionalneuralnetworkstranslationnetworkNL-50datasetradar
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that self-supervised pretraining can improve precipitation nowcasting on radar images, a task usually handled with fully supervised deep learning. It proposes SpaT-SparK, a convolutional encoder-decoder pretrained to reconstruct randomly masked precipitation maps, together with a small translation network that maps latent representations of past radar frames to future ones. On the Dutch NL-50 dataset the model reports the lowest pixel-wise MSE and the highest accuracy among the compared methods, beating the supervised SmaAt-UNet baseline by about 18% in accuracy and 8.9% in pMSE. If the result holds, precipitation nowcasting can benefit from unlabeled radar archives and may become more accurate for short-term weather-dependent decisions.

What carries the argument

The central object is SpaT-SparK, a hierarchical CNN encoder-decoder lifted from SparK and adapted to sequences. In pretraining, input radar frames are patchified and masked with a tube-masking strategy; the sparse encoder produces four resolution levels, a densify network fills masked positions with learnable embeddings, and a light decoder reconstructs the masked patches, minimizing per-patch normalized L2 error on masked positions. In fine-tuning, a translation network of per-level Conv2D+tanh layers maps the encoded past features to predicted future features; the pretrained decoder then turns those translated features into precipitation maps. The machinery's job is to let the mask-reconstruction pretraining teach the encoder useful spatial representations while the translation network handles the temporal shift, so the encoder can focus on representing rather than forecasting.

What would settle it

Run the same SpaT-SparK training pipeline with an encoder matched to SmaAt-UNet's parameter count, or train SmaAt-UNet at SpaT-SparK's scale; if the pMSE and accuracy gaps shrink or vanish, the improvement is from capacity rather than from masked-image pretraining.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that combining masked image modeling with a learned temporal translation beats both a fully supervised UNet-style model and a plain pretrained encoder-decoder on this nowcasting task. SpaT-SparK with a ResNet-18 encoder achieves the best pMSE (0.0132) and accuracy (0.913) in Table I, improves precision and false-alarm rate, and maintains an edge at each individual lead time from t=1 to t=12. The ablation study supports the contribution of each component: removing the self-supervised pretraining degrades every metric, and removing the translation network (the SparK baseline) performs worse than SpaT-SparK on pMSE, accuracy, precision, and FAR. The paper also acknowledges that visual quality of predictions did not improve sufficiently and that recall, F1, CSI, and HSS are lower than the baseline.

Load-bearing premise

The claim that self-supervised pretraining is what improves nowcasting assumes the comparison against SmaAt-UNet is fair, but SpaT-SparK is roughly 24 times larger in parameters and 7 times heavier in computation, so raw capacity differences could explain the gains.

Editorial extensions

If this is right

  • If SpaT-SparK's gains are real, self-supervised pretraining on unlabeled radar archives could reduce reliance on large labeled nowcasting datasets.
  • A translation network that maps latent sequences may be a reusable component for other spatial-temporal forecasting tasks with convolutional encoders.
  • The pMSE and accuracy improvements at every lead time suggest the method's benefit is not confined to the first prediction frame.
  • The reported inference time stays in the seconds range on one A100 GPU, so the accuracy improvement does not obviously preclude operational use.
  • Operational nowcasting systems could adopt the self-supervised pretraining stage without changing their downstream model family.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the reported accuracy and pMSE gains are confounded with model scale — SpaT-SparK uses about 24 times more parameters and 7 times more FLOPs than SmaAt-UNet, so the head-to-head comparison does not by itself prove that self-supervised pretraining caused the improvement.
  • Editorial inference: the paper's 'outperforms' statement is metric-dependent; on CSI and HSS the supervised baseline remains ahead, so the practical claim should be read as 'better on error magnitude and accuracy, not on all skill scores.'
  • A testable extension the paper leaves implicit: applying the same pretraining recipe to a smaller encoder or to a matched-parameter UNet would separate the self-supervised benefit from capacity.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper proposes SpaT-SparK, a self-supervised spatial-temporal model for precipitation nowcasting. The architecture combines a CNN encoder-decoder pretrained with masked image modeling (MIM), following the SparK framework, with a translation network of Conv2D+tanh layers that maps latent representations of past precipitation sequences to those of future sequences. The model is evaluated on the NL-50 dataset with a 12-step (1-hour) nowcasting setup. The authors report that SpaT-SparK with a ResNet-18 encoder achieves the lowest pMSE and highest accuracy among SmaAt-UNet, SparK, and their own variants, and they provide an ablation study showing that removing pretraining or freezing components degrades performance. The paper concludes that self-supervised pretraining and the translation network improve precipitation nowcasting.

Significance. If the central claim were fully supported, the paper would make a useful contribution by demonstrating that MIM pretraining can be adapted to a spatio-temporal nowcasting task with a relatively simple translation network. The release of code, the within-architecture ablation, and the inclusion of a held-out test year (2019) are strengths. However, the evidence is currently mixed: the model shows lower pMSE and higher accuracy than SmaAt-UNet, but it simultaneously has substantially worse recall, F1, CSI, and HSS on the same test set. Because the abstract and conclusion claim general outperformance, the paper's own numbers contradict the central claim. The capacity difference between SpaT-SparK and SmaAt-UNet also confounds the attribution of the improvement to self-supervised learning. The results are therefore suggestive rather than conclusive, and the manuscript requires substantive revision before the claims can be accepted.

major comments (5)
  1. [Abstract and Table I] The abstract states that SpaT-SparK 'outperforms existing baseline supervised models... providing more accurate nowcasting predictions,' but Table I shows that SmaAt-UNet is better on recall (0.846 vs 0.588), F1 (0.723 vs 0.560), CSI (0.566 vs 0.389), and HSS (0.269 vs 0.255). Only pMSE, accuracy, precision, and FAR favor SpaT-SparK. Since the paper itself reports these drops in Section V-C but still concludes general outperformance, the central claim is internally inconsistent with the presented evidence. The authors must either restrict the claim to the metrics that improve or provide a principled reason why pMSE and accuracy should be treated as the decisive skill metrics despite the consistent losses in the other four metrics.
  2. [Section V-B, Eq. (1) and Table I] The choice of pMSE and accuracy as headline metrics is not justified for an imbalanced precipitation dataset. Accuracy is dominated by true negatives because most pixels are no-rain, and pMSE is a per-pixel squared error that does not explicitly reward correct detection of rain events. The paper reports a large accuracy margin (0.913 vs 0.774) alongside a much lower recall (0.588 vs 0.846), which strongly suggests that the accuracy gain is a class-imbalance artifact: the model correctly predicts more no-rain pixels but misses a large fraction of actual rain. The authors should report a balanced skill score (e.g., F1, CSI, or HSS) as the primary metric, or report the full confusion-matrix-based metrics at each rain-rate threshold, before claiming 'more accurate nowcasting predictions.'
  3. [Table III and Section V-C] The comparison to SmaAt-UNet is confounded by a large model-capacity difference: SpaT-SparK has 96.5M parameters and 172.6 GFLOPs, whereas SmaAt-UNet has 4.0M parameters and 24.7 GFLOPs. The pMSE improvement over SmaAt-UNet (0.0132 vs 0.0145) may therefore stem primarily from increased capacity rather than from self-supervised pretraining or the translation network. The within-architecture ablation in Table II supports the contribution of pretraining and the translation network, but the comparison to SmaAt-UNet as evidence for the SSL method is not fair. The authors should either train a comparable-capacity supervised baseline or explicitly frame the result as 'with equivalent or larger capacity, the proposed method improves pMSE relative to SmaAt-UNet,' which would be a much weaker claim than 'SSL outperforms supervised baselines.'
  4. [Section V-C and Remark after Fig. 2] The paper includes a Remark stating that 'the visualizations of the predictions did not show sufficient quality improvements.' This statement directly undermines the claim of 'more accurate nowcasting predictions.' If the qualitative predictions do not improve, the quantitative gains in pMSE and accuracy may reflect only small numerical differences or a bias toward overpredicting no-rain areas. The authors should either present the visualizations they refer to, explain why the qualitative comparison is not a valid indicator in this setup, or temper the conclusion to be consistent with the admitted lack of visible improvement.
  5. [Section IV and Section V-A] No validation split or multiple-run statistics are reported. The dataset is partitioned into training (2016-2018) and test (2019) years, but hyperparameters (e.g., mask ratio 0.6, learning rates, epochs) appear to be chosen on the test set, and all results appear to come from a single run. Without error bars, standard deviations, or significance tests, the reported differences—many of which are small (e.g., pMSE 0.0132 vs 0.0135 for ResNet-18 vs ResNet-50)—cannot be distinguished from random variation. The authors should add a validation split for model selection and report mean and standard deviation over at least three runs, or use a paired significance test, before asserting consistent improvements.
minor comments (7)
  1. [Abstract and Section II] The abstract mentions a '6-hour time frame,' but the experiments use T=12 with a 5-minute temporal resolution, i.e., a 1-hour prediction horizon. The relation between the stated 6-hour nowcasting scope and the actual evaluation horizon should be clarified.
  2. [Section III, Fig. 1] The caption of Fig. 1 says the visualizations are 'adapted for illustration purposes' and do not reflect actual predictions. This is fine, but the figure should be labeled more clearly in the text so readers do not interpret the schematic outputs as real precipitation maps.
  3. [Section IV] The data filtering step is described as selecting maps where the number of pixels above 0.5 mm/h exceeds 50% of the image pixels. This is a strong filter that excludes many rain events; the authors should report how many raw samples were discarded and whether this affects the representativeness of the dataset.
  4. [Section V-A] The mask ratio is set to 0.6 in both pretraining and fine-tuning, but no sensitivity analysis is provided. Since tube masking and mask ratio are central to the MIM design, a small sweep (e.g., 0.4–0.8) would strengthen the paper.
  5. [Section V-C, Fig. 2] In Fig. 2, the curves for SparK and SpaT-SparK are labeled only in the legend as 'SparK' and 'Ours,' but the caption states both use ResNet-18. It would be helpful to explicitly state this in the figure legend itself to avoid ambiguity with the ResNet-50 variants in Table I.
  6. [Section I and References] The related work section cites several works by the same research group but does not discuss how the proposed translation network differs from the 'translator' in SimVP v2 beyond using a per-level Conv2D+tanh layer. A brief comparison would help position the novelty.
  7. [General] There are minor typos and grammatical issues, e.g., 'capturing' in the last sentence of Section VI and 'reliefs' in Section III. A careful proofreading pass is recommended.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SpaT-SparK's nowcasting claims rest on held-out 2019 evaluation and same-architecture ablations, not on definitional or self-citation reduction.

full rationale

The paper's central empirical claim is that SpaT-SparK, combining masked-image-modeling pretraining with a translation network, improves precipitation nowcasting on NL-50. The claimed improvements are not definitional: pMSE (Eq. 1) and the confusion-matrix metrics (Eqs. 2-4) are computed from held-out predictions against ground-truth radar maps on the 2019 test year, with the data split by year as described in Section IV. Pretraining minimizes per-patch normalized L2 error on masked patches; fine-tuning minimizes L2 error on future sequences; neither objective is the same function as the reported pMSE or accuracy on the test set, so no reported number is forced by construction. The ablation study in Table II compares the full SpaT-SparK to the same architecture without pretraining and with frozen components, which empirically isolates the contribution of pretraining rather than assuming it. Hyperparameters such as mask ratio 0.6, learning rates, and epochs are fixed experimental choices, not parameters solved from the test targets, and the paper does not present them as derivations. The only self-citation is to the SmaAt-UNet baseline and NL-50 dataset in [10], on which one of the present authors is a coauthor; that citation provides a comparison baseline and preprocessing recipe, but the central claim is supported by the measured held-out metrics and the within-architecture ablation, not by the citation itself. The paper's remark that visualizations 'did not show sufficient quality improvements' is a limitation statement, not a circular step. I find no equation in the paper that equates a claimed prediction to a fitted input, and no load-bearing step that reduces to its own assumptions.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The model does not introduce new physical entities. The main built-in assumptions are that SSL pretraining transfers to nowcasting, that the chosen hyperparameters and data filter are adequate, and that the per-level translation network captures temporal dependencies.

free parameters (4)
  • mask_ratio = 0.6
    Used for both pretraining and fine-tuning, chosen by hand (Section V-A). Central to the MIM pretext task and the unexplained fine-tuning masking.
  • sequence_length_T = 12
    Input and prediction horizon of 12 radar frames (likely 1 hour at 5-min resolution), chosen by hand (Section V-A).
  • data_filter_threshold = 0.5 mm/h and >50% above-threshold pixels
    Data selection rule following [10], affecting the composition of training and test sets (Section IV).
  • num_epochs_pretrain = 1400
    Pretraining epoch count, chosen by hand (Section V-A). No early stopping or validation-based selection is described.
assumptions (4)
  • domain assumption Masked image modeling with random tube masking learns representations transferable to precipitation nowcasting.
    Assumed in Section III and based on SparK/MAE findings; no theoretical justification.
  • domain assumption The NL-50 dataset and its year-based split (2016-2018 train, 2019 test) are representative for evaluating nowcasting skill.
    Implicit in Section IV; no analysis of distribution shift or sensitivity to the filtering rule.
  • ad hoc to paper The per-level Conv2D+tanh translation network can capture the temporal dynamics of latent precipitation representations.
    Introduced in Section III without ablations over translation network architectures.
  • domain assumption The reported metrics, especially pMSE and accuracy, are the appropriate primary measures of nowcasting quality.
    Used in Section V-B; the paper highlights metrics where it wins while skill scores (CSI, HSS) favor the baseline.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Self-supervised Spatial-Temporal Learner for Precipitation Nowcasting." pith.science (2026). https://pith.science/paper/XW57YX2O

@misc{pith2026241215917,
  author       = {Pith},
  title        = {Pith review of: Self-supervised Spatial-Temporal Learner for Precipitation Nowcasting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XW57YX2O}},
  note         = {Machine review of arXiv:2412.15917}
}
read the original abstract

Nowcasting, the short-term prediction of weather, is essential for making timely and weather-dependent decisions. Specifically, precipitation nowcasting aims to predict precipitation at a local level within a 6-hour time frame. This task can be framed as a spatial-temporal sequence forecasting problem, where deep learning methods have been particularly effective. However, despite advancements in self-supervised learning, most successful methods for nowcasting remain fully supervised. Self-supervised learning is advantageous for pretraining models to learn representations without requiring extensive labeled data. In this work, we leverage the benefits of self-supervised learning and integrate it with spatial-temporal learning to develop a novel model, SpaT-SparK. SpaT-SparK comprises a CNN-based encoder-decoder structure pretrained with a masked image modeling (MIM) task and a translation network that captures temporal relationships among past and future precipitation maps in downstream tasks. We conducted experiments on the NL-50 dataset to evaluate the performance of SpaT-SparK. The results demonstrate that SpaT-SparK outperforms existing baseline supervised models, such as SmaAt-UNet, providing more accurate nowcasting predictions.

Figures

Figures reproduced from arXiv: 2412.15917 by the authors.

Figure 1
Figure 1. Schematic of SpaT-SparK model in pretraining and fine-tuning mode. (a) In pretraining, only the encoder, the decoder, [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Performance of the models at each time step. SparK and [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 28 canonical work pages

  1. [1]

    Y . Wang, E. Coning, A. Harou, W. Jacobs, P. Joe, L. Nikitina, R. Roberts, J. Wang, J. Wilson, A. Atencia, B. Bica, B. Brown, S. Goodmann, A. Kann, P. W. Li, I. Monterio, F. Schmid, A. Seed, and J. Sun, Guidelines for Nowcasting Techniques , 11 2017

    work page 2017

  2. [2]

    Spatial and temporal evaluation of radar rainfall nowcasting techniques on 1,533 events,

    R. O. Imhoff, C. C. Brauer, A. Overeem, A. H. Weerts, and R. Ui- jlenhoet, “Spatial and temporal evaluation of radar rainfall nowcasting techniques on 1,533 events,” Water Resources Research, vol. 56, 2020

    work page 2020

  3. [3]

    Optimizing reservoir water management in a changing climate,

    P. Bec ¸a, A. C. Rodrigues, J. P. Nunes, P. Diogo, and B. Mujtaba, “Optimizing reservoir water management in a changing climate,” Water Resources Management, pp. 1–15, 2023

    work page 2023

  4. [4]

    Development and verification of a real-time stochastic precipitation nowcasting system for urban hydrology in belgium,

    L. Foresti, M. Reyniers, A. Seed, and L. Delobbe, “Development and verification of a real-time stochastic precipitation nowcasting system for urban hydrology in belgium,” Hydrology and Earth System Sciences , vol. 20, no. 1, pp. 505–527, 2016

    work page 2016

  5. [5]

    N. K. Johnson, Weather Prediction by Numerical Process . JSTOR, 1922

    work page 1922

  6. [6]

    Nowcasting of precipitation sys- tems,

    K. A. Browning and C. G. Collier, “Nowcasting of precipitation sys- tems,” Reviews of Geophysics , vol. 27, no. 3, pp. 345–370, 1989

    work page 1989

  7. [7]

    Bech and J

    J. Bech and J. L. Chau, Doppler radar observations: Weather radar, wind profiler, ionospheric radar, and other advanced applications . BoD–Books on Demand, 2012

    work page 2012

  8. [8]

    Convolutional lstm network: A machine learning approach for precipitation nowcasting,

    X. Shi, Z. Chen, H. Wang, D.-Y . Yeung, W.-K. Wong, and W.-c. Woo, “Convolutional lstm network: A machine learning approach for precipitation nowcasting,” Advances in neural information processing systems, vol. 28, 2015

    2015

Show all 48 references
  1. [9]

    Machine learning for precipitation nowcasting from radar images,

    S. Agrawal, L. Barrington, C. Bromberg, J. Burge, C. Gazen, and J. Hickey, “Machine learning for precipitation nowcasting from radar images,” arXiv preprint arXiv:1912.12132 , 2019

    1912 arXiv

  2. [10]

    SmaAt-UNet: Precip- itation now-casting using a small attention-unet architecture,

    K. Trebing, T. Stanczyk, and S. Mehrkanoon, “SmaAt-UNet: Precip- itation now-casting using a small attention-unet architecture,” Pattern Recognition Letters, vol. 145, pp. 178–186, May 2021

    2021

  3. [11]

    Skillful precipitation nowcasting using deep generative models of radar,

    S. Ravuri, K. Lenc, M. Willson, D. Kangin, R. Lam, P. Mirowski, M. Fitzsimons, M. Athanassiadou, S. Kashem, S. Madge, R. Prudden, A. Mandhane, A. Clark, A. Brock, K. Simonyan, R. Hadsell, N. Robin- son, E. Clancy, A. Arribas, and S. Mohamed, “Skillful precipitation nowcasting ...

    2021

  4. [12]

    Deep shared representation learning for weather ele- ments forecasting,

    S. Mehrkanoon, “Deep shared representation learning for weather ele- ments forecasting,” Knowledge-Based Systems , vol. 179, pp. 120–128, 2019

    2019

  5. [13]

    Wind speed prediction using multi- dimensional convolutional neural networks,

    K. Trebing and S. Mehrkanoon, “Wind speed prediction using multi- dimensional convolutional neural networks,” in 2020 IEEE symposium series on computational intelligence (SSCI) . IEEE, 2020, pp. 713–720

    2020

  6. [14]

    Revisiting self-supervised visual representation learning,

    A. Kolesnikov, X. Zhai, and L. Beyer, “Revisiting self-supervised visual representation learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019, pp. 1920–1929

    2019

  7. [15]

    Uncovering the structure of clinical eeg signals with self- supervised learning,

    H. Banville, O. Chehab, A. Hyv ¨arinen, D.-A. Engemann, and A. Gram- fort, “Uncovering the structure of clinical eeg signals with self- supervised learning,” Journal of Neural Engineering , vol. 18, no. 4, p. 046020, 2021

    2021

  8. [16]

    A novel dual-stream time-frequency contrastive pretext tasks framework for sleep stage classification,

    S. Kazatzidis and S. Mehrkanoon, “A novel dual-stream time-frequency contrastive pretext tasks framework for sleep stage classification,” in 2024 International Joint Conference on Neural Networks (IJCNN) . IEEE, 2024, pp. 1–8

    2024

  9. [17]

    Prottrans: Toward understanding the language of life through self-supervised learning,

    A. Elnaggar, M. Heinzinger, C. Dallago, G. Rehawi, Y . Wang, L. Jones, T. Gibbs, T. Feher, C. Angerer, M. Steinegger et al., “Prottrans: Toward understanding the language of life through self-supervised learning,” IEEE transactions on pattern analysis and machine intelligence,...

    2021

  10. [18]

    Big self-supervised models are strong semi-supervised learners,

    T. Chen, S. Kornblith, K. Swersky, M. Norouzi, and G. E. Hinton, “Big self-supervised models are strong semi-supervised learners,” Advances in neural information processing systems , vol. 33, pp. 22 243–22 255, 2020

    2020

  11. [19]

    Using self- supervised learning can improve model robustness and uncertainty,

    D. Hendrycks, M. Mazeika, S. Kadavath, and D. Song, “Using self- supervised learning can improve model robustness and uncertainty,” Advances in neural information processing systems , vol. 32, 2019

    2019

  12. [20]

    Context encoders: Feature learning by inpainting,

    D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, and A. A. Efros, “Context encoders: Feature learning by inpainting,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 2536–2544

    2016

  13. [21]

    Masked au- toencoders are scalable vision learners,

    K. He, X. Chen, S. Xie, Y . Li, P. Doll ´ar, and R. Girshick, “Masked au- toencoders are scalable vision learners,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 16 000–16 009

    2022

  14. [22]

    Siamese masked autoen- coders,

    A. Gupta, J. Wu, J. Deng, and F.-F. Li, “Siamese masked autoen- coders,” Advances in Neural Information Processing Systems , vol. 36, pp. 40 676–40 693, 2023

    2023

  15. [23]

    Multimae: Multi- modal multi-task masked autoencoders,

    R. Bachmann, D. Mizrahi, A. Atanov, and A. Zamir, “Multimae: Multi- modal multi-task masked autoencoders,” in European Conference on Computer Vision. Springer, 2022, pp. 348–367

    2022

  16. [24]

    Designing bert for convolutional networks: Sparse and hierarchical masked mod- eling,

    K. Tian, Y . Jiang, Q. Diao, C. Lin, L. Wang, and Z. Yuan, “Designing bert for convolutional networks: Sparse and hierarchical masked mod- eling,” arXiv:2301.03580, 2023

    2023 arXiv

  17. [25]

    U-net: Convolutional networks for biomedical image segmentation,

    O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in International Conference on Medical image computing and computer-assisted intervention. Springer, 2015, pp. 234–241

    2015

  18. [26]

    AA-TransUnet: Attention augmented transunet for nowcasting tasks,

    Y . Yang and S. Mehrkanoon, “AA-TransUnet: Attention augmented transunet for nowcasting tasks,” in International Joint Conference on Neural Networks (IJCNN) . IEEE, 2022, pp. 01–08

    2022

  19. [27]

    SAR-UNet: Small attention residual unet for explainable nowcasting tasks,

    M. Renault and S. Mehrkanoon, “SAR-UNet: Small attention residual unet for explainable nowcasting tasks,” inInternational Joint Conference on Neural Networks (IJCNN) . IEEE, 2023, pp. 1–8

    2023

  20. [28]

    Broad-UNet: Multi-scale feature learning for nowcasting tasks,

    J. G. Fern ´andez and S. Mehrkanoon, “Broad-UNet: Multi-scale feature learning for nowcasting tasks,” Neural Networks, vol. 144, pp. 419–427, 2021

    2021

  21. [29]

    Xception: Deep learning with depthwise separable convolu- tions,

    F. Chollet, “Xception: Deep learning with depthwise separable convolu- tions,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1251–1258

    2017

  22. [30]

    Cbam: Convolutional block attention module,

    S. Woo, J. Park, J.-Y . Lee, and I. S. Kweon, “Cbam: Convolutional block attention module,” in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 3–19

    2018

  23. [31]

    GA-SmaAt-GNET: Generative adversarial small attention gnet for extreme precipitation nowcasting,

    E. Reulen, J. Shi, and S. Mehrkanoon, “GA-SmaAt-GNET: Generative adversarial small attention gnet for extreme precipitation nowcasting,” Knowledge-Based Systems, vol. 305, p. 112612, 2024

    2024

  24. [32]

    Graph dual-stream convolutional attention fusion for precipitation nowcasting,

    L. Vatam ´any and S. Mehrkanoon, “Graph dual-stream convolutional attention fusion for precipitation nowcasting,” Engineering Applications of Artificial Intelligence , vol. 141, p. 109788, 2025

    2025

  25. [33]

    Learning skillful medium-range global weather forecasting,

    R. Lam, A. Sanchez-Gonzalez, M. Willson, P. Wirnsberger, M. For- tunato, F. Alet, S. Ravuri, T. Ewalds, Z. Eaton-Rosen, W. Hu et al. , “Learning skillful medium-range global weather forecasting,” Science, vol. 382, no. 6677, pp. 1416–1421, 2023

    2023

  26. [34]

    Accurate medium-range global weather forecasting with 3d neural networks,

    K. Bi, L. Xie, H. Zhang, X. Chen, X. Gu, and Q. Tian, “Accurate medium-range global weather forecasting with 3d neural networks,” Nature, vol. 619, no. 7970, pp. 533–538, 2023

    2023

  27. [35]

    Fourcastnet: Accelerating global high-resolution weather forecasting using adaptive fourier neural operators,

    T. Kurth, S. Subramanian, P. Harrington, J. Pathak, M. Mardani, D. Hall, A. Miele, K. Kashinath, and A. Anandkumar, “Fourcastnet: Accelerating global high-resolution weather forecasting using adaptive fourier neural operators,” in Proceedings of the platform for advanced scien...

    2023

  28. [36]

    Self- supervised Learning: Generative or Contrastive,

    X. Liu, F. Zhang, Z. Hou, Z. Wang, L. Mian, J. Zhang, and J. Tang, “Self- supervised Learning: Generative or Contrastive,” IEEE Transactions on Knowledge and Data Engineering , pp. 1–1, 2021, arXiv:2006.08218 [cs, stat]. [Online]. Available: http://arxiv.org/abs/2006.08218

    2021 arXiv

  29. [37]

    A survey on self-supervised learning: Algorithms, applications, and future trends,

    J. Gui, T. Chen, J. Zhang, Q. Cao, Z. Sun, H. Luo, and D. Tao, “A survey on self-supervised learning: Algorithms, applications, and future trends,” arXiv preprint arXiv:2301.05712 , 2023

    2023 arXiv

  30. [38]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, Oct 2018

    2018 arXiv

  31. [39]

    Roberta: A robustly optimized bert pretraining approach,

    Y . Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V . Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” arXiv preprint arXiv:1907.11692 , 2019

    1907 arXiv

  32. [40]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929 , 2020

    2010 arXiv

  33. [41]

    W-mae: Pre-trained weather model with masked autoencoder for multi-variable weather forecasting,

    X. Man, C. Zhang, C. Li, and J. Shao, “W-mae: Pre-trained weather model with masked autoencoder for multi-variable weather forecasting,” arXiv preprint arXiv:2304.08754 , Apr 2023

    2023 arXiv

  34. [42]

    The era5 global reanalysis,

    H. Hersbach, B. Bell, P. Berrisford, S. Hirahara, A. Hor ´anyi, J. Mu ˜noz- Sabater, J. Nicolas, C. Peubey, R. Radu, D. Schepers et al. , “The era5 global reanalysis,” Quarterly Journal of the Royal Meteorological Society, vol. 146, no. 730, pp. 1999–2049, 2020

    1999

  35. [43]

    Simvp: Towards simple yet powerful spatiotemporal predictive learning,

    C. Tan, Z. Gao, S. Li, and S. Z. Li, “Simvp: Towards simple yet powerful spatiotemporal predictive learning,” arXiv preprint arXiv:2211.12509 , 2022

    2022 arXiv

  36. [44]

    Openstl: A comprehensive benchmark of spatio-temporal predictive learning,

    C. Tan, S. Li, Z. Gao, W. Guan, Z. Wang, Z. Liu, L. Wu, and S. Z. Li, “Openstl: A comprehensive benchmark of spatio-temporal predictive learning,” Advances in Neural Information Processing Systems , vol. 36, pp. 69 819–69 831, 2023

    2023

  37. [45]

    Masked feature prediction for self-supervised visual pre-training,

    C. Wei, H. Fan, S. Xie, C.-Y . Wu, A. Yuille, and C. Feichtenhofer, “Masked feature prediction for self-supervised visual pre-training,”

  38. [46]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778

    2016

  39. [47]

    Large batch optimiza- tion for deep learning: Training bert in 76 minutes,

    Y . You, J. Li, S. Reddi, J. Hseu, S. Kumar, S. Bhojanapalli, X. Song, J. Demmel, K. Keutzer, and C.-J. Hsieh, “Large batch optimiza- tion for deep learning: Training bert in 76 minutes,” arXiv preprint arXiv:1904.00962, 2019

    1904 arXiv

  40. [2023]

    Available: https://arxiv.org/abs/2112.09133

    [Online]. Available: https://arxiv.org/abs/2112.09133

    arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.