Pith. sign in

REVIEW 3 major objections 5 minor 47 references

VA-MoE: Variables-Adaptive Mixture of Experts for Incremental Weather Forecasting

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Incremental training with frozen pretrained experts lets a weather forecasting model add new surface variables without retraining from scratch, and the resulting 137M-parameter model matches the fully retrained 665M-parameter model on…

desk verdict Useful incremental variable-expansion task and sensible expert-freezing design, but the efficiency claim is undermined by an unexplained 665M vs 137M parameter gap and under-reported experiment details. read the letter →

arxiv 2412.02503 v2 pith:5GP4SARB submitted 2024-12-03 cs.LG physics.ao-ph

Hao Chen , Han Tao , Guo Song , Jie Zhang , Yunlong Yu , Yonghan Dong , Lei Bai This is my paper · ORCID
classification cs.LGphysics.ao-ph
keywords incrementalweatherforecastingmixtureofexpertsvariable-adaptivegatingcatastrophicforgettingERA5parameter-efficienttrainingexpertspecialization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to establish that weather forecasting models do not need to be retrained from scratch when new observational variables arrive. It proposes VA-MoE, a transformer with a mixture of experts in which five channel-adaptive experts are first pretrained on upper-air variables (geopotential, humidity, winds, temperature) and then frozen; adding the five surface variables only trains one new expert plus the shared expert. The central empirical claim is that this incremental model, with 137M trainable parameters against 665M and a 20-year rather than 40-year dataset, matches or slightly beats the fully trained model on 500-hPa upper-air variables and stays close on surface variables, competing with strong baselines such as GraphCast. If correct, the practical consequence is that operational models could extend to new data sources, such as satellite-derived variables, at a fraction of the retraining cost currently reported for systems like Pangu-Weather.

What carries the argument

The load-bearing mechanism is the Channel-Adaptive Expert (CAE), a per-variable expert module that is selected by a gating signal derived from a one-hot variable index embedding. The index embedding is linearly projected, multiplied channel-wise with the input feature, passed through an MLP, Softmax, and TopK to produce a gate index and gate weight; the selected feature is then processed by a sparse MoE expert associated with that variable type. A shared expert processes all variables in parallel. In the incremental stage the five upper-air CAEs are frozen, a new CAE for surface variables is added, and the index embedding and its encoder are retrained to cover the expanded variable set, so the routing mechanism determines which expert receives which variable's pathway.

What would settle it

Run the method on a small grid with the losses exactly as written in Eqs. 8--10 and leave $w$ unconstrained: at perfect prediction the loss equals $w$ and the optimizer will drive $w$ to negative infinity, so a bounded training curve can only appear if some unstated constraint is active. If the released code contains such a constraint, the paper's objective as written is not the objective that was optimized.

Watch

Extended reading notes

Core claim

The central discovery is that catastrophic forgetting can be avoided in incremental weather forecasting by assigning each variable type its own expert and freezing the pretrained experts during expansion. VA-MoE first trains the full model on 65 upper-air channels; in the incremental stage it expands the encoder, decoder, position embedding, index embedding, and MoE layer to accommodate five surface variables, initializing the new parameters randomly and training only the new surface-variable expert and the shared expert. On the ERA5 benchmark, the resulting VA-MoE (IL) reaches a Z500 RMSE of $18.23\ \mathrm{m^2/s^2}$ at 6 hours, $133.14$ at 72 hours, and $292.63$ at 120 hours, slightly better than the fully trained VA-MoE ($19.28$, $134.63$, $295.52$) and close to GraphCast. The paper interprets this as evidence that the frozen-expert schedule preserves upper-air skill, and that the variable-adaptive gating, driven by one-hot index embeddings, gives each variable type its own specialized pathway without auxiliary load-balancing losses.

Load-bearing premise

The load-bearing premise is that the training objective in Eq. 8, with its learned per-channel exponent $w$, is optimized as stated; but since the paper does not state any bound, weight decay, or clipping on $w$, the loss is unbounded below even at perfect prediction, so the reported results depend on an unstated constraint on $w$.

Editorial extensions

If this is right

  • Adding surface variables incrementally does not degrade upper-air skill: VA-MoE (IL) matches or slightly improves Z500 long-term RMSE compared with its own fully trained version.
  • Surface-variable forecasts from the incremental model are competitive with models trained on all variables, and lead in long-lead U10, V10, MSL, and SP among the methods compared.
  • A model can be expanded with roughly half the training data and a quarter of the trainable parameters, making operational variable additions far cheaper than full retraining.
  • Expert specialization can be obtained from index-embedding-guided gating without auxiliary load-balancing losses, keeping the training objective simple.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct next experiment would be to apply the same frozen-expert schedule to other later-arriving variable families, such as aerosols, precipitation, or air quality, since the mechanism keys on variable type rather than on the specific variables tested.
  • The reported parameter ratio counts trainable parameters in the incremental stage; frozen experts remain in memory and in the inference graph, so the savings are in training compute and data, not necessarily in serving memory.
  • The gating design could also stage variables in multiple small batches rather than one surface-variable batch; if freezing preserves knowledge at each stage, later additions should not disturb earlier incremental variables.
  • The paper counts upper-air variables at 13 pressure levels; a similar variable-type grouping could let the index embeddings route by pressure level as well, which would be a natural and testable extension.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes VA-MoE, a mixture-of-experts transformer for incremental weather forecasting. The model is first trained on five upper-air variables (65 channels) with channel-adaptive experts and then expanded to five surface variables by adding new experts while freezing the pretrained ones. A variable index embedding and a learned per-channel weighting in the loss are introduced. Experiments on ERA5 compare VA-MoE and its incremental variant with published models, reporting competitive RMSE on surface and upper-air variables with fewer parameters and less training data.

Significance. The problem of adding new meteorological variables without full retraining is timely and practically relevant, and the paper proposes a concrete two-stage design with a public code link. If the efficiency and accuracy claims were properly controlled, the incremental training paradigm would be a useful contribution. The main weaknesses are that the parameter-efficiency comparison is confounded by unexplained architecture differences and that the training objective in Eq. (8) is underspecified; both need to be addressed before the headline claims can be accepted.

major comments (3)
  1. [Sec. 3.4.1, Eq. (8)] The dynamic prediction loss Obj_pred = (Xhat - X)^2 / e^w + w is not fully specified. If any channel residual is exactly zero, minimizing over w drives the loss to negative infinity, and the paper states no bound, clipping, or regularization on w. For nonzero residuals, the unconstrained optimum satisfies e^w equal to the per-channel squared error, so the loss reduces to a per-channel log-scale term plus 1, which does not implement the 'dynamic prioritization' described in the text. The authors should either constrain w, present Eq. (8) as a Gaussian negative log-likelihood with learned per-channel variance and discuss the zero-residual edge case, or report the exact implementation used in the experiments. As written, the training procedure is not well-defined or reproducible.
  2. [Sec. 4, Tables 2 and 3; Sec. 3.3] The claimed 25%-parameter advantage is confounded. VA-MoE (full) has 665M parameters while VA-MoE (IL) has 137M, but the manuscript never gives the architecture hyperparameters (depth, width, number of experts, CAE channel counts, or similarities/differences in the backbone) for either model. Adding one new CAE for surface variables to five frozen CAEs should increase parameters only modestly, so a 4.85x gap cannot be explained by the incremental protocol. This makes the comparison in Tables 2 and 3 one of two different model families rather than an ablation of incremental training. Please specify all architecture details and include an equal-capacity full-retraining baseline, or explicitly justify the capacity difference.
  3. [Sec. 4, Tables 2-4] All quantitative results are reported from single runs with no standard deviations, number of seeds, or statistical significance tests. Since the central claims are about 'comparable' and 'superior' performance, the lack of uncertainty estimates makes it impossible to assess whether differences such as the Z500 RMSE values in Table 3 are meaningful. Report mean and variance over at least three seeds for the proposed methods and, where practical, for the baselines.
minor comments (5)
  1. [Abstract and Sec. 4] The abstract mentions short-term forecasting of 1-3 days, while the experiments and Fig. 4 show results up to 120 hours (5 days); please make the time horizons consistent.
  2. [Sec. 3.2.1] The dimensions of the index embedding are stated inconsistently: I_h is described as R^{5 x N} while I_Z is R^{1 x 1 x N}, and N is used for both the number of channels and the number of variables. Please clarify the notation.
  3. [Sec. 3.4.1] Equation (8) writes division by e^w, and the text says 'e denotes the base of the natural logarithm'; this is confusing because e^w can be misread as a constant base raised to a vector. Use exp(w) or clarify element-wise exponentiation.
  4. [Sec. 4, Table 2] The column header 'Iteration (years) (x10^4)' mixes dataset years and iteration counts; please define the unit precisely and use consistent labels.
  5. [Sec. 4, Implementation Details] The implementation details state that the structure follows backbones [7, 8, 12], but references [7] and [8] are few-shot segmentation papers and [12] is FlashAttention; these do not appear to be weather-forecasting backbones, so the citation is likely incorrect and should be fixed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the incremental-training results are empirical comparisons against external baselines and the paper's own full-retraining baseline.

full rationale

The paper's load-bearing claims are empirical: VA-MoE (IL) matches or approaches VA-MoE full and external models on ERA5 (Sec. 4, Tables 2-4). These are measurements against fixed ground-truth test data, not outputs derived from fitted constants. The learnable vector w in Eq. 8 is part of the training objective and is optimized, not calibrated to make benchmark numbers; no predicted metric is defined as a function of w after training. The only author self-citations (refs. 6, 7, 8 for backbones and ref. 39 as a competitor) are peripheral and do not supply the central efficiency/accuracy result, which is benchmarked against GraphCast, Pangu, Stormer, etc. The unexplained parameter-count difference between VA-MoE (665M) and VA-MoE (IL) (137M) is a potential confound or reporting gap, not a circular reduction. No equation in the paper defines its output in terms of its input by construction, and no 'prediction' is a renamed fit.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The principal free parameters are loss and architecture choices not reported in the paper. One ad hoc assumption, the well-posedness of Eq. 8, is load-bearing because the training procedure as described would otherwise be degenerate. The variable index embedding is a learned mechanism rather than a new physical entity, and it is listed here because the paper presents it as a novel component with no direct evidence of its internal mechanism.

free parameters (4)
  • channel loss weight w = not reported
    Eq. 8 introduces a per-channel learnable vector w that gates the prediction loss. Its fitted value is not reported, and no bound or regularizer is stated, which is load-bearing for the loss to be well-posed.
  • reconstruction loss weight lambda = not reported
    Eq. 10 combines prediction and reconstruction losses with a scalar lambda; the paper never specifies its value or how it was chosen.
  • gate Top-K = not reported
    Eq. 3 uses a Top-K selection over channel index weights. K is a hand-chosen integer that controls expert capacity and is not reported.
  • architecture sizes (experts, hidden dims, layers) = not reported
    The 137M and 665M parameter counts depend on unstated model widths, number of CAE experts, expert hidden sizes, and transformer depth. These are chosen by the authors and not specified in the text.
assumptions (4)
  • domain assumption The incremental variable availability scenario (upper-air first, surface later) matches operational reality.
    Sec. 1 and Sec. 3.1 assume variables arrive sequentially. In ERA5, all variables are available simultaneously, so the staged scenario is imposed by the experimental design rather than measured from real asynchronous data streams.
  • domain assumption Freezing pretrained experts preserves prior-task accuracy and avoids catastrophic forgetting.
    Sec. 3.3.2 freezes all initial CAEs during incremental training. The paper tests this for one variable split and reports no forgetting, but does not justify freezing as sufficient for arbitrary new variable types or larger expansions.
  • ad hoc to paper The loss in Eq. 8 has a finite optimum under the described training procedure.
    For a perfect prediction, Eq. 8 reduces to w, which is unbounded below as w tends to negative infinity. The paper does not state a bound, weight decay, or clipping on w, so the objective as written is not mathematically well-posed.
  • domain assumption Transformer blocks with the proposed MoE have sufficient capacity to learn global weather dynamics from 65 input channels at 0.25 degrees.
    The entire approach is an empirical model-class assumption. There is no proof of capacity or convergence; the evidence is restricted to the reported experimental runs on ERA5.
invented entities (1)
  • Variable index embedding
    purpose: One-hot per-variable encodings that are transformed by a linear layer and used to route variables to experts via top-K gating (Eq. 3).
    This is a learned input representation introduced by the paper. Its effect is only demonstrated through end-to-end accuracy; the paper does not directly measure whether the embedding causes the claimed expert specialization, so it lacks an external falsifiable handle.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VA-MoE: Variables-Adaptive Mixture of Experts for Incremental Weather Forecasting." pith.science (2026). https://pith.science/paper/5GP4SARB

@misc{pith2026241202503,
  author       = {Pith},
  title        = {Pith review of: VA-MoE: Variables-Adaptive Mixture of Experts for Incremental Weather Forecasting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5GP4SARB}},
  note         = {Machine review of arXiv:2412.02503}
}
read the original abstract

This paper presents Variables Adaptive Mixture of Experts (VAMoE), a novel framework for incremental weather forecasting that dynamically adapts to evolving spatiotemporal patterns in real time data. Traditional weather prediction models often struggle with exorbitant computational expenditure and the need to continuously update forecasts as new observations arrive. VAMoE addresses these challenges by leveraging a hybrid architecture of experts, where each expert specializes in capturing distinct subpatterns of atmospheric variables (temperature, humidity, wind speed). Moreover, the proposed method employs a variable adaptive gating mechanism to dynamically select and combine relevant experts based on the input context, enabling efficient knowledge distillation and parameter sharing. This design significantly reduces computational overhead while maintaining high forecast accuracy. Experiments on real world ERA5 dataset demonstrate that VAMoE performs comparable against SoTA models in both short term (1 days) and long term (5 days) forecasting tasks, with only about 25% of trainable parameters and 50% of the initial training data.

Figures

Figures reproduced from arXiv: 2412.02503 by the authors.

Figure 1
Figure 1. Illustration of two different training paradigms of [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the incremental weather forecasting paradigm. (a), (b) are the initial and incremental stages, respectively. (c) [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Illustration of the VA-MoE. In the left subgraph, the input features comprise both upper-air and surface variables. Dur￾ing the initial training stage, five distinct CAE modules process the upper-air variables. In the incremental stage, surface variables are handled by a dedicated module, CAESV, while the original five CAEs remain frozen to preserve learned representations. The right subgraph details the CAE module.… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Comparative analysis of RMSE ↓ across 10 data-driven models for four variables, including Z500 and T850 (upper-air variables) in the initial stage, as well as T2M and U10 (incremen￾tal surface variables) in the incremental stage. 1979 to 2020 and 2000 to 2020 year, res…
Figure 5
Figure 5. Figure 5: 6-hour global weather prediction of upper-air and surface variables forecast visualization generated by VA-MoE framework. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 38 canonical work pages

  1. [1]

    Expert gate: Lifelong learning with a network of experts

    Rahaf Aljundi, Punarjay Chakravarty, and Tinne Tuytelaars. Expert gate: Lifelong learning with a network of experts. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3366–3375, 2017. 3

    work page 2017

  2. [2]

    The quiet rev- olution of numerical weather prediction.Nature, 525(7567): 47–55, 2015

    Peter Bauer, Alan Thorpe, and Gilbert Brunet. The quiet rev- olution of numerical weather prediction.Nature, 525(7567): 47–55, 2015. 1, 2

    work page 2015

  3. [3]

    Accurate medium-range global weather forecasting with 3d neural networks.Nature, 619(7970): 533–538, 2023

    Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiao- tao Gu, and Qi Tian. Accurate medium-range global weather forecasting with 3d neural networks.Nature, 619(7970): 533–538, 2023. 1, 2, 6, 7

    work page 2023

  4. [4]

    Aurora: A foundation model of the atmosphere

    Cristian Bodnar, Wessel P Bruinsma, Ana Lucic, Megan Stanley, Johannes Brandstetter, Patrick Garvan, Maik Riechert, Jonathan Weyn, Haiyu Dong, Anna Vaughan, et al. Aurora: A foundation model of the atmosphere. arXiv:2405.13063, 2024. 1

    arXiv 2024

  5. [5]

    Spherical fourier neural operators: learning stable dynamics on the sphere

    Boris Bonev, Thorsten Kurth, Christian Hundt, Jaideep Pathak, Maximilian Baust, Karthik Kashinath, and Anima Anandkumar. Spherical fourier neural operators: learning stable dynamics on the sphere. InProceedings of Interna- tional Conference on Machine Learning, 2023. 2

    work page 2023

  6. [6]

    Self-prompting perceptual edge learning for dense prediction.IEEE Transactions on Circuits and Systems for Video Technology, 34(6):4528–4541, 2023

    Hao Chen, Yonghan Dong, Zhe-Ming Lu, Yunlong Yu, and Jungong Han. Self-prompting perceptual edge learning for dense prediction.IEEE Transactions on Circuits and Systems for Video Technology, 34(6):4528–4541, 2023. 2

    work page 2023

  7. [7]

    Pixel matching network for cross-domain few- shot segmentation

    Hao Chen, Yonghan Dong, Zheming Lu, Yunlong Yu, and Jungong Han. Pixel matching network for cross-domain few- shot segmentation. InProceedings of the IEEE Winter Con- ference on Applications of Computer Vision, pages 978–987,

    work page

  8. [8]

    Multi-content interaction net- work for few-shot segmentation.ACM Transactions on Mul- timedia Computing, Communications and Applications, 20 (6):1–20, 2024

    Hao Chen, Yunlong Yu, Yonghan Dong, Zheming Lu, Ying- ming Li, and Zhongfei Zhang. Multi-content interaction net- work for few-shot segmentation.ACM Transactions on Mul- timedia Computing, Communications and Applications, 20 (6):1–20, 2024. 6

    work page 2024

Show all 47 references
  1. [9]

    Fengwu: Pushing the skillful global medium-range weather forecast beyond 10 days lead.arXiv:2304.02948,

    Kang Chen, Tao Han, Junchao Gong, Lei Bai, Fenghua Ling, Jing-Jia Luo, Xi Chen, Leiming Ma, Tianning Zhang, Rui Su, et al. Fengwu: Pushing the skillful global medium-range weather forecast beyond 10 days lead.arXiv:2304.02948,

  2. [10]

    Fuxi: A cascade ma- chine learning forecasting system for 15-day global weather forecast.npj Climate and Atmospheric Science, 6(1):190,

    Lei Chen, Xiaohui Zhong, Feng Zhang, Yuan Cheng, Yinghui Xu, Yuan Qi, and Hao Li. Fuxi: A cascade ma- chine learning forecasting system for 15-day global weather forecast.npj Climate and Atmospheric Science, 6(1):190,

  3. [11]

    Lifelong language pretraining with distribution-specialized experts

    Wuyang Chen, Yanqi Zhou, Nan Du, Yanping Huang, James Laudon, Zhifeng Chen, and Claire Cui. Lifelong language pretraining with distribution-specialized experts. InInterna- tional Conference on Machine Learning, pages 5383–5395. PMLR, 2023. 3

    2023

  4. [12]

    Flashattention: Fast and memory-efficient exact attention with io-awareness

    Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christo- pher R ´e. Flashattention: Fast and memory-efficient exact attention with io-awareness. InAdvances in Neural Informa- tion Processing Systems, pages 16344–16359, 2022. 6

    2022

  5. [13]

    An image is worth 16x16 words: Trans- formers for image recognition at scale.arXiv:2010.11929,

    Alexey Dosovitskiy. An image is worth 16x16 words: Trans- formers for image recognition at scale.arXiv:2010.11929,

    2010 arXiv

  6. [14]

    Ew- moe: An effective model for global weather forecasting with mixture-of-experts.arXiv:2405.06004, 2024

    Lihao Gan, Xin Man, Chenghong Zhang, and Jie Shao. Ew- moe: An effective model for global weather forecasting with mixture-of-experts.arXiv:2405.06004, 2024. 6, 7

    2024 arXiv

  7. [15]

    Oneforecast: A universal framework for global and regional weather forecasting.arXiv:2502.00338, 2025

    Yuan Gao, Hao Wu, Ruiqi Shu, Huanshuo Dong, Fan Xu, Rui Chen, Yibo Yan, Qingsong Wen, Xuming Hu, Kun Wang, et al. Oneforecast: A universal framework for global and regional weather forecasting.arXiv:2502.00338, 2025. 2

    2025

  8. [16]

    Resurrecting old classes with new data for exemplar- free continual learning

    Dipam Goswami, Albin Soutif-Cormerais, Yuyang Liu, Sandesh Kamath, Bart Twardowski, Joost van de Weijer, et al. Resurrecting old classes with new data for exemplar- free continual learning. InProceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition, page...

    2024

  9. [17]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016. 2

    2016

  10. [18]

    The era5 global reanalysis.Quarterly Journal of the Royal Mete- orological Society, 146(730):1999–2049, 2020

    Hans Hersbach, Bill Bell, Paul Berrisford, Shoji Hirahara, Andr´as Hor ´anyi, Joaqu ´ın Mu ˜noz-Sabater, Julien Nicolas, Carole Peubey, Raluca Radu, Dinand Schepers, et al. The era5 global reanalysis.Quarterly Journal of the Royal Mete- orological Society, 146(730):1999–2049, 2020. 6

    1999

  11. [19]

    Adaptive mixtures of local experts.Neu- ral computation, 3(1):79–87, 1991

    Robert A Jacobs, Michael I Jordan, Steven J Nowlan, and Geoffrey E Hinton. Adaptive mixtures of local experts.Neu- ral computation, 3(1):79–87, 1991. 3

    1991

  12. [20]

    Forecasting global weather with graph neural networks.arXiv:2202.07575, 2022

    Ryan Keisler. Forecasting global weather with graph neural networks.arXiv:2202.07575, 2022. 6

    2022 arXiv

  13. [21]

    Eclipse: Efficient continual learning in panoptic segmenta- tion with visual prompt tuning

    Beomyoung Kim, Joonsang Yu, and Sung Ju Hwang. Eclipse: Efficient continual learning in panoptic segmenta- tion with visual prompt tuning. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3346–3356, 2024. 3

    2024

  14. [22]

    Four- castnet: Accelerating global high-resolution weather fore- casting using adaptive fourier neural operators

    Thorsten Kurth, Shashank Subramanian, Peter Harring- ton, Jaideep Pathak, Morteza Mardani, David Hall, Andrea Miele, Karthik Kashinath, and Anima Anandkumar. Four- castnet: Accelerating global high-resolution weather fore- casting using adaptive fourier neural operators. InPro...

    2023

  15. [23]

    Learning skillful medium-range global weather forecasting

    Remi Lam, Alvaro Sanchez-Gonzalez, Matthew Willson, Peter Wirnsberger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Eaton-Rosen, Weihua Hu, et al. Learning skillful medium-range global weather forecasting. Science, 382(6677):1416–1421, 2023. 2, 6, 7

    2023

  16. [24]

    Fourier neural operator for paramet- ric partial differential equations.arXiv:2010.08895, 2020

    Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for paramet- ric partial differential equations.arXiv:2010.08895, 2020. 2

    2010 arXiv

  17. [25]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE International Conference on Com- puter Vision, pages 10012–10022, 2021. 2

    2021

  18. [26]

    The ecmwf ensemble prediction sys- tem: Methodology and validation.Quarterly journal of the royal meteorological society, 122(529):73–119, 1996

    Franco Molteni, Roberto Buizza, Tim N Palmer, and Thomas Petroliagis. The ecmwf ensemble prediction sys- tem: Methodology and validation.Quarterly journal of the royal meteorological society, 122(529):73–119, 1996. 2

    1996

  19. [27]

    Gupta, and Aditya Grover

    Tung Nguyen, Johannes Brandstetter, Ashish Kapoor, Jayesh K. Gupta, and Aditya Grover. Climax: A foundation model for weather and climate. InInternational Conference on Machine Learning, 2023. 6, 7

    2023

  20. [28]

    Scaling transformer neu- ral networks for skillful and reliable medium-range weather forecasting

    Tung Nguyen, Rohan Shah, Hritik Bansal, Troy Arco- mano, Romit Maulik, Rao Kotamarthi, Ian Foster, Sandeep Madireddy, and Aditya Grover. Scaling transformer neu- ral networks for skillful and reliable medium-range weather forecasting. InAdvances in Neural Information Processin...

    2025

  21. [29]

    Adaptive vio: Deep visual-inertial odometry with online continual learning.arXiv:2405.16754, 2024

    Youqi Pan, Wugen Zhou, Yingdian Cao, and Hongbin Zha. Adaptive vio: Deep visual-inertial odometry with online continual learning.arXiv:2405.16754, 2024. 3

    2024 arXiv

  22. [30]

    Switch diffusion trans- former: Synergizing denoising tasks with sparse mixture-of- experts

    Byeongjun Park, Hyojun Go, Jin-Young Kim, Sangmin Woo, Seokil Ham, and Changick Kim. Switch diffusion trans- former: Synergizing denoising tasks with sparse mixture-of- experts. InEuropean Conference on Computer Vision, 2024. 8

    2024

  23. [31]

    Probabilistic weather forecasting with machine learn- ing.Nature, 637(8044):84–90, 2025

    Ilan Price, Alvaro Sanchez-Gonzalez, Ferran Alet, Tom R Andersson, Andrew El-Kadi, Dominic Masters, Timo Ewalds, Jacklynn Stott, Shakir Mohamed, Peter Battaglia, et al. Probabilistic weather forecasting with machine learn- ing.Nature, 637(8044):84–90, 2025. 2

    2025

  24. [32]

    The thorpex interactive grand global ensem- ble (tigge)

    D Richardson. The thorpex interactive grand global ensem- ble (tigge). InGeophysical Research Abstracts, page 02815,

  25. [33]

    Implementation of the semi-lagrangian method in a high- resolution version of the ecmwf forecast model.Monthly Weather Review, 123(2):489–514, 1995

    Harold Ritchie, Clive Temperton, Adrian Simmons, Mari- ano Hortal, Terry Davies, David Dent, and Mats Hamrud. Implementation of the semi-lagrangian method in a high- resolution version of the ecmwf forecast model.Monthly Weather Review, 123(2):489–514, 1995. 2

    1995

  26. [34]

    Learning equi-angular represen- tations for online continual learning

    Minhyuk Seo, Hyunseo Koh, Wonje Jeung, Minjae Lee, San Kim, Hankook Lee, Sungjun Cho, Sungik Choi, Hyunwoo Kim, and Jonghyun Choi. Learning equi-angular represen- tations for online continual learning. InProceedings of the IEEE Conference on Computer Vision and Pattern Recogni...

    2024

  27. [35]

    Semantically-shifted incremental adapter-tuning is a continual vitransformer

    Yuwen Tan, Qinhao Zhou, Xiang Xiang, Ke Wang, Yuchuan Wu, and Yongbin Li. Semantically-shifted incremental adapter-tuning is a continual vitransformer. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 23252–23262, 2024. 3

    2024

  28. [36]

    ClimODE: Climate forecasting with physics-informed neu- ral ODEs

    Yogesh Verma, Markus Heinonen, and Vikas Garg. ClimODE: Climate forecasting with physics-informed neu- ral ODEs. InInternational Conference on Learning Repre- sentations, 2024. 6

    2024

  29. [37]

    Omni-smola: Boosting generalist multimodal models with soft mixture of low-rank experts

    Jialin Wu, Xia Hu, Yaqing Wang, Bo Pang, and Radu Sori- cut. Omni-smola: Boosting generalist multimodal models with soft mixture of low-rank experts. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14205–14215, 2024. 3

    2024

  30. [38]

    Koopmanlab: machine learning for solving complex physics equations.APL Machine Learning, 1(3), 2023

    Wei Xiong, Muyuan Ma, Xiaomeng Huang, Ziyang Zhang, Pei Sun, and Yang Tian. Koopmanlab: machine learning for solving complex physics equations.APL Machine Learning, 1(3), 2023. 2

    2023

  31. [39]

    Generalizing weather forecast to fine-grained temporal scales via physics-ai hybrid modeling

    Wanghan Xu, Fenghua Ling, Wenlong Zhang, Tao Han, Hao Chen, Wanli Ouyang, and Lei Bai. Generalizing weather forecast to fine-grained temporal scales via physics-ai hybrid modeling. InAdvances in Neural Information Processing Systems, 2024. 6

    2024

  32. [40]

    Orchestrate latent expertise: Advancing online contin- ual learning with multi-level supervision and reverse self- distillation

    Hongwei Yan, Liyuan Wang, Kaisheng Ma, and Yi Zhong. Orchestrate latent expertise: Advancing online contin- ual learning with multi-level supervision and reverse self- distillation. InProceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition, pages 23670–23680,

  33. [41]

    Multi-task dense prediction via mixture of low-rank experts

    Yuqi Yang, Peng-Tao Jiang, Qibin Hou, Hao Zhang, Jinwei Chen, and Bo Li. Multi-task dense prediction via mixture of low-rank experts. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 27927–27937, 2024. 3

    2024

  34. [42]

    Online task-free continual gener- ative and discriminative learning via dynamic cluster mem- ory

    Fei Ye and Adrian G Bors. Online task-free continual gener- ative and discriminative learning via dynamic cluster mem- ory. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 26202–26212, 2024. 3

    2024

  35. [43]

    Probabilistic weather forecasting with de- terministic guidance-based diffusion model

    Donggeun Yoon, Minseok Seo, Doyi Kim, Yeji Choi, and Donghyeon Cho. Probabilistic weather forecasting with de- terministic guidance-based diffusion model. InEuropean Conference on Computer Vision, pages 108–124, 2024. 2

    2024

  36. [44]

    Diffcast: A uni- fied framework via residual diffusion for precipitation now- casting

    Demin Yu, Xutao Li, Yunming Ye, Baoquan Zhang, Chuyao Luo, Kuai Dai, Rui Wang, and Xunlai Chen. Diffcast: A uni- fied framework via residual diffusion for precipitation now- casting. InProceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition, pages 27758–27767,

  37. [45]

    Boosting continual learning of vision-language models via mixture-of-experts adapters

    Jiazuo Yu, Yunzhi Zhuge, Lu Zhang, Ping Hu, Dong Wang, Huchuan Lu, and You He. Boosting continual learning of vision-language models via mixture-of-experts adapters. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 23219–23230, 2024. 3

    2024

  38. [46]

    Skilful nowcasting of extreme precipitation with nowcastnet

    Yuchen Zhang, Mingsheng Long, Kaiyuan Chen, Lanxiang Xing, Ronghua Jin, Michael I Jordan, and Jianmin Wang. Skilful nowcasting of extreme precipitation with nowcastnet. Nature, 619(7970):526–532, 2023. 2

    2023

  39. [47]

    Rcl: Reliable continual learning for uni- fied failure detection

    Fei Zhu, Zhen Cheng, Xu-Yao Zhang, Cheng-Lin Liu, and Zhaoxiang Zhang. Rcl: Reliable continual learning for uni- fied failure detection. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 12140– 12150, 2024. 3

    2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.