Pith. sign in

REVIEW 4 major objections 5 minor 53 references

ORBIT-2: Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a residual, tile-wise vision transformer can make hyper-resolution climate downscaling tractable at exascale scale, cutting self-attention cost from quadratic to linear and reaching 0.9 km global resolution with high…

desk verdict Real HPC scaling achievement with an inflated token-count headline and circular validation; worth refereeing, but the scientific claims need major corrections before they can be taken at face value. read the letter →

arxiv 2505.04802 v2 pith:YHHZ7E3P submitted 2025-05-07 cs.LG astro-ph.EPcs.AIcs.DCphysics.ao-ph

Xiao Wang , Jong-Youl Choi , Takuya Kurihaya , Isaac Lyngaas , Hong-Jun Yoon , Xi Xiao , David Pugmire , Ming Fan
show 10 more authors
Nasik M. Nafi Aristeidis Tsaris Ashwin M. Aji Maliha Hossain Mohamed Wahib Dali Wang Peter Thornton Prasanna Balaprakash Moetasim Ashfaq Dan Lu
This is my paper · ORCID
classification cs.LGastro-ph.EPcs.AIcs.DCphysics.ao-ph
keywords climatedownscalingvisiontransformersparseattentionlinearcomplexityfoundationmodelexascalecomputingBayesianregularizationtile-wisescaling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ORBIT-2 attempts to show that a single vision-transformer foundation model can downscale global climate fields from coarse reanalysis to hyper-resolution (0.9 km) by making self-attention scale linearly rather than quadratically with image size. The paper claims that its tile-wise attention (TILES) plus a residual 'slim' transformer (Reslim) that trains directly on low-resolution inputs preserves accuracy while removing the long-sequence bottleneck, and that the resulting system trains at up to 4.1 exaFLOPS sustained throughput on 65,536 GPUs with 74–98% strong scaling efficiency. If correct, this would make kilometer-scale global downscaling, and the fine-scale extremes it can resolve, computationally practical for the first time. The accuracy evidence at 7 km (R² scores of 0.98–0.99, and 0.999 for U.S. minimum temperature, against observations) is presented as supporting the method's scientific value.

What carries the argument

The two load-bearing mechanisms are TILES, which converts global self-attention into independent per-tile attention with halo overlap so that complexity becomes linear in the number of tokens, and Reslim, a residual network that trains the main ViT path on low-resolution or adaptively compressed tokens while a cheap convolutional path supplies the upsampled baseline, with a Bayesian total-variation loss enforcing spatial smoothness. Their joint effect is to shrink the effective sequence length per GPU (the paper cites a 60× reduction from skipping upsampling, 18× from channel aggregation, 16× from tiling, and 4× from compression in one configuration) while keeping the model's ability to generalize across variables and resolutions.

What would settle it

Compare ORBIT-2 against a global-attention (non-tiled) ViT of the same size on a downscaling task where the underlying field has known long-range spatial correlation, such as daily precipitation over the tropical Pacific tied to ENSO or frontal precipitation crossing a tile boundary, and measure whether tile-boundary stitching introduces discontinuities or whether R² on extreme quantiles drops. A decisive check is to compute the spatial correlation length of the coarse-to-fine mapping residuals; if it exceeds the tile size, the locality assumption is violated.

Watch

Extended reading notes

Core claim

The paper claims that the two bottlenecks of AI downscaling—quadratic self-attention cost and the ill-posed inverse problem of reconstructing fine detail from coarse input—can be removed without sacrificing accuracy. TILES partitions the input into overlapping tiles, restricts self-attention to each tile, and stitches the outputs after discarding halos, reducing complexity from quadratic to linear. Reslim avoids input upsampling in the main ViT path and instead moves it to a linear-cost residual convolution path, while a Bayesian estimation objective with a total-variation prior and an adaptive quadtree compression further cut token count and constrain uncertainty. On this basis, the authors report the longest ViT sequences (up to 4.2 billion tokens), the largest downscaling models (10 billion parameters), and high accuracy at 7 km resolution, and argue that their approach generalizes across variables and geographies.

Load-bearing premise

The load-bearing premise is that each high-resolution pixel's value can be predicted from its immediate coarse-resolution neighbourhood alone, so dropping all long-range attention across tile boundaries does not change the downscaling answer; if distant influences (teleconnections, propagating storms, synoptic fronts) matter for a variable or region, the tile-wise model will lose skill.

Editorial extensions

If this is right

  • Global, hyper-resolution climate downscaling becomes feasible on current exascale machines; the paper demonstrates 0.9 km output from a 9.5M-parameter model and 2.3 km from a 10B-parameter model.
  • Sequence length is no longer the binding constraint for vision transformers on scientific images; the reported 4.2 billion tokens exceeds prior reported ViT sequences by over four orders of magnitude.
  • A single foundation model can be fine-tuned across regions and variables (temperature, precipitation) and still reach R² above 0.97 against observations at 7 km.
  • Near-real-time inference becomes possible on a single node (4 ms per sample for the small model, 0.55 s for the 10B model), enabling edge deployment for climate services.
  • The orthogonal combination of TILES, FSDP, tensor parallelism, and data parallelism provides a blueprint for scaling other long-sequence, high-dimensional models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the point-spread locality assumption holds across variables, the same tile-and-stitch strategy transfers to other pixel-local inverse problems in remote sensing and medical imaging, where fields are also dominated by nearby sources.
  • The linear-complexity claim implies that the cost of moving from 7 km to sub-kilometer resolution is dominated by data movement and halo overhead rather than attention, so the strongest test of the method is an end-to-end training run at 0.9 km, which the paper's throughput measurements imply but do not report.
  • For variables driven by nonlocal mechanisms, such as organized convection or remote teleconnections, the locality assumption is likely to break; a testable extension would add a sparse set of global 'anchor' tokens to the tile-wise attention and measure the skill gain in monsoon or ENSO-sensitive regions.
  • The accuracy metrics (R², SSIM, RMSE) could mask systematic underprediction of extremes; reporting quantile-conditional scores and spectral fidelity, as done for U.S. minimum temperature, would sharpen the scientific evaluation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. ORBIT-2 proposes a scalable vision transformer for global climate downscaling, combining a residual slim architecture (Reslim) that avoids input upsampling and a tile-wise sequence scaling algorithm (TILES) that restricts self-attention to spatial tiles. The paper reports scaling to 10B parameters on 65,536 GPUs with up to 4.1 exaFLOPS, sequence lengths up to 4.2 billion tokens, 0.9 km resolution, and R2 scores of 0.98–0.99 against observational data.

Significance. If the reported performance is accurate, ORBIT-2 would be a notable engineering contribution: it demonstrates that tile-local attention can be combined with model parallelism to train large vision transformers efficiently on exascale machines. The strong-scaling measurements, cross-platform throughput numbers, and inference speed results are informative. The paper also honestly lists several limitations (bias inheritance, lack of conservation laws). However, two headline claims—the 'longest ViT sequence' and 'high accuracy against observations'—are currently not supported by the presented evidence, and one internal inconsistency in the speedup ablation needs clarification.

major comments (4)
  1. [Section 6, Table 3, abstract, Section 8] The paper's headline claim of processing sequences up to 4.2 billion tokens is not supported by the reported attention mechanism. In Section 6, the authors state that for the 1.1B-token configuration, 'the effective per-GPU sequence length becomes only 17,280 tokens, despite the global output representing 1.1 billion tokens.' The 4.2B figure in Table 3 is the total number of output patches in the tiled image, not the length of any sequence fed to self-attention. Since TILES restricts attention to a per-GPU tile, the actual attention sequence is at most on the order of 10^4–10^5 tokens. Consequently, the abstract's claim that ORBIT-2 'processes sequences up to 4.2 billion tokens' and Section 8's comparison to the 188K-token state of the art are misleading. This is load-bearing because the long-sequence capability is one of the four headline contributions. The token-count narrative should be corrected to refer to output patch counts or effective sequence length after tiling.
  2. [Table 1 and Section 7] The evaluation protocol for the fine-tuning tasks does not establish downscaling skill against independent observations. Table 1 lists the U.S. fine-tuning task as '[ERA5, DAYMET] → DAYMET' and the global task as '[ERA5, IMERG] → IMERG', meaning the same observational product that serves as the target is also provided as a coarse input. Under this setup, a model can achieve high R2 by learning a mapping from the coarse version of the target to its fine version, without learning true downscaling from physically independent predictors. The reported R2 values (0.98–0.99) therefore do not substantiate the abstract's claim of accuracy 'against observational data.' Please provide an evaluation where the target product is not used as an input, or at least an ablation that quantifies how much of the R2 is attributable to the coarse target input.
  3. [Table 2(a)] The 660× speedup claimed for Reslim over vanilla ViT in Table 2(a) is internally inconsistent with the reported setup. Both rows list the same sequence length (24,576 tokens) and the same conditions (no compression, no tiling). If Reslim truly operates on low-resolution inputs without upsampling, its self-attention sequence should be much shorter than the ViT's; if the comparison is at equal sequence length, the stated architectural differences (residual path and Bayesian loss) cannot plausibly produce a 660× time reduction. The time per sample for Reslim (1.1e-6 s) also seems implausibly low. Please clarify the experiment design and re-check the reported timings and speedup.
  4. [Section 4.2] The TILES algorithm rests on the assumption that downscaling is spatially local. The paper cites the point-spread effect but provides no quantitative validation that the correlation decay length is small relative to tile size, nor metrics for boundary artifacts under varying halo widths. For phenomena such as precipitation organized by synoptic systems or teleconnections, this assumption may fail. Since the linear-scaling claim is intertwined with this locality assumption, please add an experiment that varies tile size and halo width and reports boundary-region errors, or a diagnostic of whether long-range information changes the downscaling output.
minor comments (5)
  1. [Section 6, Table 3] The text says the output tensor for the 298M-token Reslim configuration is '[5760, 4520, 18]', but Table 3 lists '[5760, 11520, 18]'. This discrepancy should be resolved.
  2. [Abstract and Section 8] The abstract states R2 scores in the 'range of 0.98–0.99', while Section 8 reports '0.999 for temperature and 0.979/0.986 for precipitation'. These numbers should be made consistent.
  3. [Section 4.2] The sentence 'the correlation between pixel pairs decay rapidly when the spatial distance' is missing a word; it should read 'when the spatial distance increases' or similar.
  4. [Throughout] The 'Click Here' placeholders and 'available online' links are not appropriate for a formal publication; they should be replaced with permanent URLs or removed.
  5. [Table 2(a)] The time per sample for the Reslim 9.5M 622→156 row is listed as 1.1e-6 seconds; please verify the units or the measurement, since this is three orders of magnitude faster than the vanilla ViT's 7.3e-4 seconds despite the identical reported sequence length.

Circularity Check

2 steps flagged · score 6.0 of 10

The '4.2 billion token sequence' headline is output-pixel count rather than an attention sequence, and the fine-tuning tasks include a coarse version of the evaluation target in the input; both central claims reduce partly by construction.

  1. self definitional [Section 6, 'Maximal Sequence Length Scaling' (Table 3), and Section 8 'Impact on HPC']
    "Combining all four, the effective per-GPU sequence length becomes only 17,280 tokens, despite the global output representing 1.1 billion tokens. Finally, by scaling to 128 GPUs, we achieve our largest configuration: 4.2 billion tokens at 0.9 km resolution."

    The paper labels the column 'Max Seq. Length' and headlines 'processes sequences up to 4.2 billion tokens,' but the number is the total output patch count derived from the output tensor [21600, 43200, 18] at 2x2 patches, not the length of any sequence self-attention operates on. TILES explicitly restricts attention to per-tile patches, and the paper's own text states that the 1.1B global output corresponds to only 17,280 tokens per GPU. The 4.2B figure is therefore obtained by redefining 'sequence length' as output-pixel count, so the record claim is true by construction rather than a measured attention sequence. The comparison to the 188K-token state of the art, which is an actual attention-sequence length, is apples-to-oranges.

  2. fitted input called prediction [Section 7 'Science Performance Results' and Table 1 dataset rows]
    "For continental U.S. fine-tuning, inputs include both ERA5 and DAYMET at 28 km, with 7 km DAYMET as output ground truth. ... For global fine-tuning, we target precipitation only and inputs include both ERA5 and IMERG precipitation data at 28 km, downscaled to 7 km resolution."

    In both fine-tuning tasks, the coarse input set includes the same observational product that defines the high-resolution label (DAYMET for the U.S., IMERG for the globe). The reported R2 is computed on predictions whose input already contains a coarse version of the target, so the task reduces partly to inverting a known coarsening of the label product. The 'against observational data' accuracy is therefore not an independent external validation of downscaling from a coarse model/reanalysis field alone; the high R2 is partially forced by the information overlap built into the input construction.

full rationale

The remaining contributions are largely self-contained empirical measurements: the HPC scaling numbers, FLOPs, throughput, and inference times are reported from instrumentation and are independent of the two circular issues above. The Reslim architecture is a concrete implementation with ablations, and the self-citations to ORBIT [45] and sequence-parallelism work [40] are used as baselines or technique sources rather than as the sole evidence for a derivation. However, the token-sequence claim and the fine-tuning evaluation are load-bearing: the abstract and Section 8 repeat the 4.2B-token and R2-against-observation claims as headline results. Correcting the token count to a per-GPU attention sequence (tens of thousands) and removing the target-product-from-input evaluation would change both the record claim and the stated validation. Given that two headline claims reduce partly by construction, a score of 6 is appropriate. No additional circularity was found in the locality argument, the parallelization, or the loss formulation; those steps rest on stated assumptions and external remote-sensing references rather than on the conclusions being derived.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The central claims rest on domain assumptions about spatial locality and on hand-tuned design choices (halo width, edge thresholds, minimum patch size, latitude weights). The accuracy claims additionally depend on a dataset construction where the target observational product is also present as coarse input. No new physical entities are introduced.

free parameters (5)
  • halo_width = not specified
    Tuned empirically in Section 4.2; affects border artifacts, communication overhead, and throughput.
  • Canny_edge_threshold = not specified
    Controls adaptive spatial compression in Section 4.1; a predefined threshold that determines which quadrants are split.
  • minimum_patch_size = not specified
    Termination condition for quad-tree partitioning in Section 4.1; chosen by hand.
  • latitude_weighting_matrix_D = not specified
    Weights the MSE loss by latitude in the Bayesian training objective in Section 4.1; a modeling choice.
  • number_of_tiles_T = 1, 4, 16, 36
    Chosen per experiment in Table 2b; affects sequence reduction, halo overhead, and communication cost.
assumptions (3)
  • domain assumption Spatial locality of the point spread function
    Section 4.2 assumes pixel correlations decay rapidly with distance, so long-range attention can be ignored. If this fails, tiled downscaling loses accuracy.
  • domain assumption Accuracy transfers across resolutions
    The paper claims 0.9 km capability from sequence-length scaling, while accuracy is only demonstrated at 7 km. This assumes performance holds at untested resolutions.
  • standard math Quadratic self-attention complexity and multiplicative token reductions
    The O(N^2/T) complexity argument and the multiplication of compression factors in Section 6 rely on standard complexity arithmetic and linear token decomposition.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ORBIT-2: Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling." pith.science (2026). https://pith.science/paper/YHHZ7E3P

@misc{pith2026250504802,
  author       = {Pith},
  title        = {Pith review of: ORBIT-2: Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YHHZ7E3P}},
  note         = {Machine review of arXiv:2505.04802}
}
abstract

Sparse observations and coarse-resolution climate models limit effective regional decision-making, underscoring the need for robust downscaling. However, existing AI methods struggle with generalization across variables and geographies and are constrained by the quadratic complexity of Vision Transformer (ViT) self-attention. We introduce ORBIT-2, a scalable foundation model for global, hyper-resolution climate downscaling. ORBIT-2 incorporates two key innovations: (1) Residual Slim ViT (Reslim), a lightweight architecture with residual learning and Bayesian regularization for efficient, robust prediction; and (2) TILES, a tile-wise sequence scaling algorithm that reduces self-attention complexity from quadratic to linear, enabling long-sequence processing and massive parallelism. ORBIT-2 scales to 10 billion parameters across 65,536 GPUs, achieving up to 4.1 exaFLOPS sustained throughput and 74--98% strong scaling efficiency. It supports downscaling to 0.9 km global resolution and processes sequences up to 4.2 billion tokens. On 7 km resolution benchmarks, ORBIT-2 achieves high accuracy with $R^2$ scores in the range of 0.98--0.99 against observational data.

Figures

Figures reproduced from arXiv: 2505.04802 by the authors.

Figure 1
Figure 1. A generalized AI architecture diagram for state-of-the-art downscaling foundation models. Note that upsampling is used for each [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Reslim architecture is split into main and residual paths. No upsampling is used for the main path for ViT training, leading to reduced [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Comparison with and without adaptive spatial compression. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: (a) TILES algorithm partitions inputs and outputs into tiles, [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 6
Figure 6. Figure 6: Illustration of ERA5 surface temperature downscaling: (a) input at 622 km resolution, (b) ORBIT-2’s downscaled prediction at 156 km, [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: (a) TILES sequence scaling algorithm speedup across GPUs, compared to an 8-GPU baseline that does not utilize tiling. (b) Strong [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Power spectra of downscaled minimum temperature us [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Example region for global precipitation downscaling during fine-tuning. (a) ERA5 input at 28 km resolution; (b) ORBIT-2 downscaled [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: (a) ERA5 skill scores for global precipitation at 28 km resolution, evaluated using averaged [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 28 canonical work pages

  1. [1]

    Michael Aich, Philipp Hess, Baoxiang Pan, Sebastian Bathiany, Yu Huang, and Niklas Boers. 2024. Conditional diffusion models for downscaling & bias cor- rection of Earth system model precipitation. arXiv preprint arXiv:2404.14416 (2024)

    arXiv 2024

  2. [2]

    Nicolaas J Annau, Alex J Cannon, and Adam H Monahan. 2023. Algorithmic hallucinations of near-surface winds: Statistical downscaling with generative adversarial networks to convection-permitting scales. Artificial Intelligence for 11 Conference’17, July 2017, Washington, DC, USA X Wang et al. the Earth Systems 2, 4 (2023), e230015

    work page 2023

  3. [3]

    Moetasim Ashfaq, Tereza Cavazos, Michelle Simões Reboita, José Abraham Torres- Alavez, Eun-Soon Im, Christiana Funmilola Olusegun, Lincoln Alves, Kesondra Key, Mojisola O Adeniyi, Moustapha Tall, et al. 2021. Robust late twenty-first century shift in the regional monsoons in RegCM-CORDEX simulations. Climate Dynamics 57, 5 (2021), 1463–1488

    work page 2021

  4. [4]

    Christopher Daly and Kirk Bryant. 2013. The PRISM climate and weather sys- tem—an introduction. Corvallis, OR: PRISM climate group 2 (2013)

    work page 2013

  5. [5]

    Tri Dao. 2023. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning. arXiv:2307.08691 [cs.LG]

    arXiv 2023

  6. [6]

    Veronika Eyring, Pierre Gentine, Gustau Camps-Valls, David M Lawrence, and Markus Reichstein. 2024. AI-empowered next-generation multiscale climate modelling for mitigation and adaptation. Nature Geoscience 17, 10 (2024), 963– 971

    work page 2024

  7. [7]

    Fuhrer, T

    O. Fuhrer, T. Chadha, T. Hoefler, G. Kwasniewski, X. Lapillonne, D. Leutwyler, D. Lüthi, C. Osuna, C. Schär, T. C. Schulthess, and H. Vogt. 2018. Near-global climate simulation at 1 km resolution: establishing a performance baseline on 4888 GPUs with COSMO 5.0. Geoscientific Model Development 11, 4 (2018), 1665–1681

    work page 2018

  8. [8]

    Filippo Giorgi. 2019. Thirty Years of Regional Climate Modeling: Where Are We and Where Are We Going next? Journal of Geophysical Research: At- mospheres 124, 11 (2019), 5696–5723. https://doi.org/10.1029/2018JD030094 arXiv:https://agupubs.onlinelibrary.wiley.com/doi/pdf/10.1029/2018JD030094

    doi:10.1029/2018jd030094 2019

Show all 53 references
  1. [9]

    Filippo Giorgi and William J Gutowski Jr. 2015. Regional dynamical downscaling and the CORDEX initiative. Annual review of environment and resources 40, 1 (2015), 467–490

    2015

  2. [10]

    Evan H Girvetz, Edwin P Maurer, Philip B Duffy, Aaron Ruesch, Bridget Thrasher, and Chris Zganjar. 2013. Making climate data relevant to decision making: the important details of spatial and temporal downscaling. World Bank (2013)

    2013

  3. [11]

    Iratxe González-Aparicio, Fabio Monforti, Patrick Volker, Andreas Zucker, Francesco Careri, Thomas Huld, and Jake Badger. 2017. Simulating European wind power generation applying statistical downscaling to reanalysis data. Ap- plied Energy 199 (2017), 155–168

    2017

  4. [12]

    Ethan Gutmann, Tom Pruitt, Martyn P Clark, Levi Brekke, Jeffrey R Arnold, David A Raff, and Roy M Rasmussen. 2014. An intercomparison of statistical downscaling methods used for water resource assessments in the U nited S tates. Water Resources Research 50, 9 (2014), 7167–7186

    2014

  5. [13]

    Dongchen Han, Xuran Pan, Yizeng Han, Shiji Song, and Gao Huang. 2023. FLatten Transformer: Vision Transformer using Focused Linear Attention. arXiv:2308.00442 [cs.CV] https://arxiv.org/abs/2308.00442

    2023 arXiv

  6. [14]

    Chaoyang He, Shen Li, Mahdi Soltanolkotabi, and Salman Avestimehr. 2021. PipeTransformer: Automated Elastic Pipelining for Distributed Training of Trans- formers. arXiv:2102.03161 [cs.LG]

    2021 arXiv

  7. [15]

    Hans Hersbach et al. 2020. The ERA5 global reanalysis. Quarterly Journal of the Royal Meteorological Society 146, 730 (2020), 1999–2049. https://doi.org/10.1002/ qj.3803 arXiv:https://rmets.onlinelibrary.wiley.com/doi/pdf/10.1002/qj.3803

    2020 doi

  8. [16]

    Bruce C Hewitson and Robert George Crane. 1996. Climate downscaling: tech- niques and application. Climate Research 7, 2 (1996), 85–95

    1996

  9. [17]

    Townshend, Shunlin Liang, Satya N.V

    Chengquan Huang, John R.G. Townshend, Shunlin Liang, Satya N.V. Kalluri, and Ruth S. DeFries. 2002. Impact of sensor’s point spread function on land cover characterization: assessment and deconvolution. Remote Sensing of Environment 80, 2 (2002), 203–212. https://doi.org/10.10...

    2002 doi

  10. [18]

    Le, Yonghui Wu, and Zhifeng Chen

    Yanping Huang, Youlong Cheng, Ankur Bapna, Orhan Firat, Mia Xu Chen, Dehao Chen, HyoukJoong Lee, Jiquan Ngiam, Quoc V. Le, Yonghui Wu, and Zhifeng Chen. 2019. GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism. arXiv:1811.06965 [cs.CV]

    2019 arXiv

  11. [19]

    George J Huffman et al. 2020. Integrated multi-satellite retrievals for the global precipitation measurement (GPM) mission (IMERG). Satellite precipitation mea- surement: Volume 1 (2020), 343–353

    2020

  12. [20]

    Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Shuai- wen Leon Song, Samyam Rajbhandari, and Yuxiong He. 2023. DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Trans- former Models. arXiv:2309.14509 [cs.LG] https://arxiv.o...

    2023 arXiv

  13. [21]

    Peter G Jones and Philip K Thornton. 2013. Generating downscaled weather data from a suite of climate models for agricultural modelling applications. Agricul- tural Systems 114 (2013), 1–5

    2013

  14. [22]

    Chiheon Kim, Heungsub Lee, Myungryong Jeong, Woonhyuk Baek, Boogeon Yoon, Ildoo Kim, Sungbin Lim, and Sungwoong Kim. 2020. torchgpipe: On-the-fly Pipeline Parallelism for Training Giant Models. arXiv:2004.09910 [cs.DC]

    2020 arXiv

  15. [23]

    Nikolay Koldunov, Thomas Rackow, Christian Lessig, Sergey Danilov, Suvarchal K Cheedela, Dmitry Sidorenko, Irina Sandu, and Thomas Jung. 2024. Emerging AI- based weather prediction models as downscaling tools. preprint arXiv:2406.17977 (2024)

    2024 arXiv

  16. [24]

    Bipin Kumar, Rajib Chattopadhyay, Manmeet Singh, Niraj Chaudhari, Karthik Kodari, and Amit Barve. 2021. Deep learning–based downscaling of summer monsoon rainfall data over Indian region. Theoretical and Applied Climatology 143 (2021), 1145–1156

    2021

  17. [25]

    Haoying Li, Yifan Yang, Meng Chang, Shiqi Chen, Huajun Feng, Zhihai Xu, Qi Li, and Yueting Chen. 2022. Srdiff: Single image super-resolution with diffusion probabilistic models. Neurocomputing 479 (2022), 47–59

    2022

  18. [26]

    Shen Li et al . 2020. PyTorch Distributed: Experiences on Accelerating Data Parallel Training. Proceedings of the VLDB Endowment 13 (2020), 12 pages. https: //doi.org/10.14778/3415478.3415530

    2020

  19. [27]

    Yuhao Liu, James Doss-Gollin, Guha Balakrishnan, and Ashok Veeraraghavan

  20. [28]

    Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, Furu Wei, and Baining Guo. 2022. Swin Transformer V2: Scaling Up Capacity and Resolution. arXiv:2111.09883 [cs.CV] https://arxiv. org/abs/2111.09883

    2022 arXiv

  21. [29]

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. arXiv:2103.14030 [cs.CV] https://arxiv.org/abs/2103.14030

    2021 arXiv

  22. [30]

    Zhisheng Lu, Juncheng Li, Hong Liu, Chaoyan Huang, Linlin Zhang, and Tieyong Zeng. 2022. Transformer for single image super-resolution. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 457–466

    2022

  23. [31]

    Isaac Lyngaas, Murali Gopalakrishnan Meena, Evan Calabrese, Mohamed Wahib, Peng Chen, Jun Igarashi, Yuankai Huo, and Xiao Wang. 2024. Efficient Distributed Sequence Parallelism for Transformer-Based Image Segmentation. Electronic Imaging 36, 12 (2024), 199–1–199–1. https://doi...

    2024 doi

  24. [32]

    Microsoft. 2024. Microsoft Deepspeed. https://github.com/microsoft/DeepSpeed. Accessed: 2024-04-05

    2024

  25. [33]

    Tung Nguyen, Johannes Brandstetter, Ashish Kapoor, Jayesh K Gupta, and Aditya Grover. 2023. ClimaX: A foundation model for weather and climate.arXiv preprint arXiv:2301.10343 (2023)

    2023 arXiv

  26. [34]

    Tung Nguyen, Jason Jewik, Hritik Bansal, Prakhar Sharma, and Aditya Grover

  27. [35]

    Pytorch. 2024. Automatic Mixed Precision Package - TORCH.AMP. https:// pytorch.org/docs/stable/amp.html#torch.cuda.amp.GradScaler. Accessed: 2024- 04-05

    2024

  28. [36]

    Johannes Schmude et al. 2024. Prithvi WxC: Foundation Model for Weather and Climate. arXiv:2409.13598 [cs.LG] https://arxiv.org/abs/2409.13598

    2024 arXiv

  29. [37]

    Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. 2020. Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism. arXiv:1909.08053 [cs.CL]

    2020 arXiv

  30. [38]

    Taylor, Peter M

    Mark A. Taylor, Peter M. Caldwell, Luca Bertagna, Conrad Clevenger, Aaron S. Donahue, James G. Foucar, Oksana Guba, Benjamin R. Hillman, Noel Keen, Jayesh Krishna, Matthew R. Norman, Sarat Sreepathi, Christopher R. Terai, James B. White, Danqing Wu, Andrew G. Salinger, Renata ...

    2023

  31. [39]

    Peter E Thornton, Michele M Thornton, Benjamin W Mayer, Nate Wilhelmi, Yaxing Wei, Ranjeet Devarakonda, and Robert B Cook. 2014. Daymet: Daily Surface Weather Data on a 1-km Grid for North America, Version 2. Technical Report. Oak Ridge National Lab.(ORNL), Oak Ridge, TN (Unit...

    2014

  32. [40]

    Aristeidis Tsaris, Chengming Zhang, Xiao Wang, Junqi Yin, Siyan Liu, Moetasim Ashfaq, Ming Fan, Jong Youl Choi, Mohamed Wahib, Dan Lu, Prasanna Bal- aprakash, and Feiyi Wang. 2024. Sequence Length Scaling in Vision Trans- formers for Scientific Images on Frontier. arXiv:2405.1...

    2024 arXiv

  33. [41]

    Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan Bovik, and Yinxiao Li. 2022. MaxViT: Multi-Axis Vision Transformer. arXiv:2204.01697 [cs.CV] https://arxiv.org/abs/2204.01697

    2022 arXiv

  34. [42]

    Fang Wang, Di Tian, Lisa Lowe, Latif Kalin, and John Lehrter. 2021. Deep learning for daily precipitation and temperature downscaling. Water Resources Research 57, 4 (2021), e2020WR029308

    2021

  35. [43]

    Atkinson

    Qunming Wang, Yijie Tang, and Peter M. Atkinson. 2020. The effect of the point spread function on downscaling continua. ISPRS Journal of Photogrammetry and Remote Sensing 168 (2020), 251–267. https://doi.org/10.1016/j.isprsjprs.2020.08. 016

    2020 doi

  36. [44]

    Atkinson

    Qunming Wang, Chengyuan Zhang, Xiaohua Tong, and Peter M. Atkinson. 2020. General solution to reduce the point spread function effect in subpixel mapping. Remote Sensing of Environment 251 (2020), 112054. https://doi.org/10.1016/j.rse. 2020.112054

    2020

  37. [45]

    Xiao Wang, Siyan Liu, Aristeidis Tsaris, Jong-Youl Choi, Ashwin Aji, Ming Fan, Wei Zhang, Junqi Yin, Moetasim Ashfaq, Dan Lu, and Prasanna Balaprakash

  38. [46]

    Xiao Wang, Isaac Lyngaas, Aristeidis Tsaris, Peng Chen, Sajal Dash, Mayanka Chandra Shekar, Tao Luo, Hong-Jun Yoon, Mohamed Wahib, and John Gouley. 2023. Ultra-Long Sequence Distributed Transformer. arXiv:2311.02382 [cs.DC] https://arxiv.org/abs/2311.02382 12 ORBIT-2: Scaling ...

    2023 arXiv

  39. [47]

    Robert L Wilby and Christian W Dawson. 2013. The Statistical Downscaling Model: insights from one decade of application. International Journal of Clima- tology 33, 7 (2013)

    2013

  40. [48]

    arXiv:2404.14712 [physics.ao-ph] https://arxiv.org/abs/2404.14712

    ORBIT: Oak Ridge Base Foundation Model for Earth System Predictability. arXiv:2404.14712 [physics.ao-ph] https://arxiv.org/abs/2404.14712

    arXiv

  41. [49]

    Enzhi Zhang, Isaac Lyngaas, Peng Chen, Xiao Wang, Jun Igarashi, Yuankai Huo, Mohamed Wahib, and Masaharu Munetomo. 2024. Adaptive Patching for High- resolution Image Segmentation with Transformers. arXiv:2404.09707 [cs.CV] https://arxiv.org/abs/2404.09707

    2024 arXiv

  42. [50]

    Yanli Zhao et al. 2023. PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel. Proceeding of VLDB Endowment 16, 12 (August 2023), 3848–3860. https://doi.org/10.14778/3611540.3611569 13

    2023

  43. [51]

    Robert L Wilby, TML Wigley, D Conway, PD Jones, BC Hewitson, J Main, and DS Wilks. 1998. Statistical downscaling of general circulation model output: A comparison of methods. Water resources research 34, 11 (1998), 2995–3008

    1998

  44. [2023]

    arXiv:2307.01909 [cs.LG]

    ClimateLearn: Benchmarking Machine Learning for Weather and Climate Modeling. arXiv:2307.01909 [cs.LG]

    arXiv

  45. [2024]

    arXiv preprint arXiv:2410.00381 (2024)

    Generative Precipitation Downscaling using Score-based Diffusion with Wasserstein Regularization. arXiv preprint arXiv:2410.00381 (2024)

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.