Pith. sign in

REVIEW 5 major objections 8 minor 1 cited by

SeaMo: A Season-Aware Multimodal Foundation Model for Remote Sensing

T0 review · 5 major / 8 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SeaMo fuses four seasons and SAR-optical data to top benchmarks

desk verdict SeaMo's season-aware pretraining is a plausible, well-benchmarked recipe, but the core ablation conflates data volume with temporal modeling, so the headline claim isn't yet supported. read the letter →

arxiv 2412.19237 v2 pith:SQTQHFCZ submitted 2024-12-26 cs.CV cs.LG

Xuyang Li , Chenyu Li , Gemine Vivone , Danfeng Hong This is my paper · ORCID
classification cs.CVcs.LG
keywords remotesensingfoundationmodelself-supervisedlearningmaskedimagemodelingmultimodalfusionseasonalawarenessSentinel-2Sentinel-1
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SeaMo is a remote sensing foundation model built to answer one question: does explicitly teaching a masked autoencoder to relate images of the same place across seasons and across sensors make the learned representations better for downstream Earth-observation tasks? The paper argues yes. It pretrains a Vision Transformer jointly on four seasonal snapshots of Sentinel-2 optical and Sentinel-1 SAR imagery from SSL4EO-S12, using partially overlapping crops, a progressive two-phase schedule, and temporal-multimodal fusion blocks. After fine-tuning on classification, segmentation, and change detection benchmarks, SeaMo reports the best or second-best result on nearly every task it evaluates, including gains over models pretrained on more data or fine-tuned with full supervision. A sympathetic reader would take the central claim to be that season-aware, modality-fused pretraining is a recipe that transfers across optical and SAR tasks.

What carries the argument

The load-bearing machinery is the Temporal-Multimodal (TM) fusion block, a cascade of cross-attention layers that runs after the encoder: for each season t, the optical tokens act as queries over SAR tokens of the same season plus tokens from season t−1, and vice versa, so temporal and modal information are fused in one pass. Around it sits the partial-overlap geospatial region selection, which crops each seasonal image from a slightly different offset so that regions overlap only partially, and the progressive pretraining strategy that first trains the encoder on a single season and then unfreezes temporal fusion across all four seasons. The masked autoencoder objective remains the same throughout: reconstruct masked patches in each modality, with the loss summed over seasons and modalities. What carries the argument is that these three choices are varied in ablations, and each is shown to improve downstream performance over its obvious alternative.

What would settle it

Pretrain SeaMo exactly as described but replace each location's four seasonal images with four images sampled from four different geographic locations, so no two crops share a ground footprint; if downstream performance stays roughly equal, the seasonal-consistency assumption is not what drives the gains.

Watch

Extended reading notes

Core claim

The central claim is that SeaMo's pretraining design—jointly encoding optical and SAR tokens, cropping partially overlapping regions across the four seasonal snapshots, and passing the visible tokens through temporal-multimodal fusion blocks before reconstruction—significantly improves the quality of the learned encoder compared with unimodal, single-season, or non-fusing alternatives. On fine-tuning, SeaMo is reported as state of the art or near it on EuroSAT (99.37% accuracy), fMoW-S2 (58.25% with 10% data), BigEarthNet optical (88.54 mAP), DFC2020 segmentation (49.79 mIoU), SegMunich (51.3 mIoU), OSCD change detection (54.54 F1), EuroSAT-SAR (89.69%), BigEarthNet-SAR (82.23 mAP), and DFC2020-SAR (49.54 mIoU). The paper attributes the gains to two mechanisms: the partial-overlap cropping forces the model to find correlations between regions that do not overlap across seasons, and the TM block lets each modality at each time point attend to the other modality and to the previous season, so the reconstruction task cannot be solved by copying a single aligned image. The progressive schedule—first uni-season multimodal learning, then multi-season temporal fusion—is presented as the stabilizer that makes the harder temporal task trainable.

Load-bearing premise

The paper assumes that images of one location taken in different seasons show enough stable geological structure that forcing the model to relate partially overlapping crops teaches time-invariant features rather than injecting seasonal noise; if transient surface changes dominate, the pretraining signal weakens.

Editorial extensions

If this is right

  • Adding a temporal dimension to masked autoencoder pretraining on remote sensing data improves downstream performance; the ablation shows longer temporal sequences (up to four seasons) monotonically help.
  • Multimodal pretraining transfers to unimodal downstream tasks: SeaMo outperforms SAR-only baselines on EuroSAT-SAR and BigEarthNet-SAR, and optical-only baselines on optical tasks.
  • Partially overlapping crops across seasons beat both identical-region crops, which make the reconstruction too easy, and fully random non-overlapping crops.
  • The TM block's fused design is preferred over a decoupled design because it matches or exceeds it while using fewer cross-attention layers and parameters.
  • The model generalizes to unseen image sizes via positional embedding interpolation, but a reduction in spectral bands (from 12 to 9 or 4) hurts accuracy, signalling a remaining tokenizer limitation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the seasonal-consistency mechanism is real, SeaMo's gains should grow when pretraining data cover regions with strong but regular seasonal cycles such as agriculture, and shrink when the four snapshots capture transient surface change such as snow or flood; this could be tested by splitting the pretraining set by biome.
  • A reader could reinterpret the partial-overlap cropping as a cheap form of multi-view consistency that does not need true geology: any set of partially overlapping crops from the same location, even same-season repeats, might give similar gains; the paper does not run that control.
  • The TM block is a general spatiotemporal fusion primitive; it could be lifted into video masked modeling or multi-sensor time-series models beyond optical/SAR pairs, though the paper only evaluates it in the remote sensing setting.
  • The reported gains at 10% fine-tuning data suggest the encoder learns transferable features; an obvious next stress test is zero-shot or linear-probe evaluation on unseen sensors and resolutions, which the paper only partially covers.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 8 minor

Summary. SeaMo is a masked-autoencoder visual foundation model pretrained on SSL4EO-S12, jointly modeling four-season Sentinel-2 optical and Sentinel-1 SAR inputs. The paper's contributions are (i) partially overlapping spatial region selection across seasons, (ii) a Temporal-Multimodal (TM) fusion block that propagates cross-attention information across seasons and modalities, and (iii) a progressive two-phase pretraining strategy (single-time multimodal learning followed by multi-time fusion). The model is evaluated by fine-tuning and linear probing on EuroSAT, fMoW-S2, BigEarthNet(-SAR), DFC2020(-SAR), SegMunich, OSCD, GLH-Water, and S1S2-Water, where it reports the best or second-best result in most tables.

Significance. If the claimed gains are reproducible, SeaMo offers a practical pretraining recipe for season-aware multimodal remote sensing foundation models. The paper's strengths include broad benchmark coverage across optical and SAR tasks, systematic ablations of the cropping strategy, TM-block design, temporal length, pretraining strategy, decoder depth, mask ratio, reconstruction target, and positional embedding; explicit reporting of computational cost; and several honest limitation statements, such as the OSCD result being acknowledged as not state-of-the-art and the pretraining-strategy ablation being acknowledged as difficult to interpret. The central architectural idea, the TM block, is clearly described and directly ablated. The main weakness is that the key causal claim—that explicit seasonal fusion "significantly enhances" performance—rests on single-run margins that are often smaller than typical fine-tuning noise, so the statistical significance and protocol-matched validity of the gains are not yet established.

major comments (5)
  1. [§4.5.3, Table 14] The temporal-length ablation in Table 14 conflates the amount of pretraining data with the presence of temporal structure. The T=1 row uses only the first season, which is 25% of SSL4EO-S12, and excludes the TM block, while the T=4 row uses all four seasons, the TM block, and the progressive two-phase schedule. The comparison therefore changes the number of distinct training images, the total number of training epochs, the architecture, and the training schedule simultaneously. A masked autoencoder trained on four times as many distinct images would be expected to improve even without any season-aware mechanism, so the inference from this table to "explicit seasonal modeling helps" is not supported. The paper should provide a data-matched and schedule-matched control, such as the Multimodal-Temporal row in Table 12 run on the same total epochs, or an epoch-scaled T=1 condition.
  2. [§4.2, §4.5.2, Tables 1-14] No error bars, confidence intervals, or multiple-seed results appear anywhere in the paper. Several of the margins that carry the central claim are very small: in Table 11 the default Fuse design is 99.37 vs 99.41 for Decouple on EuroSAT and 82.23 vs 82.30 on BEN-SAR; in Table 14 the T=3-to-T=4 improvement is 0.23 mAP on BEN and 0.07 mAP on BEN-SAR; in Table 1 the EuroSAT advantage over DOFA is 0.07 points. The manuscript itself states in Section 4.5.4 that "It is challenging to draw a definitive conclusion from these experiments," which is in tension with the abstract's and Section 3.3.2's claim that seasonal fusion "significantly enhance[s]" performance. At minimum, the central ablations and the comparisons against the closest baselines should be repeated with at least three seeds and reported as mean ± std.
  3. [§4.5.2, §4.5.4, Tables 11-12] The chosen default configuration is not the best on all tasks. In Table 11, Decouple outperforms the adopted Fuse design on EuroSAT (99.41 vs 99.37) and BEN-SAR (82.30 vs 82.23), and Table 12 shows that Siamese-Temporal beats the final Multimodal-Temporal-TM on BEN-SAR (+0.45 mAP). The paper acknowledges the latter but chooses Fuse on computational grounds. That is a legitimate efficiency trade-off, but it should be presented as such, not as evidence that the proposed Fuse design is the accuracy-maximizing instantiation of the TM block. Reporting both variants with variance would clarify whether the differences are meaningful.
  4. [§4.2, Tables 1-8] The benchmark comparison protocol is underspecified. The tables appear to mix numbers reproduced by the authors with numbers quoted from prior publications (e.g., SatMAE is described in the text as pretrained on fMoW-Sentinel, while other baselines come from different pretraining corpora), and the table notes only partially clarify the 10% fine-tuning rule. Since several reported advantages are below 0.1 point (Table 1 EuroSAT: 99.37 vs 99.30; Table 5 OSCD: 54.54 vs 54.29), small differences in downstream protocol could change the ranking. For each entry, the paper should state whether the result was obtained under the authors' own fine-tuning pipeline or quoted from the original paper, and provide the corresponding protocol details.
  5. [§3.2, Table 10] The design rationale for partial-overlap cropping depends on the Section 3.2 assumption that "the geological attributes of a given area tend to remain consistent over time." When seasonal imagery is dominated by transient surface changes, the cross-season reconstruction objective could inject noise rather than stable structure. Table 10 provides empirical support for partial overlap on the tested tasks, but it does not distinguish between learning stable temporal correlations and simply benefiting from a harder multi-view reconstruction task. A concrete control would be to shuffle the seasonal order or to pair crops from different locations during temporal fusion; if the benefit persists, the "season-aware" interpretation would need to be revised.
minor comments (8)
  1. [Table 10] The column header "OCSD" should be "OSCD."
  2. [§4.2.1] The sentence about SatMAE being "pretrained and then fine-tuned on the fMoW-Sentinel" is confusing; please clarify which pretraining corpus and which fine-tuning split are used for each baseline in Tables 1 and 2.
  3. [§3.2] The phrase "geological attributes" is too narrow; land-cover or surface attributes would better describe the properties that are expected to remain consistent across seasons.
  4. [Algorithm 1] The fusion operator f[·] is used in the pseudocode before it is defined; please define it in the text and use a consistent notation for the concatenation inside the brackets.
  5. [References] References [53] and [55] are the same paper and should be consolidated.
  6. [Figure 15] Please add axis labels and specify whether the x-axis counts epochs of the second pretraining phase or total pretraining epochs, and state the exact epoch values at each plotted point.
  7. [Table 9] The baselines in Table 9 (UNet, ResNet, ViT) are not foundation models; a sentence in the caption or text should make this explicit so the comparison is not overinterpreted.
  8. [Data availability] The Data Availability section lists only the datasets; the authors should consider releasing code and pretrained weights to support reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SeaMo's season-aware fusion claim rests on benchmark comparisons and ablations, not on self-referential derivation or fitted predictions.

full rationale

The paper's contribution is an architecture and pretraining recipe evaluated on downstream benchmarks, and no load-bearing step reduces to its own inputs by construction. The TM block is tested by ablations on held-out tasks (Tables 10-12), the cropping strategy is ablated in Table 10, and the pretraining objective is a standard masked reconstruction MSE over visible and masked patches, with no fitted parameter relabeled as a prediction. Self-citations to SpectralGPT and DOFA appear only as published baselines or evaluation protocols, not as premises that force the central claim. The temporal-length ablation in Table 14 is confounded with data quantity and training schedule, but that is a validity concern rather than a circular reduction: the conclusion does not hold by definition or by self-citation. The structural assumption that geological attributes remain consistent over time motivates the partial-overlap cropping strategy but is not equivalent to the downstream performance claims. Because the paper is self-contained against external benchmarks and its ablations are the actual evidence for the season-aware mechanism, no circular step with a quotable reduction can be exhibited.

Assumptions & free parameters 5 free parameters · 4 assumptions · 1 invented entities

The central claim rests on standard MAE machinery plus a handful of hand-tuned hyperparameters (mask ratio, crop overlap, decoder depth, temporal length, schedule). The main domain-level axiom is seasonal consistency of geographic areas. No new physical entities are proposed; the TM block is an architectural module with in-paper only evidence.

free parameters (5)
  • Mask ratio = 75%
    Chosen as default in Section 4.5; ablation Table 13b shows 75% beats 50% and 90% on BEN/EuroSAT/BEN-SAR.
  • Partial overlap crop rate = stated default [75%,100%], empirically best [50%,100%]
    Section 4.5 states default; Table 10 shows [50%,100%] strictly best on all five tasks. Ambiguous which was used for main results.
  • Decoder depth = 4 blocks (default), 6 blocks best
    Table 13a shows depth 6 gives 88.58 vs 88.54 mAP on BEN; default is 4.
  • Temporal length = 4 seasons
    Table 14 shows monotonic gains from 1 to 4; 4 is default.
  • Progressive schedule = 20 epochs single-time + 200 epochs multi-time
    Hand-chosen; Figure 15 shows longer pretraining helps but no formal optimization.
assumptions (4)
  • domain assumption Geological attributes of a given area remain consistent over time, aside from human-induced changes.
    Invoked in Section 3.2 to justify partial-overlap cropping as a mechanism for learning cross-temporal correlations. If seasonal changes dominate, the reconstruction task may inject noise.
  • domain assumption SSL4EO-S12's four seasonal snapshots and Sentinel-1/Sentinel-2 pairs are adequately aligned in time and geography.
    Used throughout Section 3 to treat multi-season and multimodal inputs as observations of the same locations; the partial-overlap crop assumes spatial consistency across modalities.
  • standard math MAE-style reconstruction of normalized masked patches is a valid proxy for learning transferable features in multimodal, multi-seasonal RS data.
    Borrowed from MAE (He et al. 2022) and used for pretraining; not re-derived.
  • domain assumption A single tokenizer per modality can adequately encode 12-band optical and 2-band SAR channels for the fusion encoder.
    Section 3.1.1 states dedicated embedding modules but no explicit architecture; Section 4.5.6 shows reducing bands from 12 to 9 or 4 degrades accuracy, indicating sensitivity.
invented entities (1)
  • Temporal-Multimodal (TM) fusion block
    purpose: Fuses optical and SAR tokens across seasons and modalities via cross-attention, with previous-season tokens as keys/values.
    The block is new to this paper. Its efficacy is evidenced only by author-run ablations on downstream benchmarks; no publicly released checkpoint or code allows independent falsification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SeaMo: A Season-Aware Multimodal Foundation Model for Remote Sensing." pith.science (2026). https://pith.science/paper/SQTQHFCZ

@misc{pith2026241219237,
  author       = {Pith},
  title        = {Pith review of: SeaMo: A Season-Aware Multimodal Foundation Model for Remote Sensing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SQTQHFCZ}},
  note         = {Machine review of arXiv:2412.19237}
}
read the original abstract

Remote Sensing (RS) data encapsulates rich multi-dimensional information essential for Earth observation. Its vast volume, diverse sources, and temporal continuity make it particularly well-suited for developing large Visual Foundation Models (VFMs). These models serve as powerful feature extractors, leveraging extensive RS data for pretraining and subsequent fine-tuning in various geoscientific applications. However, existing VFMs in the RS domain often concentrate on specific image characteristics, neglecting the full season-aware potential of RS data. To bridge this gap, we introduce SeaMo, a novel VFM that effectively integrates multimodal and multi-seasonal RS information. SeaMo leverages a masked image modeling framework to fully exploit the spatial, spectral, and seasonal dimensions of RS data. Specifically, we employ unaligned spatial region selection to capture spatial heterogeneity, incorporate multi-source inputs for enhanced multimodal integration, and introduce temporal-multimodal fusion blocks to assimilate seasonal variations effectively. By explicitly modeling the complex, season-dependent attributes of RS data, SeaMo enhances generalization, robustness, and adaptability across geoscientific tasks. Extensive experiments and ablation studies demonstrate its superior performance, underscoring its potential as a foundational model for Earth observation.

Figures

Figures reproduced from arXiv: 2412.19237 by the authors.

Figure 1
Figure 1. Pretraining workflow of the SeaMo foundation model. The SeaMo architecture integrates three primary components: encoders, Temporal-Multimodal fusion blocks (TM blocks), and decoders. Our approach incorporates a partially overlapping spatial selecting strategy, ensuring that images from the same temporal instance are selected identically across various modalities, while images from different instances exhibit partial… view at source ↗
Figure 2
Figure 2. Different region selection strategies for temporal data. The solid boxes indicate the image regions that are selected and fed into the network. (a) Images from different seasons are selected from the same section. (b) Images from different seasons are selected based on a specific proportion of the full image, ensuring partial overlap. (c) Images from different seasons are selected with no overlap. multi-sensor and e… view at source ↗
Figure 3
Figure 3. An illustration of the Temporal-Multimodal (TM) block. In this block, data from each modality not only participate in fusion interactions during the current season but also influence the fusion process in subsequent seasons. For clarity, the symbols in the figure are defined as follows: k denotes the key vector, q denotes the query vector, and v denotes the value vector; CA represents cross-attention; and f indicate… view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Three distinct multimodal pretraining strategies. Left: The MIM model is weight-sharing, however, lacks interaction across different modalities. Middle: Data from two modalities are concatenated and then fed into the MIM model. Right: A time series of multimodal images…
Figure 5
Figure 5. Figure 5: Sample visualization of the SSL4EO-S12 dataset. Odd-numbered rows represent Sentinel-2 (multi-spectral), and even￾numbered rows represent Sentinel-1 (SAR). Each location is docu￾mented with four seasonal snapshots. EuroSAT [45] is a dataset comprising 27,000 Sentinel￾2…
Figure 6
Figure 6. Figure 6: Visual results obtained by using different pre-trained foundation models for the downstream segmentation task on the SegMunich dataset. images due to low quality. Consistent with previous stud￾ies [48], the dataset is divided into 354,196 training sam￾ples and 118,065 …
Figure 7
Figure 7. Figure 7: Visual results obtained by using different pretrained foundation models for the downstream semantic segmentation task on the SegMunich dataset. 4.2.3. Semantic segmentation We selected the DFC2020 [49] and SegMunich [2] datasets to evaluate our model on optical Sentine…
Figure 8
Figure 8. Figure 8: Visual results obtained by using different pretrained foundation models for the downstream change detection task on the OSCD dataset [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Visual results obtained by using different pretrained foundation models for the downstream change detection task on the OSCD dataset. models. The BigEarthNet-SAR (BEN-SAR) dataset cor￾responds to the optical dataset and forms part of the BigEarthNet-MultiModal (MM) dat…
Figure 10
Figure 10. Figure 10: Additional visualizations of ground truth labels and SeaMo’s predicted results on the SegMunich dataset [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: Visualizations of ground truth labels and SeaMo’s predicted results on the GLH-Water dataset. consistency in all variables except for spatial cropping, us￾ing multimodal inputs and incorporating a TM block ar￾chitecture. In [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]
Figure 12
Figure 12. Figure 12: SAR visual comparisons of ground truth labels and SeaMo’s predicted results on the S1S2-Water dataset. We fine-tuned our model using only the Sentinel-1 data from the dataset [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]
Figure 13
Figure 13. Figure 13: Optical visual comparisons of ground truth labels and SeaMo’s predicted results on the S1S2-Water dataset. We fine-tuned our model using only the Sentinel-2 data from the dataset [PITH_FULL_IMAGE:figures/full_fig_p021_13.png]
Figure 14
Figure 14. Figure 14: Different designs for TM blocks. (a) Fuse: In this design, a linear layer is used to combine tokens from previous sea￾sons with tokens from the current season. (b) Decouple: In this design, two independent sets of cross-attention layers interact with the modality toke…
Figure 15
Figure 15. Figure 15: Pretraining Schedule. SeaMo was evaluated on the BigEarthNet-optical (BEN), EuroSAT, and BigEarthNet-SAR (BEN-SAR) datasets. Our experiments demonstrate that a longer pretraining duration significantly boosts performance [PITH_FULL_IMAGE:figures/full_fig_p022_15.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Deploying Geospatial Foundation Models in the Real World: Lessons from WorldCereal

    cs.LG 2025-07 conditional novelty 5.0 of 10

    A structured protocol for deploying geospatial foundation models is introduced and validated in WorldCereal, where fine-tuned Presto outperforms a fully-supervised CatBoost baseline in crop mapping.

Reference graph

Works this paper leans on

57 extracted references · 31 canonical work pages · cited by 1 Pith paper

  1. [1]

    Vivone, L.-J

    G. Vivone, L.-J. Deng, S. Deng, D. Hong, M. Jiang, C. Li, W. Li, H. Shen, X. Wu, J.-L. Xiao, J. Yao, M. Zhang, J. Chanussot, S. Garc ´ ıa, A. Plaza, Deep learning in remote sensing image fusion: Methods, protocols, data, and future perspectives, IEEE Geoscience and Remote Sensing Magazine 13 (1) (2025) 269–

    work page 2025

  2. [2]

    D. Hong, B. Zhang, X. Li, Y. Li, C. Li, J. Yao, N. Yokoya, H. Li, P. Ghamisi, X. Jia, A. Plaza, P. Gamba, J. A. Benediktsson, J. Chanussot, SpectralGPT: Spectral remote sensing foundation model, IEEE Transactions on Pattern Analysis and Machine Intelligence 46 (8) (2024) 5227–5244. doi:10.1109/TPAMI.2024. 3362475. 24

    doi:10.1109/tpami.2024 2024

  3. [3]

    C. Li, B. Zhang, D. Hong, J. Zhou, G. Vivone, S. Li, J. Chanus- sot, Casformer: Cascaded transformers for fusion-aware compu- tational hyperspectral imaging, Inf. Fusion 108 (2024) 102408. doi:10.1016/J.INFFUS.2024.102408

    arXiv 2024

  4. [4]

    D. Hong, C. Li, B. Zhang, N. Yokoya, J. A. Benediktsson, J. Chanussot, Multimodal artificial intelligence foundation mod- els: Unleashing the power of remote sensing big data in earth observation, The Innovation Geoscience 2 (1) (2024) 100055. doi:10.59717/j.xinn-geo.2024.100055

    arXiv 2024

  5. [5]

    Y. Xu, T. Bai, W. Yu, S. Chang, P. M. Atkinson, P. Ghamisi, AI security for geoscience and remote sensing: Challenges and future trends, IEEE Geoscience and Remote Sensing Magazine 11 (2) (2023) 60–85. doi:10.1109/MGRS.2023.3272825

    work page arXiv 2023

  6. [6]

    Bommasani, D

    R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brun- skill, et al., On the opportunities and risks of foundation models (2022). arXiv:2108.07258

    arXiv 2022

  7. [7]

    Wanyan, S

    X. Wanyan, S. Seneviratne, S. Shen, M. Kirley, Extending global-local view alignment for self-supervised learning with re- mote sensing imagery, in: 2024 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition Workshops (CVPR W), 2024, pp. 2443–2453. doi:10.1109/CVPRW63382.2024.00251

    arXiv 2024

  8. [8]

    D. Wang, Q. Zhang, Y. Xu, J. Zhang, B. Du, D. Tao, L. Zhang, Advancing plain vision transformer toward remote sensing foun- dation model, IEEE Transactions on Geoscience and Remote Sensing 61 (2023) 1–15. doi:10.1109/TGRS.2022.3222818

    work page arXiv 2023

Show all 57 references
  1. [12]

    X. Sun, P. Wang, W. Lu, Z. Zhu, X. Lu, Q. He, J. Li, X. Rong, Z. Yang, H. Chang, Q. He, G. Yang, R. Wang, J. Lu, K. Fu, Ringmo: A remote sensing foundation model with masked im- age modeling, IEEE Transactions on Geoscience and Remote Sensing 61 (2023) 1–22. doi:10.1109/TGRS.2...

    2023

  2. [13]

    Y. Cong, S. Khanna, C. Meng, P. Liu, E. Rozi, Y. He, M. Burke, D. Lobell, S. Ermon, SatMAE: Pre-training transformers for temporal and multi-spectral satellite imagery, in: Advances in Neural Information Processing Systems, Vol. 35, Curran Asso- ciates, Inc., 2022, pp. 197–211

    2022

  3. [14]

    K. He, X. Chen, S. Xie, Y. Li, P. Doll´ ar, R. Girshick, Masked autoencoders are scalable vision learners, in: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 15979–15988. doi:10.1109/CVPR52688. 2022.01553

    2022

  4. [17]

    Fuller, K

    A. Fuller, K. Millard, J. Green, CROMA: Remote sensing representations with contrastive radar-optical masked autoen- coders, in: Advances in Neural Information Processing Systems, Vol. 36, Curran Associates, Inc., 2023, pp. 5506–5538

    2023

  5. [18]

    X. Guo, J. Lao, B. Dang, Y. Zhang, L. Yu, L. Ru, L. Zhong, Z. Huang, K. Wu, D. Hu, H. He, J. Wang, J. Chen, M. Yang, Y. Zhang, Y. Li, SkySense: A multi-modal remote sensing foun- dation model towards universal interpretation for earth observa- tion imagery, in: 2024 IEEE/CVF C...

    2024

  6. [19]

    Bastani, P

    F. Bastani, P. Wolters, R. Gupta, J. Ferdinando, A. Kembhavi, Satlaspretrain: A large-scale dataset for remote sensing image understanding, in: 2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 16726–16736. doi:10. 1109/ICCV51070.2023.01538

    2023

  7. [20]

    Srivastava, E

    N. Srivastava, E. Mansimov, R. Salakhutdinov, Unsupervised learning of video representations using lstms, in: Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37, ICML’15, JMLR.org, 2015, p. 843–852

    2015

  8. [21]

    Vondrick, H

    C. Vondrick, H. Pirsiavash, A. Torralba, Anticipating visual representations from unlabeled video, in: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 98–106. doi:10.1109/CVPR.2016.18

    2016 doi

  9. [22]

    T. Chen, S. Kornblith, M. Norouzi, G. Hinton, A simple framework for contrastive learning of visual representations, in: Proceedings of the 37th International Conference on Ma- chine Learning, Vol. 119 of Proceedings of Machine Learning 25 Research, PMLR, 2020, pp. 1597–1607

    2020

  10. [23]

    K. He, H. Fan, Y. Wu, S. Xie, R. Girshick, Momentum con- trast for unsupervised visual representation learning, in: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR), 2020, pp. 9726–9735. doi:10.1109/CVPR42600. 2020.00975

    2020

  11. [24]

    Z. Xie, Z. Zhang, Y. Cao, Y. Lin, J. Bao, Z. Yao, Q. Dai, H. Hu, SimMIM: a simple framework for masked image mod- eling, in: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 9643–9653. doi: 10.1109/CVPR52688.2022.00943

    2022

  12. [26]

    Bachmann, D

    R. Bachmann, D. Mizrahi, A. Atanov, A. Zamir, MultiMAE: Multi-modal multi-task masked autoencoders, in: Computer Vision – ECCV 2022, Springer Nature Switzerland, Cham, 2022, pp. 348–367. doi:10.1007/978-3-031-19836-6_20

    2022 doi

  13. [27]

    Feichtenhofer, H

    C. Feichtenhofer, H. Fan, Y. Li, K. He, Masked autoencoders as spatiotemporal learners, in: Proceedings of the 36th Inter- national Conference on Neural Information Processing Systems (NeurIPS 2022), NIPS ’22, Curran Associates Inc., Red Hook, NY, USA, 2022, pp. 35946–35958

    2022

  14. [28]

    L. Zhou, H. Liu, J. Bae, J. He, D. Samaras, P. Prasanna, Self pre-training with masked autoencoders for medical image classification and segmentation, in: 2023 IEEE 20th Interna- tional Symposium on Biomedical Imaging (ISBI), 2023, pp. 1–6. doi:10.1109/ISBI53787.2023.10230477

    2023

  15. [29]

    Nguyen, J

    T. Nguyen, J. Brandstetter, A. Kapoor, J. K. Gupta, A. Grover, ClimaX: a foundation model for weather and climate, in: Proceedings of the 40th International Conference on Machine Learning, ICML’23, JMLR.org, 2023, pp. 25904–25938

    2023

  16. [30]

    Mendieta, B

    M. Mendieta, B. Han, X. Shi, Y. Zhu, C. Chen, Towards geospatial foundation models via continual pretraining, in: 2023 IEEE/CVF International Conference on Computer Vi- sion (ICCV), 2023, pp. 16760–16770. doi:10.1109/ICCV51070. 2023.01541

    2023

  17. [31]

    M. Tang, A. Cozma, K. Georgiou, H. Qi, Cross-scale mae: a tale of multi-scale exploitation in remote sensing, in: Proceedings of the 37th International Conference on Neural Information Pro- cessing Systems, NIPS ’23, Curran Associates Inc., Red Hook, NY, USA, 2023, pp. 20054–20066

    2023

  18. [32]

    Ayush, B

    K. Ayush, B. Uzkent, C. Meng, K. Tanmay, M. Burke, D. Lo- bell, S. Ermon, Geography-aware self-supervised learning, in: 2021 IEEE/CVF International Conference on Computer Vi- sion (ICCV), 2021, pp. 10161–10170. doi:10.1109/ICCV48922. 2021.01002

    2021

  19. [33]

    Y. Wang, C. M. Albrecht, X. X. Zhu, Multilabel-guided soft contrastive learning for efficient earth observation pretrain- ing, IEEE Transactions on Geoscience and Remote Sensing 62 (2024) 1–16. doi:10.1109/TGRS.2024.3466896

    2024

  20. [34]

    F. Liu, D. Chen, Z. Guan, X. Zhou, J. Zhu, Q. Ye, L. Fu, J. Zhou, Remoteclip: A vision language foundation model for remote sensing, IEEE Transactions on Geoscience and Remote Sensing 62 (2024) 1–16. doi:10.1109/TGRS.2024.3390838

    2024

  21. [35]

    V. V. Cepeda, G. K. Nayak, M. Shah, GeoCLIP: clip-inspired alignment between locations and images for effective worldwide geo-localization, in: Proceedings of the 37th International Con- ference on Neural Information Processing Systems, NIPS ’23, Curran Associates Inc., Red Ho...

    2023

  22. [36]

    X. Li, C. Wen, Y. Hu, Z. Yuan, X. X. Zhu, Vision-language models in remote sensing: Current progress and future trends, IEEE Geoscience and Remote Sensing Magazine 12 (2) (2024) 32–66. doi:10.1109/MGRS.2024.3383473

    2024

  23. [37]

    Y. Wang, N. A. A. Braham, Z. Xiong, C. Liu, C. M. Albrecht, X. X. Zhu, Ssl4eo-s12: A large-scale multimodal, multitempo- ral dataset for self-supervised learning in earth observation [soft- ware and data sets], IEEE Geoscience and Remote Sensing Mag- azine 11 (3) (2023) 98–106...

    2023

  24. [38]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, I. Polosukhin, Attention is all you need, in: Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, Curran Associates Inc., Red Hook, NY, USA, 2017,...

    2017

  25. [39]

    Dosovitskiy, L

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, N. Houlsby, An image is worth 16x16 words: Transformers for image recognition at scale, ICLR

  26. [40]

    Gupta, J

    A. Gupta, J. Wu, J. Deng, L. Fei-Fei, Siamese masked autoen- coders, in: Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Curran Associates Inc., Red Hook, NY, USA, 2023, pp. 40676–40693

    2023

  27. [41]

    Eyma¨ el, R

    A. Eyma¨ el, R. Vandeghen, A. Cioppa, S. Giancola, B. Ghanem, M. Van Droogenbroeck, Efficient image pre- training with siamese cropped masked autoencoders, in: Com- puter Vision – ECCV 2024, Springer Nature Switzerland, Cham, 2025, pp. 348–366. doi:10.1007/978-3-031-73337-6_ 20

    2024 doi

  28. [42]

    Xiong, Y

    Z. Xiong, Y. Wang, F. Zhang, A. J. Stewart, J. Hanna, D. Borth, I. Papoutsis, B. L. Saux, G. Camps-Valls, X. X. Zhu, Neural plasticity-inspired multimodal foundation model for earth observation (2024). arXiv:2403.15356

    2024

  29. [43]

    Caron, H

    M. Caron, H. Touvron, I. Misra, H. Jegou, J. Mairal, P. Bo- 26 janowski, A. Joulin, Emerging properties in self-supervised vi- sion transformers, in: 2021 IEEE/CVF International Confer- ence on Computer Vision (ICCV), 2021, pp. 9630–9640. doi: 10.1109/ICCV48922.2021.00951

    2021

  30. [44]

    Assran, Q

    M. Assran, Q. Duval, I. Misra, P. Bojanowski, P. Vincent, M. Rabbat, Y. LeCun, N. Ballas, Self-supervised learning from images with a joint-embedding predictive architecture, in: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 15619–15629....

    2023

  31. [45]

    Helber, B

    P. Helber, B. Bischke, A. Dengel, D. Borth, Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 12 (7) (2019) 2217–

    2019

  32. [46]

    Sumbul, M

    G. Sumbul, M. Charfuelan, B. Demir, V. Markl, Bigearthnet: A large-scale benchmark archive for remote sensing image un- derstanding, in: IGARSS 2019 - 2019 IEEE International Geo- science and Remote Sensing Symposium, 2019, pp. 5901–5904. doi:10.1109/IGARSS.2019.8900532

    2019

  33. [47]

    Sumbul, A

    G. Sumbul, A. de Wall, T. Kreuziger, F. Marcelino, H. Costa, P. Benevides, M. Caetano, B. Demir, V. Markl, Bigearthnet- mm: A large-scale, multimodal, multilabel benchmark archive for remote sensing image classification and retrieval [software and data sets], IEEE Geoscience a...

    2021

  34. [48]

    Neumann, A

    M. Neumann, A. S. Pinto, X. Zhai, N. Houlsby, In-domain rep- resentation learning for remote sensing, in: AI for Earth Sci- ences Workshop at International Conference on Learning Rep- resentations (ICLR), 2020, pp. 1–20

    2020

  35. [49]

    Robinson, K

    C. Robinson, K. Malkin, N. Jojic, H. Chen, R. Qin, C. Xiao, M. Schmitt, P. Ghamisi, R. H¨ ansch, N. Yokoya, Global land- cover mapping with weak supervision: Outcome of the 2020 ieee grss data fusion contest, IEEE Journal of Selected Topics in Applied Earth Observations and Re...

    2021

  36. [50]

    Wanyan, S

    X. Wanyan, S. Seneviratne, S. Shen, M. Kirley, DINO-MC: self- supervised contrastive learning for remote sensing imagery with multi-sized local crops, CoRR abs/2303.06670. arXiv:2303. 06670, doi:10.48550/ARXIV.2303.06670

    doi arXiv

  37. [51]

    R. C. Daudt, B. Le Saux, A. Boulch, Y. Gousseau, Urban change detection for multispectral earth observation using con- volutional neural networks, in: IGARSS 2018 - 2018 IEEE In- ternational Geoscience and Remote Sensing Symposium, 2018, pp. 2115–2118. doi:10.1109/IGARSS.2018.8518015

    2018

  38. [52]

    T. Xiao, Y. Liu, B. Zhou, Y. Jiang, J. Sun, Unified perceptual parsing for scene understanding, in: Computer Vision – ECCV 2018: 15th European Conference, Munich, Germany, September 8–14, 2018, Proceedings, Part V, Springer-Verlag, Berlin, Hei- delberg, 2018, p. 432–448. doi:1...

    2018 doi

  39. [54]

    Y. Wang, H. H. Hern´ andez, C. M. Albrecht, X. X. Zhu, Fea- ture guided masked autoencoder for self-supervised learning in remote sensing, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 18 (2025) 321–336. doi:10.1109/JSTARS.2024.3493237

    2025

  40. [55]

    Y. Wang, C. M. Albrecht, X. X. Zhu, Self-supervised vision transformers for joint sar-optical representation learning, in: IGARSS 2022 - 2022 IEEE International Geoscience and Re- mote Sensing Symposium, 2022, pp. 139–142. doi:10.1109/ IGARSS46834.2022.9883983

    2022

  41. [56]

    Fuller, K

    A. Fuller, K. Millard, J. R. Green, Satvit: Pretraining transformers for earth observation, IEEE Geoscience and Re- mote Sensing Letters 19 (2022) 1–5. doi:10.1109/LGRS.2022. 3201489

    2022 doi

  42. [57]

    Chan-To-Hing, B

    H. Chan-To-Hing, B. Veeravalli, Fus-mae: A cross-attention- based data fusion approach for masked autoencoders in remote sensing, in: IGARSS 2024 - 2024 IEEE International Geoscience and Remote Sensing Symposium, 2024, pp. 6953–6958. doi: 10.1109/IGARSS53475.2024.10642424

    2024

  43. [58]

    Ronneberger, P

    O. Ronneberger, P. Fischer, T. Brox, U-Net: Convolutional networks for biomedical image segmentation, in: N. Navab, J. Hornegger, W. M. Wells, A. F. Frangi (Eds.), Medical Im- age Computing and Computer-Assisted Intervention – MICCAI 2015, Springer International Publishing, Ch...

    2015

  44. [59]

    K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778. doi:10.1109/CVPR.2016.90

    2016 doi

  45. [60]

    Y. Li, B. Dang, W. Li, Y. Zhang, Glh-water: a large-scale dataset for global surface water detection in large-size very- high-resolution satellite imagery, in: Proceedings of the Thirty- Eighth AAAI Conference on Artificial Intelligence and Thirty- Sixth Conference on Innovati...

    2024 doi

  46. [61]

    Wieland, F

    M. Wieland, F. Fichtner, S. Martinis, S. Groth, C. Krullikowski, S. Plank, M. Motagh, S1s2-water: A global dataset for semantic segmentation of water bodies from sentinel- 1 and sentinel-2 27 satellite images, IEEE Journal of Selected Topics in Applied Earth Observations and R...

    2024

  47. [241]

    doi:10.1007/978-3-319-24574-4\_28

    doi

  48. [310]

    doi:10.1109/MGRS.2024.3495516

    2024

  49. [2226]

    doi:10.1109/JSTARS.2019.2918242

    2019

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.