Pith. sign in

REVIEW 3 major objections 5 minor 30 references

Pretraining loss is a bad guide for Earth-observation models; grow the encoder and data, freeze the projector, then distill.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 22:47 UTC pith:CM7XCJ2P

load-bearing objection Largest controlled EO scaling sweep to date; the loss-vs-downstream finding and encoder+data allocation rule are real contributions, with the usual suite-specificity caveat. the 3 major comments →

arxiv 2607.03949 v1 pith:CM7XCJ2P submitted 2026-07-04 cs.CV cs.LG

TESSERA v2: Scaling Pixel-wise Earth Foundation Models

Zhengpeng Feng , Sadiq Jaffer , Ira Shokar , Jovana Knezevic , Mark Elvers , Clement Atzberger , Robin Young , Aneesh Naik
show 5 more authors
Niall Robinson Andrew Blake David Coomes Anil Madhavapeddy Srinivasan Keshav
This is my paper
classification cs.CV cs.LG
keywords Earth observationfoundation modelsscaling lawsBarlow Twinsknowledge distillationMatryoshka representationspixel-wise embeddingsembeddings-as-data
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Pixel-wise Earth-observation foundation models now deliver strong spatial embeddings, but the field still lacks a clear rule for how to spend pretraining compute. This paper runs 395 controlled training experiments inside one fixed model family and scores every run on 15 downstream tasks. It finds that the self-supervised pretraining loss barely tracks real task performance, so picking models by loss alone can waste a large fraction of the budget. The compute-optimal path instead grows encoder capacity and training data together while leaving the projector essentially fixed. Using that rule, the authors train large teachers and distill them into compact students that serve analysis-ready embeddings; the 21-million-parameter student leads a 29-task composite, and nested Matryoshka prefixes let users trade storage for accuracy without retraining. The practical recipe is: train a large encoder, select by downstream scores, and distill into flexible students.

Core claim

Within a fixed pixel-wise Barlow Twins family, pretraining loss is nearly independent of composite downstream score (|Pearson r| < 0.2), so loss-based selection needs roughly 254% more compute to match score-based selection. Compute-optimal allocation instead grows encoder size and data with budget (N★_enc ∝ C^0.36, D★ ∝ C^0.63) while projector size stays flat (N★_proj ∝ C^0.00). Following that rule and distilling a large teacher yields compact Matryoshka students that lead open and proprietary baselines on a 29-task suite.

What carries the argument

Downstream-driven scaling laws: 395 iso-FLOP pretraining runs, each scored on a 15-task composite, with quadratic fits per compute bucket and power laws through the vertices that prescribe how to split budget among encoder, projector, and data.

Load-bearing premise

The claim rests on treating the average of 15 chance-adjusted tasks as a general enough measure of 'downstream utility' that the resulting allocation rule is trustworthy beyond this one model family and evaluation suite.

What would settle it

Train a matched set of models selected by pretraining loss versus by the same 15-task composite, then measure whether the loss-selected models still need substantially more compute to match the score-selected models on a fresh held-out task battery that was never used for selection.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper reports a 395-run controlled scaling study of pixel-wise Barlow Twins EO encoders on 1,024 GH200s, each evaluated on 15 AlphaEarth-suite tasks. It claims that pretraining loss is nearly uncorrelated with downstream composite score (|Pearson r|≈0.18), so loss-based selection wastes roughly 254% more compute than score-based selection, and that compute-optimal allocation grows encoder size and data (N★_enc∝C^0.36, D★∝C^0.63) while the projector is essentially fixed (N★_proj∝C^0.00). Guided by this rule, the authors train large teachers (0.5B/1B) and distill them into compact N/S/M/L Matryoshka students; the 21M TESSERA v2-1B-M leads a 29-task composite (0.611) against open and proprietary baselines, with a 16-d prefix retaining ~92% of 128-d performance at 1/8 storage. Nested prefixes are argued to require distillation against a frozen teacher rather than self-supervised prefix losses, because redundancy reduction identifies subspaces only up to rotation.

Significance. If the empirical findings hold, this is a concrete, actionable production recipe for pixel-wise EO embeddings: spend pretraining budget on large encoders and matched data, select by downstream score not loss, and recover deployability via Matryoshka distillation. The scale of the controlled sweep (fixed family, iso-FLOP buckets, quadratic vertices, bootstrap and leave-one-bucket-out exponent stability, recovery of C∝N·D balance) is unusual in EO and strengthens the claim relative to prior loss-driven or architecture-entangled studies. The product contributions—analysis-ready students, storage-adaptive prefixes, planned global 2017–2025 embeddings, and code release—are of clear practical value. The distillation-vs-pretraining argument for nested coordinates is a useful technical insight for other embedding products. Limitations are stated honestly (family/objective/suite-specific laws).

major comments (3)
  1. §3.2–3.3 and Figure 2: The 254% compute-waste figure and the three power-law exponents are obtained from vertices of the same 15-task AlphaEarth composite. The manuscript itself notes that AlphaEarth edges TESSERA v1 on this suite but falls behind on the full 29-task suite (§5.1), so the composite is not invariant to task composition. Without leave-one-task-out or leave-one-group-out (e.g., drop fine-class/regression tasks where d=64 is preferred) re-fits of the per-bucket vertices, it is unclear whether the quantitative waste factor and the exponents N★_enc∝C^0.36, D★∝C^0.63, N★_proj∝C^0.00 are stable properties of the family or artefacts of a few high-variance tasks. A short sensitivity table would make F1–F2 load-bearing rather than suite-contingent.
  2. §3.1 Eq. (1) and §4: The encoder law is fit on a sweep that tops out at 278M parameters, then extrapolated to a 1B teacher (and a 2B model in training). The paper does not report intermediate points between 278M and 1B, nor a check that the C^0.36 exponent continues to hold in that regime. Because the production rule and the justification for the expensive teacher rest on this extrapolation, either (i) a small set of mid-scale confirmation runs or (ii) an explicit caveat that the 1B size is a rule-of-thumb extrapolation, not a measured optimum, is needed.
  3. §5.1 baseline comparison: The 29-task headline pits distilled embedding-as-data students (fixed lightweight heads, no backbone fine-tuning) against a mix of embedding products and RSFMs/generic backbones. For fairness, the manuscript should state more clearly which baselines are evaluated under the same frozen-embedding + light-head protocol versus full fine-tuning, and whether any RSFM scores use task-specific adaptation that the students are denied. Without that protocol table, the claim that the 21M student “outperforms all open and proprietary models tested, some of which are orders of magnitude larger” risks overstating like-for-like gains.
minor comments (5)
  1. Figure 2a: Report exact Pearson/Spearman values and sample size in the panel or caption; the abstract’s |r|<0.2 and the body r=−0.18 should be reconciled in one place.
  2. §3.1: The constant 12 and L_ref=240 in Eq. (1) deserve a one-sentence justification that variable L∈{8,16} training and adaptive inference buckets do not bias the iso-FLOP ranking across the grid.
  3. §5 / Figure 4d: “~92% of the d=128 composite” should state whether this is the mean over students, the M student only, or the best student, and whether it is unweighted or task-weighted.
  4. Typos and formatting: “TESSERAV2: SCALINGPIXEL-WISEEARTH FOUNDATIONMODELS” title spacing; “BARLOWTWINSfamily” missing spaces appear repeatedly in the preprint header; “F1prices the alternative” (§3.2) looks like a missing space/punctuation glitch.
  5. Reproducibility statement promises code, sweeps, and checkpoints; please confirm that the 395-run grid metadata (N_enc, N_proj, D, loss, 15-task scores) will be released as a table, not only training code, so others can re-fit the vertices.

Circularity Check

0 steps flagged

No significant circularity: scaling exponents, loss–downstream correlation, and student gains are measured against external task scores, not forced by construction from the pretraining objective or self-citation.

full rationale

The paper’s load-bearing claims are empirical measurements inside a fixed Barlow Twins family, not first-principles derivations that reduce to their inputs. Finding F1 (Pearson r ≈ −0.18 between converged Barlow Twins loss and the 15-task composite; ~254% extra compute for loss-based selection) is a direct comparison of two selection criteria on the same held-out task scores; the waste figure is a measured ratio of fitted power-law curves, not a tautology of the loss. Finding F2 (N★_enc ∝ C^0.36, D★ ∝ C^0.63, N★_proj ∝ C^0.00) is obtained by iso-FLOP quadratic vertices and power-law fits to those vertices (Eq. 1, Fig. 2); the exponents are reported with bootstrap CIs and leave-one-bucket-out checks, and the paper explicitly limits them to this family/objective/suite. The 1B teacher is an extrapolation of that measured rule, then frozen; students are trained by cosine reconstruction of the teacher embedding (Eq. 2), not by re-minimizing the pretraining loss, and are scored on both the AlphaEarth suite and 14 held-out datasets against external baselines (AlphaEarth, PRESTO, OlmoEarth, etc.). Self-citation of TESSERA v1 supplies only the architectural family and product framing; it does not underwrite the scaling exponents or the student superiority claims. Matryoshka nesting is justified by a negative result (prefix Barlow fails under rotational symmetry of redundancy reduction) plus distillation against a fixed target—again an empirical argument, not a definitional loop. Suite-specificity of the composite is a robustness/external-validity concern, not circularity under the stated patterns.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 3 invented entities

The central claims rest on empirical power-law fits inside one architectural family and on the assumption that a fixed 15-task composite is a valid selection target. Free parameters are the fitted exponents and the compute formula constants; axioms are standard SSL/distillation practice plus domain choices about pixel-wise Sentinel time series and evaluation design. Invented entities are product/method constructs (allocation rule, Matryoshka students, adaptive buckets), not new physical objects.

free parameters (5)
  • Encoder scaling exponent α_enc = 0.36
    Fitted as N★_enc ∝ C^0.36 from iso-FLOP vertices; 95% CI [0.29, 0.45]. Load-bearing for the 'grow encoder with compute' rule.
  • Data scaling exponent α_D = 0.63
    Fitted as D★ ∝ C^0.63; CI [0.53, 0.70]. Paired with α_enc to justify matched data growth.
  • Projector scaling exponent α_proj = 0.00
    Fitted as N★_proj ∝ C^0.00; CI [−0.03, 0.08]. Justifies holding projector fixed.
  • Compute formula constant and L_ref = 12; L_ref=240
    C = 12 D (N_enc L_ref + N_proj) with L_ref = 240 and 12 = 6×2 from textbook FLOPs and two Barlow views; defines the x-axis of all scaling fits.
  • Iso-FLOP bucket quadratic vertices
    Per-bucket optimal sizes are vertices of quadratics in log N; these discrete peaks are the data points for the power laws.
axioms (5)
  • domain assumption Within a fixed architecture family, iso-FLOP sweeps and power-law fits to downstream composite scores yield a transferable compute-allocation rule.
    Section 3 frames the study as family-specific, not universal EO laws; still used as the production recipe for the 1B teacher.
  • domain assumption Averaged chance-adjusted metrics over the 15 AlphaEarth-suite tasks are a valid scalar proxy for embedding utility.
    Section 3.1 defines the composite y-axis used for all vertex selection and F1/F2 claims.
  • domain assumption Barlow Twins redundancy reduction on temporally subsampled Sentinel-1/2 d-pixels is an appropriate pretraining objective for the family under study.
    Inherited from TESSERA v1 / Lisaius et al.; fixed before the scaling sweep.
  • ad hoc to paper Knowledge distillation against a frozen teacher can impose coordinate order that self-supervised prefix losses cannot, because redundancy reduction identifies subspaces only up to rotation.
    Section 5 argument for why Matryoshka must be learned via distillation rather than prefix Barlow losses.
  • standard math Standard FLOPs accounting (≈6ND, two views) and quadratic-in-logN optima are valid for locating compute-optimal sizes.
    Equation (1) and Section 3.1 methodology.
invented entities (3)
  • Downstream-driven EO compute-allocation rule (encoder+data grow, projector fixed) no independent evidence
    purpose: Replace loss-based model selection and guide teacher training budget.
    Empirical construct fitted on the 395-run sweep; independent evidence is the reported exponents and stability checks, not external theory.
  • TESSERA v2 Matryoshka student family (N/S/M/L with d∈{16,32,64,128}) independent evidence
    purpose: Deployable embeddings-as-data with storage/accuracy knobs.
    Product of distillation; performance claims are benchmark-relative, not a new physical entity.
  • Adaptive bucket sampling at inference independent evidence
    purpose: Pack variable-length valid observations without discarding data as in v1 fixed-L sampling.
    Engineering mechanism in Section 4; falsifiable via ablation on artifact/quality metrics.

pith-pipeline@v1.1.0-grok45 · 19867 in / 3940 out tokens · 28511 ms · 2026-07-11T22:47:57.208613+00:00 · methodology

0 comments
read the original abstract

Pixel-wise Earth-observation (EO) foundation models are now achieving state-of-the-art performance via generated spatial embeddings. However, how these models scale and how best to spend a pretraining budget remain poorly understood. We present the largest controlled scaling study for EO to date: 395 training runs on 1,024 GH200 superchips within a fixed pixel-wise Barlow Twins family, each evaluated on 15 downstream tasks. We find that pretraining loss barely predicts downstream performance (|Pearson r| < 0.2), so selecting models by loss wastes a large share of the compute. We also find that, as the training budget grows, the encoder and the data should grow together while the projector stays fixed, which gives a simple rule for allocating compute. Using this rule, we train a family of pixel-wise models (0.5B and 1B, with a 2B model in training) and distill them into compact students for embeddings-as-data deployment. The 21-million-parameter distilled TESSERA v2-1B-M in aggregate outperforms all open and proprietary models tested, some of which are orders of magnitude larger. These students produce Matryoshka representations that are inexpensive to serve: a 16-dimensional prefix keeps 92% of the full 128-dimensional performance at 1/8 of the storage. Upon completion of training we plan to release v2 global embeddings covering 2017-2025. Together, these results give a concrete, empirically grounded recipe for scaling pixel-wise EO foundation models: train large encoders, select by downstream performance, and distil into flexible student models. All code will be released at https://github.com/ucam-eo/tessera.

Figures

Figures reproduced from arXiv: 2607.03949 by Andrew Blake, Aneesh Naik, Anil Madhavapeddy, Clement Atzberger, David Coomes, Ira Shokar, Jovana Knezevic, Mark Elvers, Niall Robinson, Robin Young, Sadiq Jaffer, Srinivasan Keshav, Zhengpeng Feng.

Figure 1
Figure 1. Figure 1: Overview of TESSERA v2. (a) Sparse, irregular Sentinel-2/Sentinel-1 sampling at one location. (b) 395-run downstream-driven scaling sweep and compute-optimal fits; inset: pretraining loss vs. downstream score. (c) MATRYOSHKA distillation of the 1 B teacher into N/S/M/L students, d ∈ {16, 32, 64, 128}. (d) Artefact removal and inter-annual stability vs. v1. (e) Deployment via GEOTESSERA. (f) Leading composi… view at source ↗
Figure 2
Figure 2. Figure 2: On the same suite, TESSERA v1 scores 0.541 and ALPHAEARTH 0.560. Both are baselines in Figure 2a–c. Both baselines are full-budget production systems, whereas the sweep grid deliberately spans many small, data-limited configurations in order to trace out the compute frontier; most individual runs therefore fall below the baselines, while the upper envelope of the sweep approaches them. For every run we als… view at source ↗
Figure 3
Figure 3. Figure 3: TESSERA v2 architecture. (a) Pretraining: two views per d-pixel at random length L∈ {8, 16}, per-modality Transformers with day-of-year encoding, cross-modal fusion, and BARLOW TWINS + mix-up. (b) Distillation: MATRYOSHKA prefix heads at d∈ {16, 32, 64, 128} reconstruct the frozen teacher embedding. (c) Inference: each pixel’s k valid observations are packed into the smallest bucket B⋆ ≥k, with residual sl… view at source ↗
Figure 4
Figure 4. Figure 4: ALPHAEARTH suite results (15 tasks) and held-out generalisation (14 datasets). (a) Ge￾ographic density of all downstream labels. (b) Per-task heatmap with mean rank. (c) Composite score per model; the four markers per TESSERA v2 row are d ∈ {16, 32, 64, 128}. (d) Score vs. prefix dimension: d=16 retains ∼92% of the d=128 composite at 1/8 of the storage. (e) Score vs. global annual storage, relative to ALPH… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

30 extracted references · 4 canonical work pages

  1. [1]

    org/abs/2412.14123

    URL https://arxiv. org/abs/2412.14123. Wele Gedara Chaminda Bandara, Celso M. de Melo, and Vishal M. Patel. Guarding Barlow Twins against overfitting with mixed samples.arXiv preprint arXiv:2312.02151,

    Pith/arXiv arXiv

  2. [2]

    URL https: //arxiv.org/abs/2507.22291. Shuang Chen, Jie Wang, Shuai Yuan, Jiayang Li, Yu Xia, Yuanhong Liao, Junbo Wei, Jincheng Yuan, Xiaoqing Xu, Xiaolin Zhu, Peng Zhu, Hongsheng Zhang, Yuyu Zhou, Haohuan Fu, Huabing Huang, Bin Chen, Fan Dai, and Peng Gong. Democratizing planetary-scale analysis: An ultra- lightweight Earth embedding database for accura...

    Pith/arXiv arXiv

  3. [3]

    URL https://essd

    doi: 10.5194/essd-2026-57. URL https://essd. copernicus.org/preprints/essd-2026-57/. Preprint. Yezhen Cong, Samar Khanna, Chenlin Meng, Patrick Liu, Erik Rozi, Yutong He, Marshall Burke, David B. Lobell, and Stefano Ermon. SatMAE: Pre-training transformers for temporal and multi- spectral satellite imagery. InAdvances in Neural Information Processing Syst...

    work page doi:10.5194/essd-2026-57 2026

  4. [4]

    Scaling laws for geospatial foundation models: A case study on PhilEO bench.arXiv preprint arXiv:2506.14765,

    Nikolaos Dionelis, Riccardo Musto, Jente Bosmans, Simone Sarti, Giancarlo Paoletti, Peter Naylor, Valerio Marsocci, Sébastien Lefèvre, Bertrand Le Saux, and Nicolas Longépé. Scaling laws for geospatial foundation models: A case study on PhilEO bench.arXiv preprint arXiv:2506.14765,

    arXiv

  5. [5]

    Stewart, Isaac Corley, Xiao Xiang Zhu, and Hossein Azizpour

    Heng Fang, Adam J. Stewart, Isaac Corley, Xiao Xiang Zhu, and Hossein Azizpour. Earth embeddings as products: Taxonomy, ecosystem, and standardized access.arXiv preprint arXiv:2601.13134,

    arXiv

  6. [7]

    11 Preprint

    doi: 10.1109/ CVPR52733.2024.02631. 11 Preprint. Henry Herzog, Favyen Bastani, Yawen Zhang, Gabriel Tseng, Joseph Redmon, Hadrien Sablon, Ryan Park, Jacob Morrison, Alexandra Buraczynski, Karen Farley, et al. Olmoearth: Stable latent image modeling for multimodal earth observation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Re...

    arXiv 2024

  7. [8]

    Michael Hu et al

    doi: 10.20944/preprints202605.0981.v1. Michael Hu et al. Scaling laws are unreliable for downstream tasks: A reality check. InFindings of the Association for Computational Linguistics: EMNLP 2025,

    work page doi:10.20944/preprints202605.0981.v1 2025

  8. [9]

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B

    arXiv:2507.00885. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361,

    arXiv 2001

  9. [10]

    Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ra- manujan, William Howard-Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, and Ali Farhadi

    doi: 10.31223/X5HX9S. Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ra- manujan, William Howard-Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, and Ali Farhadi. Matryoshka representation learning. InAdvances in Neural Information Processing Systems (NeurIPS),

    doi:10.31223/x5hx9s

  10. [11]

    CITYREP: A unified benchmark for urban representations across cities, tasks, and modalities.arXiv preprint arXiv:2605.26036,

    Junyuan Liu, Xinglei Wang, Zichao Zeng, Jiazhuang Feng, Quan Qin, Ilya Ilyankou, Guangsheng Dong, and Tao Cheng. CITYREP: A unified benchmark for urban representations across cities, tasks, and modalities.arXiv preprint arXiv:2605.26036,

    Pith/arXiv arXiv

  11. [12]

    SkySenseGPT: A fine-grained instruction tuning dataset and model for remote sensing vision-language understanding.arXiv preprint arXiv:2406.10100,

    Junwei Luo, Zhen Pang, Yongjun Zhang, Tingzhu Wang, Linlin Wang, Bo Dang, Jiangwei Lao, Jian Wang, Jingdong Chen, Yihua Tan, et al. SkySenseGPT: A fine-grained instruction tuning dataset and model for remote sensing vision-language understanding.arXiv preprint arXiv:2406.10100,

    Pith/arXiv arXiv

  12. [13]

    Structure-semantic decoupled modulation of global geospatial embeddings for high-resolution remote sensing mapping.arXiv preprint arXiv:2604.19591,

    Jienan Lyu, Miao Yang, Jinchen Cai, Yiwen Hu, Guanyi Lu, Junhao Qiu, and Runmin Dong. Structure-semantic decoupled modulation of global geospatial embeddings for high-resolution remote sensing mapping.arXiv preprint arXiv:2604.19591,

    Pith/arXiv arXiv

  13. [14]

    Anil Madhavapeddy, Frank Feng, Robin Young, Janne Mäyrä, Sadiq Jaffer, Olli Niemitalo, Srini- vasan Keshav, and David Allsopp

    doi: 10.1109/ICCV48922.2021.00928. Anil Madhavapeddy, Frank Feng, Robin Young, Janne Mäyrä, Sadiq Jaffer, Olli Niemitalo, Srini- vasan Keshav, and David Allsopp. ucam-eo/geotessera,

    doi:10.1109/iccv48922.2021.00928 2021

  14. [15]

    Daniel B

    doi: 10.1109/ICCV51070.2023.01541. Daniel B. Metcalfe, Emily Anders, Hanna Axén, et al. Gaps in tropical science from unrepre- sentative distribution of sampling and citation across natural terrestrial environments.Nature Communications,

    doi:10.1109/iccv51070.2023.01541 2023

  15. [16]

    Andrea Nascetti, Ritu Yadav, Kirill Brodt, Qixun Qu, Hongwei Fan, Yuri Shendryk, Isha Shah, and Christine Chung

    doi: 10.1038/s41467-025-67617-4. Andrea Nascetti, Ritu Yadav, Kirill Brodt, Qixun Qu, Hongwei Fan, Yuri Shendryk, Isha Shah, and Christine Chung. Biomassters: A benchmark dataset for forest biomass estimation using multi- modal satellite time-series.Advances in Neural Information Processing Systems, 36:20409–20420,

    work page doi:10.1038/s41467-025-67617-4

  16. [17]

    Yohann Perron, Guillaume Astruc, Nicolas Gonthier, Clement Mallet, and Loic Landrieu

    doi: 10.1109/CVPR52733.2024.02627. Yohann Perron, Guillaume Astruc, Nicolas Gonthier, Clement Mallet, and Loic Landrieu. Uni- versat: Resolution- and modality-agnostic transformers for earth observation.arXiv preprint arXiv:2606.23503,

    doi:10.1109/cvpr52733.2024.02627 2024

  17. [18]

    Esther Rolf, Jonathan Proctor, Tamma Carleton, Ian Bolliger, Vaishaal Shankar, Miyabi Ishihara, Benjamin Recht, and Solomon Hsiang

    doi: 10.1109/ICCV51070.2023.00378. Esther Rolf, Jonathan Proctor, Tamma Carleton, Ian Bolliger, Vaishaal Shankar, Miyabi Ishihara, Benjamin Recht, and Solomon Hsiang. A generalizable and accessible approach to machine learning with global satellite imagery.Nature Communications, 12(1):4392,

    doi:10.1109/iccv51070.2023.00378 2023

  18. [19]

    Johannes Schmude, Sujit Roy, Will Trojak, Johannes Jakubik, Daniel Salles Civitarese, Shraddha Singh, et al

    doi: https://doi.org/10.1016/j.isprsjprs.2022.03.012. Johannes Schmude, Sujit Roy, Will Trojak, Johannes Jakubik, Daniel Salles Civitarese, Shraddha Singh, et al. Prithvi-WxC: Foundation model for weather and climate,

    work page doi:10.1016/j.isprsjprs.2022.03.012 2022

  19. [20]

    Xian Sun, Peijin Wang, Wanxuan Lu, Zicong Zhu, Xiaonan Lu, Qibin He, Junxi Li, Xuee Rong, Zhujun Yang, Hao Chang, Qinglin He, Guang Yang, Ruiping Wang, Jiwen Lu, and Kun Fu

    URL https: //arxiv.org/abs/2409.13598. Xian Sun, Peijin Wang, Wanxuan Lu, Zicong Zhu, Xiaonan Lu, Qibin He, Junxi Li, Xuee Rong, Zhujun Yang, Hao Chang, Qinglin He, Guang Yang, Ruiping Wang, Jiwen Lu, and Kun Fu. RingMo: A remote sensing foundation model with masked image modeling.IEEE Transactions on Geoscience and Remote Sensing, 61:1–22,

    Pith/arXiv arXiv

  20. [21]

    Daniela Szwarcman, Sujit Roy, Paolo Fraccaro, Þorsteinn Eli Gislason, Benedikt Blumenstiel, Rinki Ghosal, et al

    doi: 10.1109/TGRS.2022.3194732. Daniela Szwarcman, Sujit Roy, Paolo Fraccaro, Þorsteinn Eli Gislason, Benedikt Blumenstiel, Rinki Ghosal, et al. Prithvi-EO-2.0: A versatile multi-temporal foundation model for earth observation applications,

    doi:10.1109/tgrs.2022.3194732 2022

  21. [22]

    Maofeng Tang, Andrei Cozma, Konstantinos Georgiou, and Hairong Qi

    URLhttps://arxiv.org/abs/2412.02732. Maofeng Tang, Andrei Cozma, Konstantinos Georgiou, and Hairong Qi. Cross-scale MAE: A tale of multiscale exploitation in remote sensing.Advances in Neural Information Processing Systems (NeurIPS), 36:20054–20066,

    arXiv

  22. [23]

    Gabriel Tseng, Anthony Fuller, Marlena Reil, Henry Herzog, Patrick Beukema, Favyen Bastani, James R

    URL https:// arxiv.org/abs/2304.14065. Gabriel Tseng, Anthony Fuller, Marlena Reil, Henry Herzog, Patrick Beukema, Favyen Bastani, James R. Green, Evan Shelhamer, Hannah Kerner, and David Rolnick. Galileo: Learning global & local features of many remote sensing modalities,

    Pith/arXiv arXiv

  23. [24]

    Rikard Vinge, Isabelle Wittmann, Jannik Schneider, Michael Marszalek, Luis Gilch, Thomas Brun- schwiler, and Conrad M

    URL https://arxiv.org/abs/ 2502.09356. Rikard Vinge, Isabelle Wittmann, Jannik Schneider, Michael Marszalek, Luis Gilch, Thomas Brun- schwiler, and Conrad M. Albrecht. Neuco-bench: A novel benchmark framework for neural embeddings in earth observation,

    Pith/arXiv arXiv

  24. [25]

    Yi Wang, Conrad M

    arXiv:2510.17914. Yi Wang, Conrad M. Albrecht, and Xiao Xiang Zhu. Self-supervised vision transformers for joint SAR-optical representation learning,

    arXiv

  25. [26]

    Yuelei Wang, Ting Zhang, Liangjin Zhao, Lin Hu, Zhechao Wang, Ziqing Niu, Peirui Cheng, Kaiqiang Chen, Xuan Zeng, Zhirui Wang, Hongqi Wang, and Xian Sun

    URL https://arxiv.org/abs/2204.05381. Yuelei Wang, Ting Zhang, Liangjin Zhao, Lin Hu, Zhechao Wang, Ziqing Niu, Peirui Cheng, Kaiqiang Chen, Xuan Zeng, Zhirui Wang, Hongqi Wang, and Xian Sun. RingMo-lite: A remote sensing multi-task lightweight network with CNN-transformer hybrid framework,

    Pith/arXiv arXiv

  26. [27]

    Xinye Wanyan, Sachith Seneviratne, Shuchang Shen, and Michael Kirley

    URL https://arxiv.org/abs/2309.09003. Xinye Wanyan, Sachith Seneviratne, Shuchang Shen, and Michael Kirley. Extending global-local view alignment for self-supervised learning with remote sensing imagery,

    Pith/arXiv arXiv

  27. [28]

    Charith Wickrema, Eliza Mace, Hunter Brown, Heidys Cabrera, Nick Krall, Matthew O’Neill, Shivangi Sarkar, Lowell Weissman, Eric Hughes, and Guido Zarrella

    URL https: //arxiv.org/abs/2303.06670. Charith Wickrema, Eliza Mace, Hunter Brown, Heidys Cabrera, Nick Krall, Matthew O’Neill, Shivangi Sarkar, Lowell Weissman, Eric Hughes, and Guido Zarrella. Scaling remote sensing foun- dation models: Data domain tradeoffs at the peta-scale. InApplied Imagery Pattern Recognition Workshop, pp. 132–153. Springer,

    Pith/arXiv arXiv

  28. [29]

    Stewart, Joëlle Hanna, Damian Borth, Ioannis Papoutsis, Bertrand Le Saux, Gustau Camps-Valls, and Xiao Xiang Zhu

    Zhitong Xiong, Yi Wang, Fahong Zhang, Adam J. Stewart, Joëlle Hanna, Damian Borth, Ioannis Papoutsis, Bertrand Le Saux, Gustau Camps-Valls, and Xiao Xiang Zhu. Neural plasticity-inspired multimodal foundation model for earth observation.arXiv preprint arXiv:2403.15356,

    arXiv

  29. [30]

    Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny

    doi: 10.1109/TGRS.2023.3316166. Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow Twins: Self-supervised learning via redundancy reduction.arXiv preprint arXiv:2103.03230,

    doi:10.1109/tgrs.2023.3316166 2023

  30. [31]

    Yingying Zhang, Lixiang Ru, Kang Wu, Lei Yu, Lei Liang, Yansheng Li, and Jingdong Chen

    14 Preprint. Yingying Zhang, Lixiang Ru, Kang Wu, Lei Yu, Lei Liang, Yansheng Li, and Jingdong Chen. SkySense V2: A unified foundation model for multi-modal remote sensing.arXiv preprint arXiv:2507.13812,

    Pith/arXiv arXiv