From Surface Forecasting to Observability Forecasting: A Latent World Model for Cloud-Aware EO Monitoring

Mohanad Albughdadi

REVIEW 3 major objections 5 minor 18 references

The paper reframes Earth observation monitoring as forecasting when the ground is visible, and shows a latent world model beats persistence at predicting the next usable acquisition and its return time.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 04:32 UTC pith:R3637XZR

load-bearing objection Honest, well-scoped empirical study that reframes EarthNet2021 as observability forecasting; LeWM beats persistence and mostly beats a frozen LightGBM on exact timing, though the global-latent limitation and single-run reporting should be named. the 3 major comments →

arxiv 2607.13651 v1 pith:R3637XZR submitted 2026-07-15 cs.CV

From Surface Forecasting to Observability Forecasting: A Latent World Model for Cloud-Aware EO Monitoring

Mohanad Albughdadi This is my paper

classification cs.CV

keywords observability forecastingcloud-aware monitoringlatent world modeljoint-embedding predictive architectureEarth observationEarthNet2021linear probingcloud occlusion

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper reframes Earth observation monitoring as an observability forecasting problem: given recent multispectral imagery and weather drivers, predict whether the next acquisition will be cloud-free enough to use and, if not, when a usable view returns. On the EarthNet2021 benchmark, the authors adapt a latent world model (LeWorldModel) that predicts future latent states rather than pixels, then fit linear probes on those predicted latents under a locked protocol. The model consistently beats the persistence baseline on next-step usability (balanced accuracy 0.769–0.887 versus 0.493–0.556) and on exact first-usable-horizon prediction (0.602–0.806 versus 0.120–0.369), and beats a frozen tabular gradient-boosting baseline on continuous cloud/clear regression and recovery timing on most splits. The paper's central point is that a latent temporal state, not image reconstruction, is the right carrier of cloud-aware monitoring signals.

Core claim

The central claim is that the useful signal for cloud-aware Earth observation is the temporal state of scene visibility, not pixel-level reconstruction. The authors adapt LeWorldModel, a joint-embedding predictive architecture, to EarthNet2021 by keeping all frames, adding the binary cloud mask as a fifth image channel, and supplying eight weather and calendar covariates. The model maps each observation to a 192-dimensional latent state via the encoder class token, predicts future latents with a conditional autoregressive transformer over a ten-step teacher-forced window, and is regularized toward an isotropic Gaussian geometry. Linear probes trained only on the training split then recover n

What carries the argument

The central object is LeWorldModel, a joint-embedding predictive architecture (JEPA) world model. It encodes each 128×128 five-channel frame (blue, green, red, near-infrared, binary cloud mask) with a ViT-tiny encoder, keeps only the class token, and projects it to a 192-dimensional latent state; an eight-dimensional covariate vector (time gap, day-of-year cycles, precipitation, pressure, temperatures) is embedded to the same width. A six-layer conditional autoregressive transformer predicts the next latent given the past ten latents and covariates, with teacher-forced one-step targets, plus an isotropic-Gaussian regularizer. The mechanism that carries the argument is latent-space forecastin

Load-bearing premise

The load-bearing premise is that the model's single global latent state per frame preserves enough spatial cloud detail for the downstream probes to recover usability and recovery timing; if the latent discards where clouds are located, the observability forecast loses accuracy no matter how well the temporal dynamics are learned.

What would settle it

Fit the same linear probes directly on the last observed latent state instead of on the predicted future latent and compare first-usable-horizon accuracy; if accuracy does not drop, the temporal prediction component contributes nothing and the reported gains come from static features.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

EO processing chains could use the next-step usability estimate to decide whether to spend compute and attention on a new acquisition or defer analysis.
When the next view is unusable, the predicted first-usable horizon gives an operational wait time, avoiding repeated failed processing attempts.
The ranking-based anomaly signal can serve as a temporal-consistency check to flag frames that do not fit the recent context for quality control.
On the extreme split, which has the most clouds and the longest sequences, the model still holds a timing advantage over persistence and the tabular baseline, indicating the latent state tracks recovery dynamics beyond easy near-IID structure.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If the global class-token latent truly preserves the temporal state of visibility, extending the architecture to emit patch-level latents could sharpen spatial cloud-cover forecasts and recovery timing without changing the latent-prediction objective.
The paper reports point predictions; an operational deployment would likely need calibrated probabilities for 'usable next' and for the recovery horizon, which the current evaluation does not provide.
The OOD drop suggests a concrete extension: conditioning the latent dynamics on weather-regime or geography could reduce distribution shift and improve out-of-distribution robustness.
The observability-forecasting framing transfers naturally to any revisit-constrained sensor — optical, SAR, or fused — where the usable cadence is set by obscuration rather than data arrival.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Honest, well-scoped empirical study that reframes EarthNet2021 as observability forecasting; LeWM beats persistence and mostly beats a frozen LightGBM on exact timing, though the global-latent limitation and single-run reporting should be named.

read the letter

Quick take: this is a solid, narrowly scoped empirical paper. The new thing is the problem framing — observability forecasting (will next acquisition be usable, when will a usable view return) rather than surface forecasting — and a careful locked evaluation on EarthNet2021 with a JEPA world model. It doesn't claim a new architecture; it adapts LeWorldModel and is upfront about that. The protocol is a real strength: probes fit on train, calibration on valsplit, frozen heads for IID/OOD/extreme. That's the right way to avoid split-specific adaptation, and the code/weights are provided.

The results are believable as far as they go. LeWM beats persistence everywhere, which is not shocking since persistence is weak here. The more informative comparison is against the frozen LightGBM baseline, and the paper honestly reports a task split: LightGBM wins on the simple binary any-usable-within-six and on OOD, while LeWM wins on continuous clear/cloud regression and on exact first-usable timing on valsplit/IID/extreme. The extreme split result, where the model holds a timing advantage despite much cloudier and longer episodes, is the most convincing evidence that the latent captures something useful about recovery dynamics.

Soft spots, in proportion. First, there is a single checkpoint, no error bars or repeated runs. For a benchmark with ~342k valsplit windows the point estimates are probably stable, but you can't rule out that some of the smaller margins (IID usable-next tie, OOD timing gap) would flip with a different seed. Second, the valsplit is used to set calibration thresholds, so those numbers should be read as calibration diagnostics; the IID/extreme comparisons are the independent evidence. Third, the acknowledged global-latent limitation is real: the class token may discard spatial cloud structure, and the paper doesn't prove otherwise. That said, the targets are global clear/valid fractions, so the stress-test is right that this is not a fatal flaw — it just bounds what the model can claim. Fourth, the anomaly-detection section is synthetic and on sampled windows; it's a supporting diagnostic, not a shipped capability.

Overall: the paper is honest, the claims are scoped, and the failure modes are discussed rather than hidden. The central claim — that a latent world model provides a useful observability-forecasting signal beyond persistence and a strong tabular baseline on most splits — holds up. I'd send this to peer review; it's a good benchmark contribution for the cloud-aware EO monitoring community, even though it's not a broad scientific breakthrough.

Referee Report

3 major / 5 minor

Summary. The paper reframes EarthNet2021 from surface forecasting to observability forecasting: given recent multispectral imagery and weather covariates, the goal is to predict whether the next acquisition will be usable, whether any usable view appears within six steps, and when the first usable view returns. The authors adapt LeWorldModel (a JEPA-style latent predictor) to 5-channel image patches plus 8 covariates, train from scratch, and attach linear probes under a locked protocol: probes fit on train, calibration on an internal valsplit, then frozen for IID, OOD, and extreme evaluation. They report consistent improvements over persistence on all observability tasks, and better continuous clear/cloud regression and exact first-usable timing than a frozen LightGBM baseline on valsplit, IID, and extreme, with LightGBM stronger on the binary any-usable-within-six task and on OOD. Additional diagnostics cover latent forecasting, latent geometry, anomaly ranking, and weather sensitivity.

Significance. The paper's strengths are its clean problem formulation, the locked frozen-probe evaluation protocol, comparisons to persistence and a strong tabular baseline, use of a public dataset, and the release of code, weights, and a reproducible pipeline. The honest disaggregation — showing where LightGBM wins and where LeWM degrades on OOD — adds credibility. The acknowledged global-not-spatial limitation does not undercut the main claim, because the observability targets are global clear/valid fractions; a 192-d class-token latent can carry that state, and the tasks do not require spatial prediction. If the empirical claims hold, the model provides an operationally relevant signal for cloud-aware EO monitoring. However, the overlap between the calibration split and the reported 'valsplit' evaluation, and the absence of any uncertainty quantification, mean the paper as written overstates parts of its headline result.

major comments (3)

[§7.2, Table 5] The protocol tunes calibration choices on valsplit and then reports valsplit as an evaluation split in the frozen-bundle benchmark. Under the stated protocol, valsplit is not a clean held-out test: thresholds are selected on it, so its numbers are optimistically biased. The abstract and conclusion claim gains 'on valsplit, IID, and extreme,' so the valsplit part of that claim is internally inconsistent with the protocol. The IID and extreme results remain valid, but the paper should either present valsplit only as calibration diagnostics or create a true held-out calibration split (e.g., by partitioning the training HDF5 further) and re-run the benchmark.
[§8.4–8.5, Tables 5, 13–17] All results come from a single LeWM checkpoint with no error bars, confidence intervals, or seed variability. The exact-timing differences on IID (0.787 vs 0.762) and extreme (0.705 vs 0.633) have large effective sample sizes, so they are likely real, but the absence of any uncertainty measure makes the magnitudes hard to assess and violates standard practice for empirical comparisons. Please provide bootstrap confidence intervals over windows, or at least multiple seeds for the main benchmark.
[§7.2, §7.4] The manuscript does not explicitly state whether the linear probes for the rollout-based tasks (any-usable-within-six, first-usable horizon) are trained on autoregressive rollout latents from training windows or on teacher-forced one-step latents. Since evaluation uses six-step rollout latents, a train/eval latent-distribution mismatch would bias the results. Please clarify and, if the probes were trained on one-step latents, re-fit them on rollout latents before finalizing the claims.

minor comments (5)

[Title/Abstract/§5] Inconsistent capitalization: the abstract uses 'leWorldModel' while the body generally uses 'LeWM' and 'LeWorldModel'. Please standardize.
[§5.2] Define the dimensions of z_t and a_t (ℝ^192) in the main text before first use; the notation is currently introduced only via the equation.
[Figures 2–4] The 'MSE ratio' axes should specify the persistence denominator in the figure labels, not only in the caption, to avoid ambiguity.
[Table 5] Include the number of evaluation windows n in the table or caption, since the splits differ substantially in size (e.g., valsplit vs extreme) and some metrics are sensitive to support.
[§8.6, Table 18] The text says temperature perturbations produce smaller effects, but on OOD the temperature deltas (cooler +0.0020, warmer −0.0015) are larger than the rain deltas (−0.0001/−0.0005). Please qualify this claim.

Circularity Check

0 steps flagged

No significant circularity: fully empirical locked-protocol comparison against external baselines.

full rationale

The paper's central claims are empirical comparisons on a public benchmark (EarthNet2021) against persistence and a frozen LightGBM baseline, under a locked protocol: LeWM is trained once, linear probes are fit on train windows only, calibration choices are set on valsplit, and the fitted heads are frozen for valsplit, IID, OOD, and extreme evaluation (Section 7.2). The observability targets are defined directly from the data (e.g., 'A step is usable when both clear fraction and valid fraction are at least 0.8'), not from model parameters, so there is no self-definitional loop. The latent forecasting result is not circular: the training objective L_pred = (1/10D) sum || zhat_{t+1} - z_{t+1} ||^2 compares predicted latents to encoded latents, while persistence merely copies the last latent state; beating persistence is a substantive empirical finding. The only self-citation is to the author's earlier lightweight MoE masked autoencoder [13], which appears in related work as context and is not load-bearing for any result. The acknowledged limitation that the model is 'global, not spatial' is a stated boundary, not a hidden circularity. Valsplit numbers are calibration diagnostics as the paper itself notes, but the IID and extreme comparisons are independent of calibration and show the same direction. No fitted parameter is renamed as a prediction, and no uniqueness or ansatz is imported from self-citation. The paper is self-contained against external benchmarks, so the circularity score is 0.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The model introduces no new physical entities. The free parameters are standard hyperparameters plus the usability definition. The main load-bearing assumption is that a global latent state preserves the cloud-structure information needed for recoverability probes.

free parameters (3)

usability_threshold = 0.8
The binary usable label is defined as min(clear fraction, valid fraction) ≥ 0.8. This threshold determines all labels and is a modeling choice, not fitted.
calibration_threshold_or_method = tuned on valsplit (actual value unspecified)
The locked protocol tunes minor calibration choices on the internal validation split, freezing them for IID/OOD/extreme. The exact calibration (threshold/platt scaling) is not detailed in the main text.
lewm_hyperparameters = history=10, λ_sig=0.05, lr=3e-4, epochs=24, latent_width=192, batch=32
Training hyperparameters chosen without systematic search or sensitivity analysis; they affect the single reported checkpoint.

axioms (4)

domain assumption EarthNet2021's binary cloud mask and valid fraction correctly represent ground-truth cloud presence and observation validity.
All observability targets are derived from these fields; if the cloud mask is noisy, the labels are noisy.
domain assumption A step is operationally 'usable' when both clear fraction and valid fraction are at least 0.8.
This is a definitional choice. Different thresholds could change the ranking of models.
domain assumption Future meteorological covariates and calendar features are available at inference time.
The experiments condition on the future weather from EarthNet2021; the model does not forecast these covariates (Section 9).
domain assumption The LeWM JEPA training objective (1-step teacher-forced latent prediction + SIGReg) creates a latent representation sufficient for linear probes of observability.
The paper relies on this without a formal guarantee; it is an architectural assumption inherited from LeWM [2].

pith-pipeline@v1.3.0-alltime-deepseek · 16012 in / 11537 out tokens · 101293 ms · 2026-08-02T04:32:14.066720+00:00 · methodology

0 comments

read the original abstract

The bottleneck of Earth Observation processing chains is not the arrival of new imagery but whether the surface is actually visible when the image arrives. We study this as an observability forecasting problem on EarthNet2021. Given recent multispectral imagery and exogenous weather drivers, the goal is to predict whether the next acquisition will be usable and, if not, when a usable view is likely to return. To do this, we adapt LeWorldModel, a joint-embedding predictive architecture world model, to cloud-aware Earth Observation sequences. The final pipeline converts raw minicubes into episodic HDF5 sequences with five image channels (blue, green, red, near-infrared, cloud mask) and eight meteorological and calendar covariates. The resulting model has 18.0M trainable parameters and is trained from scratch on 23,904 training episodes. The trained leWorldModel is evaluated under a locked protocol: linear probes are fit on train only, calibration choices are set on an internal validation split, and the fitted heads are then frozen for valsplit, IID, OOD, and extreme evaluation. On the full frozen-bundle observability benchmark, LeWorldModel consistently outperforms persistence. For next-step usability, balanced accuracy ranges from 0.769 to 0.887, compared with 0.493 to 0.556 for persistence. For exact first-usable-horizon prediction, accuracy ranges from 0.602 to 0.806, compared with 0.120 to 0.369 for persistence. Against a frozen LightGBM baseline fit on the same training windows, LeWorldModel is better on continuous clear/cloud regression and on exact recovery timing on valsplit, IID, and extreme, while LightGBM is stronger on the simpler binary any-usable-within-six task and is more robust on OOD. In separate sampled diagnostic analyses, LeWM also produces strong ranking-based anomaly signals under synthetic temporal inconsistencies.

Figures

Figures reproduced from arXiv: 2607.13651 by Mohanad Albughdadi.

**Figure 1.** Figure 1: , plotted from the final training log, shows that training is stable and converges smoothly. Both the total loss and the latent prediction loss decrease throughout training, while the validation regularization term remains controlled. The final validation prediction loss is 0.0164 and the final validation SIGReg term is 1.83 [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗

**Figure 2.** Figure 2: Sampled latent-forecast diagnostics across splits from the executed notebooks. Left: mean-squared-error ratio [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 3.** Figure 3: Detailed next-step diagnostics from a fixed sampled subset of the [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

**Figure 4.** Figure 4: Detailed rollout diagnostics from a fixed sampled subset of the extreme split. LeWM maintains lower rollout [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗

**Figure 5.** Figure 5: Latent-geometry diagnostics from a fixed sampled subset of [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗

**Figure 6.** Figure 6: Sampled ranking-based anomaly detection from fixed notebook windows under synthetic future inconsisten [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗

**Figure 7.** Figure 7: Full frozen-bundle benchmark for next-step observation quality. LeWM is consistently better than persistence [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗

**Figure 8.** Figure 8: Full frozen-bundle comparison on cloud-aware rollout monitoring tasks. Top row: binary prediction of [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗

**Figure 9.** Figure 9: Normalized support of the first-usable-horizon target in the full frozen-bundle evaluation windows. Each [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗

**Figure 10.** Figure 10: Per-split next-step diagnostics from the executed notebooks on fixed sampled windows. For each split we [PITH_FULL_IMAGE:figures/full_fig_p016_10.png] view at source ↗

**Figure 11.** Figure 11: Per-split rollout diagnostics from fixed sampled notebook windows. Each split shows horizon-wise latent [PITH_FULL_IMAGE:figures/full_fig_p017_11.png] view at source ↗

**Figure 12.** Figure 12: Per-split latent-geometry diagnostics from fixed sampled notebook windows. Each panel shows the per [PITH_FULL_IMAGE:figures/full_fig_p017_12.png] view at source ↗

**Figure 13.** Figure 13: Row-normalized confusion matrices for first-usable-future timing from the full frozen-bundle benchmark. [PITH_FULL_IMAGE:figures/full_fig_p021_13.png] view at source ↗

**Figure 14.** Figure 14: Weather-sensitivity deltas from the executed notebooks on fixed sampled windows. Bars show the change in [PITH_FULL_IMAGE:figures/full_fig_p022_14.png] view at source ↗

discussion (0)

Reference graph

Works this paper leans on

18 extracted references · 9 linked inside Pith

[1]

EarthNet2021: A large-scale dataset and challenge for earth surface forecasting as a guided video prediction task

Christian Requena-Mesa, Vitus Benson, Markus Reichstein, Jakob Runge, and Joachim Denzler. EarthNet2021: A large-scale dataset and challenge for earth surface forecasting as a guided video prediction task. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1132–1142, 2021

2021
[2]

LeWorldModel: Stable end-to-end Joint-Embedding Predictive architecture from pixels.arXiv preprint arXiv:2603.19312, 2026

Lucas Maes, Quentin Le Lidec, Damien Scieur, Yann LeCun, and Randall Balestriero. LeWorldModel: Stable end-to-end Joint-Embedding Predictive architecture from pixels.arXiv preprint arXiv:2603.19312, 2026

Pith/arXiv arXiv 2026
[3]

EarthNet2021: A novel large-scale dataset and challenge for forecasting localized climate impacts.arXiv preprint arXiv:2012.06246, 2020

Christian Requena-Mesa, Vitus Benson, Joachim Denzler, Jakob Runge, and Markus Reichstein. EarthNet2021: A novel large-scale dataset and challenge for forecasting localized climate impacts.arXiv preprint arXiv:2012.06246, 2020

Pith/arXiv arXiv 2012
[4]

Earthformer: Exploring space-time transformers for earth system forecasting.Advances in Neural Information Processing Systems, 35:25390–25403, 2022

Zhihan Gao, Xingjian Shi, Hao Wang, Yi Zhu, Yuyang Bernie Wang, Mu Li, and Dit-Yan Yeung. Earthformer: Exploring space-time transformers for earth system forecasting.Advances in Neural Information Processing Systems, 35:25390–25403, 2022

2022
[5]

EO-WM: A physically informed world model for probabilistic earth observation forecasting.arXiv preprint arXiv:2606.27277, 2026

Junwei Luo, Shuai Yuan, Zhenya Yang, Yansheng Li, Zhe Liu, and Hengshuang Zhao. EO-WM: A physically informed world model for probabilistic earth observation forecasting.arXiv preprint arXiv:2606.27277, 2026

Pith/arXiv arXiv 2026
[6]

SEN12MS-CR-TS: A remote sensing data set for multi-modal multi-temporal cloud removal.IEEE Transactions on Geoscience and Remote Sensing, 60:1–14, 2022

Patrick Ebel, Yajin Xu, Michael Schmitt, and Xiao Xiang Zhu. SEN12MS-CR-TS: A remote sensing data set for multi-modal multi-temporal cloud removal.IEEE Transactions on Geoscience and Remote Sensing, 60:1–14, 2022

2022
[7]

UnCRtainTS: Uncertainty quantification for cloud removal in optical satellite time series

Patrick Ebel, Vivien Sainte Fare Garnot, Michael Schmitt, Jan Dirk Wegner, and Xiao Xiang Zhu. UnCRtainTS: Uncertainty quantification for cloud removal in optical satellite time series. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2086–2096, 2023

2086
[8]

Marius Appel. Efficient data-driven gap filling of satellite image time series using deep neural networks with partial convolutions.Artificial Intelligence for the Earth Systems, 3(2):220055, 2024

2024
[9]

Seeing through the clouds: Cloud gap imputation with Prithvi foundation model.arXiv preprint arXiv:2404.19609, 2024

Denys Godwin, Hanxi Li, Michael Cecil, and Hamed Alemohammad. Seeing through the clouds: Cloud gap imputation with Prithvi foundation model.arXiv preprint arXiv:2404.19609, 2024. 22 APREPRINT- JULY16, 2026

Pith/arXiv arXiv 2024
[10]

Satmae: Pre-training transformers for temporal and multi-spectral satellite imagery.Advances in Neural Information Processing Systems, 35:197–211, 2022

Yezhen Cong, Samar Khanna, Chenlin Meng, Patrick Liu, Erik Rozi, Yutong He, Marshall Burke, David Lobell, and Stefano Ermon. Satmae: Pre-training transformers for temporal and multi-spectral satellite imagery.Advances in Neural Information Processing Systems, 35:197–211, 2022

2022
[11]

Scale-MAE: A scale-aware masked autoencoder for multiscale geospatial representation learning

Colorado J Reed, Ritwik Gupta, Shufan Li, Sarah Brockman, Christopher Funk, Brian Clipp, Kurt Keutzer, Salvatore Candido, Matt Uyttendaele, and Trevor Darrell. Scale-MAE: A scale-aware masked autoencoder for multiscale geospatial representation learning. pages 4088–4099, 2023

2023
[12]

Johannes Jakubik, Sujit Roy, C. E. Phillips, Paolo Fraccaro, Denys Godwin, Bianca Zadrozny, et al. Foundation models for generalist geospatial artificial intelligence.arXiv preprint arXiv:2310.18660, 2023

Pith/arXiv arXiv 2023
[13]

Lightweight metadata-aware Mixture-of-Experts masked autoencoder for earth observation

Mohanad Albughdadi. Lightweight metadata-aware Mixture-of-Experts masked autoencoder for earth observation. arXiv preprint arXiv:2509.10919, 2025

Pith/arXiv arXiv 2025
[14]

World models.arXiv preprint arXiv:1803.10122, 2018

David Ha and Jürgen Schmidhuber. World models.arXiv preprint arXiv:1803.10122, 2018

Pith/arXiv arXiv 2018
[15]

Learning latent dynamics for planning from pixels

Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. Learning latent dynamics for planning from pixels. pages 2555–2565, 2019

2019
[16]

Dream to control: Learning behaviors by latent imagination.arXiv preprint arXiv:1912.01603, 2019

Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to control: Learning behaviors by latent imagination.arXiv preprint arXiv:1912.01603, 2019

Pith/arXiv arXiv 1912
[17]

Self-supervised learning from images with a Joint-Embedding Predictive architecture

Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a Joint-Embedding Predictive architecture. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15619–15629, 2023

2023
[18]

V-JEPA 2: Self-supervised video models enable understanding, prediction and planning.arXiv preprint arXiv:2506.09985, 2025

Mido Assran, Adrien Bardes, David Fan, Quentin Garrido, Russell Howes, Mojtaba Komeili, Matthew Muckley, Ammar Rizvi, Claire Roberts, Koustuv Sinha, et al. V-JEPA 2: Self-supervised video models enable understanding, prediction and planning.arXiv preprint arXiv:2506.09985, 2025. 23

Pith/arXiv arXiv 2025

[1] [1]

EarthNet2021: A large-scale dataset and challenge for earth surface forecasting as a guided video prediction task

Christian Requena-Mesa, Vitus Benson, Markus Reichstein, Jakob Runge, and Joachim Denzler. EarthNet2021: A large-scale dataset and challenge for earth surface forecasting as a guided video prediction task. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1132–1142, 2021

2021

[2] [2]

LeWorldModel: Stable end-to-end Joint-Embedding Predictive architecture from pixels.arXiv preprint arXiv:2603.19312, 2026

Lucas Maes, Quentin Le Lidec, Damien Scieur, Yann LeCun, and Randall Balestriero. LeWorldModel: Stable end-to-end Joint-Embedding Predictive architecture from pixels.arXiv preprint arXiv:2603.19312, 2026

Pith/arXiv arXiv 2026

[3] [3]

EarthNet2021: A novel large-scale dataset and challenge for forecasting localized climate impacts.arXiv preprint arXiv:2012.06246, 2020

Christian Requena-Mesa, Vitus Benson, Joachim Denzler, Jakob Runge, and Markus Reichstein. EarthNet2021: A novel large-scale dataset and challenge for forecasting localized climate impacts.arXiv preprint arXiv:2012.06246, 2020

Pith/arXiv arXiv 2012

[4] [4]

Earthformer: Exploring space-time transformers for earth system forecasting.Advances in Neural Information Processing Systems, 35:25390–25403, 2022

Zhihan Gao, Xingjian Shi, Hao Wang, Yi Zhu, Yuyang Bernie Wang, Mu Li, and Dit-Yan Yeung. Earthformer: Exploring space-time transformers for earth system forecasting.Advances in Neural Information Processing Systems, 35:25390–25403, 2022

2022

[5] [5]

EO-WM: A physically informed world model for probabilistic earth observation forecasting.arXiv preprint arXiv:2606.27277, 2026

Junwei Luo, Shuai Yuan, Zhenya Yang, Yansheng Li, Zhe Liu, and Hengshuang Zhao. EO-WM: A physically informed world model for probabilistic earth observation forecasting.arXiv preprint arXiv:2606.27277, 2026

Pith/arXiv arXiv 2026

[6] [6]

SEN12MS-CR-TS: A remote sensing data set for multi-modal multi-temporal cloud removal.IEEE Transactions on Geoscience and Remote Sensing, 60:1–14, 2022

Patrick Ebel, Yajin Xu, Michael Schmitt, and Xiao Xiang Zhu. SEN12MS-CR-TS: A remote sensing data set for multi-modal multi-temporal cloud removal.IEEE Transactions on Geoscience and Remote Sensing, 60:1–14, 2022

2022

[7] [7]

UnCRtainTS: Uncertainty quantification for cloud removal in optical satellite time series

Patrick Ebel, Vivien Sainte Fare Garnot, Michael Schmitt, Jan Dirk Wegner, and Xiao Xiang Zhu. UnCRtainTS: Uncertainty quantification for cloud removal in optical satellite time series. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2086–2096, 2023

2086

[8] [8]

Marius Appel. Efficient data-driven gap filling of satellite image time series using deep neural networks with partial convolutions.Artificial Intelligence for the Earth Systems, 3(2):220055, 2024

2024

[9] [9]

Seeing through the clouds: Cloud gap imputation with Prithvi foundation model.arXiv preprint arXiv:2404.19609, 2024

Denys Godwin, Hanxi Li, Michael Cecil, and Hamed Alemohammad. Seeing through the clouds: Cloud gap imputation with Prithvi foundation model.arXiv preprint arXiv:2404.19609, 2024. 22 APREPRINT- JULY16, 2026

Pith/arXiv arXiv 2024

[10] [10]

Satmae: Pre-training transformers for temporal and multi-spectral satellite imagery.Advances in Neural Information Processing Systems, 35:197–211, 2022

Yezhen Cong, Samar Khanna, Chenlin Meng, Patrick Liu, Erik Rozi, Yutong He, Marshall Burke, David Lobell, and Stefano Ermon. Satmae: Pre-training transformers for temporal and multi-spectral satellite imagery.Advances in Neural Information Processing Systems, 35:197–211, 2022

2022

[11] [11]

Scale-MAE: A scale-aware masked autoencoder for multiscale geospatial representation learning

Colorado J Reed, Ritwik Gupta, Shufan Li, Sarah Brockman, Christopher Funk, Brian Clipp, Kurt Keutzer, Salvatore Candido, Matt Uyttendaele, and Trevor Darrell. Scale-MAE: A scale-aware masked autoencoder for multiscale geospatial representation learning. pages 4088–4099, 2023

2023

[12] [12]

Johannes Jakubik, Sujit Roy, C. E. Phillips, Paolo Fraccaro, Denys Godwin, Bianca Zadrozny, et al. Foundation models for generalist geospatial artificial intelligence.arXiv preprint arXiv:2310.18660, 2023

Pith/arXiv arXiv 2023

[13] [13]

Lightweight metadata-aware Mixture-of-Experts masked autoencoder for earth observation

Mohanad Albughdadi. Lightweight metadata-aware Mixture-of-Experts masked autoencoder for earth observation. arXiv preprint arXiv:2509.10919, 2025

Pith/arXiv arXiv 2025

[14] [14]

World models.arXiv preprint arXiv:1803.10122, 2018

David Ha and Jürgen Schmidhuber. World models.arXiv preprint arXiv:1803.10122, 2018

Pith/arXiv arXiv 2018

[15] [15]

Learning latent dynamics for planning from pixels

Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. Learning latent dynamics for planning from pixels. pages 2555–2565, 2019

2019

[16] [16]

Dream to control: Learning behaviors by latent imagination.arXiv preprint arXiv:1912.01603, 2019

Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to control: Learning behaviors by latent imagination.arXiv preprint arXiv:1912.01603, 2019

Pith/arXiv arXiv 1912

[17] [17]

Self-supervised learning from images with a Joint-Embedding Predictive architecture

Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a Joint-Embedding Predictive architecture. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15619–15629, 2023

2023

[18] [18]

V-JEPA 2: Self-supervised video models enable understanding, prediction and planning.arXiv preprint arXiv:2506.09985, 2025

Mido Assran, Adrien Bardes, David Fan, Quentin Garrido, Russell Howes, Mojtaba Komeili, Matthew Muckley, Ammar Rizvi, Claire Roberts, Koustuv Sinha, et al. V-JEPA 2: Self-supervised video models enable understanding, prediction and planning.arXiv preprint arXiv:2506.09985, 2025. 23

Pith/arXiv arXiv 2025