pith. sign in

arxiv: 2503.19719 · v1 · pith:GD725ZJOnew · submitted 2025-03-25 · 💻 cs.LG · cs.AI· cs.CV

On What Depends the Robustness of Multi-source Models to Missing Data in Earth Observation?

Pith reviewed 2026-05-22 22:02 UTC · model grok-4.3

classification 💻 cs.LG cs.AIcs.CV
keywords multi-source modelsmissing dataEarth observationrobustnesspredictive performancedata complementaritymodel design
0
0 comments X

The pith

Multi-source model robustness to missing data in Earth observation hinges on task type, source complementarity, and model design.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests six multi-source models on Earth observation prediction tasks under conditions where one data source is missing or only one source remains available. It finds that how well these models maintain performance depends on the specific task being solved, how much the different sources add distinct information to each other, and the internal structure of the model itself. In several cases the models actually produced higher accuracy after a data source was removed. This challenges the common view that combining every available data source will always improve results.

Core claim

The predictive performance of multi-source models when data sources are missing is determined by the nature of the task, the complementarity among data sources, and the model design, with cases where removing certain sources improves accuracy.

What carries the argument

Evaluation of six state-of-the-art multi-source models on single-source-missing and single-source-available scenarios in Earth observation tasks.

If this is right

  • Performance gains from additional sources are not guaranteed and must be checked per task.
  • Model design choices affect how gracefully performance degrades when sources disappear.
  • Some data sources can act as noise rather than signal for certain prediction problems.
  • Streamlined sensor or data-collection strategies may be viable without loss of accuracy.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Data-acquisition plans in Earth observation could be optimized by testing source subsets rather than collecting everything.
  • Dynamic source-selection modules inside models might exploit the observed complementarity effects.
  • The same dependence on task and complementarity could appear in other multi-modal settings outside Earth observation.

Load-bearing premise

The chosen missing-data scenarios and the six selected models capture the main factors that control robustness in real Earth observation applications.

What would settle it

An experiment on a wider range of tasks or models in which performance no longer varies systematically with task type, source complementarity, or design, and in which removing sources never improves accuracy.

Figures

Figures reproduced from arXiv: 2503.19719 by Andreas Dengel, Diego Arenas, Francisco Mena, Miro Miranda.

Figure 1
Figure 1. Figure 1: The spatial-wise is when a portion of the image is [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 1
Figure 1. Figure 1: Types of missing data in the EO field. For the temporal missing, two cases are shown (spatial and feature wise). missing data. In our work, we focus on the latter case when it occurs during inference in multi-source learning models. A. How to handle source-wise missing data? In order to use the trained multi-source model in source￾wise missing data scenarios, an intervention is required, usu￾ally involving… view at source ↗
read the original abstract

In recent years, the development of robust multi-source models has emerged in the Earth Observation (EO) field. These are models that leverage data from diverse sources to improve predictive accuracy when there is missing data. Despite these advancements, the factors influencing the varying effectiveness of such models remain poorly understood. In this study, we evaluate the predictive performance of six state-of-the-art multi-source models in predicting scenarios where either a single data source is missing or only a single source is available. Our analysis reveals that the efficacy of these models is intricately tied to the nature of the task, the complementarity among data sources, and the model design. Surprisingly, we observe instances where the removal of certain data sources leads to improved predictive performance, challenging the assumption that incorporating all available data is always beneficial. These findings prompt critical reflections on model complexity and the necessity of all collected data sources, potentially shaping the way for more streamlined approaches in EO applications.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 0 minor

Summary. The manuscript evaluates six state-of-the-art multi-source models for Earth Observation tasks under single-source-missing or single-source-available conditions. It claims that predictive efficacy depends on task nature, source complementarity, and model design, and reports instances where removing certain sources improves performance, challenging the assumption that more sources are always better.

Significance. If the empirical observations hold under rigorous controls, the work would provide evidence against the default assumption in multi-source EO modeling that all available sources should be retained, potentially guiding more parsimonious model and data-collection strategies. The abstract, however, contains no quantitative results, so the practical significance cannot yet be assessed.

major comments (2)
  1. [Abstract] Abstract: the central claims—that efficacy is tied to task, complementarity, and model design, and that source removal can improve performance—are stated without any metrics, statistical tests, dataset statistics, missingness mechanisms, or experimental controls. These omissions are load-bearing because the entire contribution rests on the evaluation results.
  2. [Abstract] Abstract: no details are supplied on model-selection criteria, dataset characteristics, or controls for confounding factors such as model capacity. Without this information it is impossible to determine whether the reported robustness patterns generalize or are artifacts of the chosen six models and single-source scenarios.

Simulated Author's Rebuttal

2 responses · 1 unresolved

We thank the referee for the comments. We address each major comment below. Because only the abstract was provided for this response, our ability to reference specific experimental details is limited.

read point-by-point responses
  1. Referee: [Abstract] Abstract: the central claims—that efficacy is tied to task, complementarity, and model design, and that source removal can improve performance—are stated without any metrics, statistical tests, dataset statistics, missingness mechanisms, or experimental controls. These omissions are load-bearing because the entire contribution rests on the evaluation results.

    Authors: We agree that the abstract contains no quantitative metrics, tests, or controls. The abstract is written as a concise summary of conclusions; all supporting numbers, mechanisms, and controls appear in the body of the manuscript. Without the full text available here we cannot quote those results. revision: no

  2. Referee: [Abstract] Abstract: no details are supplied on model-selection criteria, dataset characteristics, or controls for confounding factors such as model capacity. Without this information it is impossible to determine whether the reported robustness patterns generalize or are artifacts of the chosen six models and single-source scenarios.

    Authors: We agree that the abstract supplies none of these details. Model-selection criteria, dataset descriptions, and capacity controls are stated in the experimental-setup section of the full paper. Because only the abstract is accessible in the current exchange, we cannot demonstrate those controls or selection rationale. revision: no

standing simulated objections not resolved
  • Full experimental results, metrics, dataset statistics, missingness mechanisms, model-selection criteria, and capacity controls are not present in the provided manuscript excerpt (only the abstract), so we cannot supply the concrete evidence the referee requests.

Circularity Check

0 steps flagged

No circularity: empirical benchmarking with no derivations or self-referential reductions

full rationale

The paper is an empirical benchmarking study that evaluates six multi-source models on single-source-missing and single-source-available scenarios in Earth Observation tasks. The abstract reports observational findings on how performance depends on task nature, source complementarity, and model design, including occasional improvements from source removal. No equations, derivations, fitted parameters, or predictions are present. No self-citations, uniqueness theorems, or ansatzes are invoked. The central claims rest on experimental results rather than any reduction to inputs by construction, making the study self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

No mathematical model, free parameters, axioms, or invented entities are introduced; the work is an empirical comparison of existing models.

pith-pipeline@v0.9.0 · 5671 in / 936 out tokens · 47597 ms · 2026-05-22T22:02:38.238505+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

29 extracted references · 29 canonical work pages · 1 internal anchor

  1. [1]

    Camps-Valls, D

    G. Camps-Valls, D. Tuia, X. X. Zhu, and M. Reichstein, Deep learning for the Earth Sciences: A comprehensive approach to remote sensing, climate science and geosciences . John Wiley & Sons, 2021

  2. [2]

    Common practices and taxonomy in deep multi-view fusion for remote sensing applications,

    F. Mena, D. Arenas, M. Nuske, and A. Dengel, “Common practices and taxonomy in deep multi-view fusion for remote sensing applications,” IEEE Journal of Selected Topics in Ap- plied Earth Observations and Remote Sensing , pp. 4797–4818, 2024

  3. [3]

    More diverse means better: Multimodal deep learning meets remote-sensing imagery classification,

    D. Hong, L. Gao, N. Yokoya, J. Yao, J. Chanussot, Q. Du, and B. Zhang, “More diverse means better: Multimodal deep learning meets remote-sensing imagery classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 5, pp. 4340–4354, 2021

  4. [4]

    Multi- modal temporal attention models for crop mapping from satel- lite time series,

    V . Sainte Fare Garnot, L. Landrieu, and N. Chehata, “Multi- modal temporal attention models for crop mapping from satel- lite time series,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 187, pp. 294–305, 2022

  5. [5]

    Impact assessment of missing data in model predictions for Earth observation applications,

    F. Mena, D. Arenas, M. Charfuelan, M. Nuske, and A. Dengel, “Impact assessment of missing data in model predictions for Earth observation applications,” in IEEE International Geo- science and Remote Sensing Symposium , 2024, pp. 967–971

  6. [6]

    Missing information reconstruction of remote sens- ing data: A technical review,

    H. Shen, X. Li, Q. Cheng, C. Zeng, G. Yang, H. Li, and L. Zhang, “Missing information reconstruction of remote sens- ing data: A technical review,” IEEE Geoscience and Remote Sensing Magazine, vol. 3, no. 3, pp. 61–85, 2015

  7. [7]

    Hy- perspectral image classification using random occlusion data augmentation,

    J. M. Haut, M. E. Paoletti, J. Plaza, A. Plaza, and J. Li, “Hy- perspectral image classification using random occlusion data augmentation,” IEEE Geoscience and Remote Sensing Letters , vol. 16, no. 11, pp. 1751–1755, 2019

  8. [8]

    Robust input layer for neural networks for hyperspectral classification of data with missing bands,

    L. Fasnacht, P. Renard, and P. Brunner, “Robust input layer for neural networks for hyperspectral classification of data with missing bands,” Applied Computing and Geosciences , vol. 8, 2020

  9. [9]

    Increasing the Robustness of Model Predictions to Missing Sensors in Earth Observation

    F. Mena, D. Arenas, and A. Dengel, “Increasing the robustness of model predictions to missing sensors in earth observation,” arXiv preprint arXiv:2407.15512 , 2024

  10. [10]

    Urban land cover classification with missing data modalities using deep convolutional neural networks,

    M. Kampffmeyer, A.-B. Salberg, and R. Jenssen, “Urban land cover classification with missing data modalities using deep convolutional neural networks,”IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , vol. 11, no. 6, pp. 1758–1768, 2018

  11. [11]

    A co-learning method to utilize optical images and photogrammetric point clouds for building extraction,

    Y . Xie, J. Tian, and X. X. Zhu, “A co-learning method to utilize optical images and photogrammetric point clouds for building extraction,” International Journal of Applied Earth Observation and Geoinformation, vol. 116, 2023

  12. [12]

    Skysense: A multi-modal re- mote sensing foundation model towards universal interpretation for Earth observation imagery,

    X. Guo, J. Lao, B. Dang, Y . Zhang, L. Yu, L. Ru, L. Zhong, Z. Huang, K. Wu, D. Hu et al., “Skysense: A multi-modal re- mote sensing foundation model towards universal interpretation for Earth observation imagery,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 27 672–27 683

  13. [13]

    Better, not just more: Data-centric machine learning for earth observation,

    R. Roscher, M. Rußwurm, C. Gevaert, M. Kampffmeyer, J. A. Dos Santos, M. Vakalopoulou, R. H ¨ansch, S. Hansen, K. Nogueira, J. Prexl et al., “Better, not just more: Data-centric machine learning for earth observation,” IEEE Geoscience and Remote Sensing Magazine , 2024

  14. [14]

    Deep occlusion framework for mul- timodal Earth observation data,

    B. Ekim and M. Schmitt, “Deep occlusion framework for mul- timodal Earth observation data,” IEEE Geoscience and Remote Sensing Letters, 2024

  15. [15]

    A novel approach to incomplete multimodal learning for remote sensing data fusion,

    Y . Chen, M. Zhao, and L. Bruzzone, “A novel approach to incomplete multimodal learning for remote sensing data fusion,” IEEE Transactions on Geoscience and Remote Sensing , 2024

  16. [16]

    Handling unexpected inputs: Incorporating source-wise out-of-distribution detection into SAR-optical data fusion for scene classification,

    J. Gawlikowski, S. Saha, J. Niebling, and X. X. Zhu, “Handling unexpected inputs: Incorporating source-wise out-of-distribution detection into SAR-optical data fusion for scene classification,” EURASIP Journal on Advances in Signal Processing, vol. 2023, no. 1, 2023

  17. [17]

    Understanding urban landuse from the above and ground perspectives: A deep learning, multimodal solution,

    S. Srivastava, J. E. Vargas-Munoz, and D. Tuia, “Understanding urban landuse from the above and ground perspectives: A deep learning, multimodal solution,” Remote Sensing of Environment, vol. 228, pp. 129–143, 2019

  18. [18]

    Omnisat: Self-supervised modality fusion for earth observation,

    G. Astruc, N. Gonthier, C. Mallet, and L. Landrieu, “Omnisat: Self-supervised modality fusion for earth observation,” in Eu- ropean Conference on Computer Vision . Springer, 2025, pp. 409–427

  19. [19]

    Fine- grained landuse characterization using ground-based pictures: A deep learning solution based on globally available data,

    S. Srivastava, J. E. Vargas Munoz, S. Lobry, and D. Tuia, “Fine- grained landuse characterization using ground-based pictures: A deep learning solution based on globally available data,” International Journal of Geographical Information Science , vol. 34, no. 6, pp. 1117–1136, 2020

  20. [20]

    Deep multisensor learning for missing-modality all-weather mapping,

    Z. Zheng, A. Ma, L. Zhang, and Y . Zhong, “Deep multisensor learning for missing-modality all-weather mapping,” ISPRS Journal of Photogrammetry and Remote Sensing , vol. 174, pp. 254–264, 2021

  21. [21]

    Decoupling common and unique representa- tions for multimodal self-supervised learning,

    Y . Wang, C. M. Albrecht, N. A. A. Braham, C. Liu, Z. Xiong, and X. X. Zhu, “Decoupling common and unique representa- tions for multimodal self-supervised learning,” in18th European Conference on Computer Vision, ECCV 2024 . Springer, 2024

  22. [22]

    DisCoM-KD: Cross-modal knowl- edge distillation via disentanglement representation and adver- sarial learning,

    D. Ienco and C. F. Dantas, “DisCoM-KD: Cross-modal knowl- edge distillation via disentanglement representation and adver- sarial learning,” arXiv preprint arXiv:2408.07080 , 2024

  23. [23]

    MMEarth: Exploring multi-modal pretext tasks for geospatial representation learning,

    V . Nedungadi, A. Kariryaa, S. Oehmcke, S. Belongie, C. Igel, and N. Lang, “MMEarth: Exploring multi-modal pretext tasks for geospatial representation learning,” arXiv preprint arXiv:2405.02771, 2024

  24. [24]

    CropHarvest: A global dataset for crop-type classification,

    G. Tseng, I. Zvonkov, C. L. Nakalembe, and H. Kerner, “CropHarvest: A global dataset for crop-type classification,” Proceedings of NIPS Datasets and Benchmarks Track , 2021

  25. [25]

    TreeSatAI benchmark archive: A multi-sensor, multi-label dataset for tree species classification in remote sensing,

    S. Ahlswede, C. Schulz, C. Gava, P. Helber, B. Bischke, M. F ¨orster, F. Arias, J. Hees, B. Demir, and B. Kleinschmit, “TreeSatAI benchmark archive: A multi-sensor, multi-label dataset for tree species classification in remote sensing,” Earth System Science Data Discussions , vol. 2022, 2022

  26. [26]

    Transformer-based incomplete multi-modal learning for land cover classification,

    G. Xu, X. Jiang, Y . Zhou, J. Fu, Z. Huang, and X. Liu, “Transformer-based incomplete multi-modal learning for land cover classification,” in IEEE International Geoscience and Remote Sensing Symposium . IEEE, 2024, pp. 7276–7281

  27. [27]

    Temporal convolu- tional neural network for the classification of satellite image time series,

    C. Pelletier, G. I. Webb, and F. Petitjean, “Temporal convolu- tional neural network for the classification of satellite image time series,” Remote Sensing, vol. 11, no. 5, 2019

  28. [28]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2016, pp. 770–778

  29. [29]

    Data-centric machine learning for Earth observation: Necessary and sufficient fea- tures,

    H. Najjar, M. Nuske, and A. Dengel, “Data-centric machine learning for Earth observation: Necessary and sufficient fea- tures,” arXiv preprint arXiv:2408.11384 , 2024