On What Depends the Robustness of Multi-source Models to Missing Data in Earth Observation?
Pith reviewed 2026-05-22 22:02 UTC · model grok-4.3
The pith
Multi-source model robustness to missing data in Earth observation hinges on task type, source complementarity, and model design.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The predictive performance of multi-source models when data sources are missing is determined by the nature of the task, the complementarity among data sources, and the model design, with cases where removing certain sources improves accuracy.
What carries the argument
Evaluation of six state-of-the-art multi-source models on single-source-missing and single-source-available scenarios in Earth observation tasks.
If this is right
- Performance gains from additional sources are not guaranteed and must be checked per task.
- Model design choices affect how gracefully performance degrades when sources disappear.
- Some data sources can act as noise rather than signal for certain prediction problems.
- Streamlined sensor or data-collection strategies may be viable without loss of accuracy.
Where Pith is reading between the lines
- Data-acquisition plans in Earth observation could be optimized by testing source subsets rather than collecting everything.
- Dynamic source-selection modules inside models might exploit the observed complementarity effects.
- The same dependence on task and complementarity could appear in other multi-modal settings outside Earth observation.
Load-bearing premise
The chosen missing-data scenarios and the six selected models capture the main factors that control robustness in real Earth observation applications.
What would settle it
An experiment on a wider range of tasks or models in which performance no longer varies systematically with task type, source complementarity, or design, and in which removing sources never improves accuracy.
Figures
read the original abstract
In recent years, the development of robust multi-source models has emerged in the Earth Observation (EO) field. These are models that leverage data from diverse sources to improve predictive accuracy when there is missing data. Despite these advancements, the factors influencing the varying effectiveness of such models remain poorly understood. In this study, we evaluate the predictive performance of six state-of-the-art multi-source models in predicting scenarios where either a single data source is missing or only a single source is available. Our analysis reveals that the efficacy of these models is intricately tied to the nature of the task, the complementarity among data sources, and the model design. Surprisingly, we observe instances where the removal of certain data sources leads to improved predictive performance, challenging the assumption that incorporating all available data is always beneficial. These findings prompt critical reflections on model complexity and the necessity of all collected data sources, potentially shaping the way for more streamlined approaches in EO applications.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript evaluates six state-of-the-art multi-source models for Earth Observation tasks under single-source-missing or single-source-available conditions. It claims that predictive efficacy depends on task nature, source complementarity, and model design, and reports instances where removing certain sources improves performance, challenging the assumption that more sources are always better.
Significance. If the empirical observations hold under rigorous controls, the work would provide evidence against the default assumption in multi-source EO modeling that all available sources should be retained, potentially guiding more parsimonious model and data-collection strategies. The abstract, however, contains no quantitative results, so the practical significance cannot yet be assessed.
major comments (2)
- [Abstract] Abstract: the central claims—that efficacy is tied to task, complementarity, and model design, and that source removal can improve performance—are stated without any metrics, statistical tests, dataset statistics, missingness mechanisms, or experimental controls. These omissions are load-bearing because the entire contribution rests on the evaluation results.
- [Abstract] Abstract: no details are supplied on model-selection criteria, dataset characteristics, or controls for confounding factors such as model capacity. Without this information it is impossible to determine whether the reported robustness patterns generalize or are artifacts of the chosen six models and single-source scenarios.
Simulated Author's Rebuttal
We thank the referee for the comments. We address each major comment below. Because only the abstract was provided for this response, our ability to reference specific experimental details is limited.
read point-by-point responses
-
Referee: [Abstract] Abstract: the central claims—that efficacy is tied to task, complementarity, and model design, and that source removal can improve performance—are stated without any metrics, statistical tests, dataset statistics, missingness mechanisms, or experimental controls. These omissions are load-bearing because the entire contribution rests on the evaluation results.
Authors: We agree that the abstract contains no quantitative metrics, tests, or controls. The abstract is written as a concise summary of conclusions; all supporting numbers, mechanisms, and controls appear in the body of the manuscript. Without the full text available here we cannot quote those results. revision: no
-
Referee: [Abstract] Abstract: no details are supplied on model-selection criteria, dataset characteristics, or controls for confounding factors such as model capacity. Without this information it is impossible to determine whether the reported robustness patterns generalize or are artifacts of the chosen six models and single-source scenarios.
Authors: We agree that the abstract supplies none of these details. Model-selection criteria, dataset descriptions, and capacity controls are stated in the experimental-setup section of the full paper. Because only the abstract is accessible in the current exchange, we cannot demonstrate those controls or selection rationale. revision: no
- Full experimental results, metrics, dataset statistics, missingness mechanisms, model-selection criteria, and capacity controls are not present in the provided manuscript excerpt (only the abstract), so we cannot supply the concrete evidence the referee requests.
Circularity Check
No circularity: empirical benchmarking with no derivations or self-referential reductions
full rationale
The paper is an empirical benchmarking study that evaluates six multi-source models on single-source-missing and single-source-available scenarios in Earth Observation tasks. The abstract reports observational findings on how performance depends on task nature, source complementarity, and model design, including occasional improvements from source removal. No equations, derivations, fitted parameters, or predictions are present. No self-citations, uniqueness theorems, or ansatzes are invoked. The central claims rest on experimental results rather than any reduction to inputs by construction, making the study self-contained against external benchmarks.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
G. Camps-Valls, D. Tuia, X. X. Zhu, and M. Reichstein, Deep learning for the Earth Sciences: A comprehensive approach to remote sensing, climate science and geosciences . John Wiley & Sons, 2021
work page 2021
-
[2]
Common practices and taxonomy in deep multi-view fusion for remote sensing applications,
F. Mena, D. Arenas, M. Nuske, and A. Dengel, “Common practices and taxonomy in deep multi-view fusion for remote sensing applications,” IEEE Journal of Selected Topics in Ap- plied Earth Observations and Remote Sensing , pp. 4797–4818, 2024
work page 2024
-
[3]
More diverse means better: Multimodal deep learning meets remote-sensing imagery classification,
D. Hong, L. Gao, N. Yokoya, J. Yao, J. Chanussot, Q. Du, and B. Zhang, “More diverse means better: Multimodal deep learning meets remote-sensing imagery classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 5, pp. 4340–4354, 2021
work page 2021
-
[4]
Multi- modal temporal attention models for crop mapping from satel- lite time series,
V . Sainte Fare Garnot, L. Landrieu, and N. Chehata, “Multi- modal temporal attention models for crop mapping from satel- lite time series,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 187, pp. 294–305, 2022
work page 2022
-
[5]
Impact assessment of missing data in model predictions for Earth observation applications,
F. Mena, D. Arenas, M. Charfuelan, M. Nuske, and A. Dengel, “Impact assessment of missing data in model predictions for Earth observation applications,” in IEEE International Geo- science and Remote Sensing Symposium , 2024, pp. 967–971
work page 2024
-
[6]
Missing information reconstruction of remote sens- ing data: A technical review,
H. Shen, X. Li, Q. Cheng, C. Zeng, G. Yang, H. Li, and L. Zhang, “Missing information reconstruction of remote sens- ing data: A technical review,” IEEE Geoscience and Remote Sensing Magazine, vol. 3, no. 3, pp. 61–85, 2015
work page 2015
-
[7]
Hy- perspectral image classification using random occlusion data augmentation,
J. M. Haut, M. E. Paoletti, J. Plaza, A. Plaza, and J. Li, “Hy- perspectral image classification using random occlusion data augmentation,” IEEE Geoscience and Remote Sensing Letters , vol. 16, no. 11, pp. 1751–1755, 2019
work page 2019
-
[8]
Robust input layer for neural networks for hyperspectral classification of data with missing bands,
L. Fasnacht, P. Renard, and P. Brunner, “Robust input layer for neural networks for hyperspectral classification of data with missing bands,” Applied Computing and Geosciences , vol. 8, 2020
work page 2020
-
[9]
Increasing the Robustness of Model Predictions to Missing Sensors in Earth Observation
F. Mena, D. Arenas, and A. Dengel, “Increasing the robustness of model predictions to missing sensors in earth observation,” arXiv preprint arXiv:2407.15512 , 2024
work page internal anchor Pith review Pith/arXiv arXiv 2024
-
[10]
M. Kampffmeyer, A.-B. Salberg, and R. Jenssen, “Urban land cover classification with missing data modalities using deep convolutional neural networks,”IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , vol. 11, no. 6, pp. 1758–1768, 2018
work page 2018
-
[11]
Y . Xie, J. Tian, and X. X. Zhu, “A co-learning method to utilize optical images and photogrammetric point clouds for building extraction,” International Journal of Applied Earth Observation and Geoinformation, vol. 116, 2023
work page 2023
-
[12]
X. Guo, J. Lao, B. Dang, Y . Zhang, L. Yu, L. Ru, L. Zhong, Z. Huang, K. Wu, D. Hu et al., “Skysense: A multi-modal re- mote sensing foundation model towards universal interpretation for Earth observation imagery,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 27 672–27 683
work page 2024
-
[13]
Better, not just more: Data-centric machine learning for earth observation,
R. Roscher, M. Rußwurm, C. Gevaert, M. Kampffmeyer, J. A. Dos Santos, M. Vakalopoulou, R. H ¨ansch, S. Hansen, K. Nogueira, J. Prexl et al., “Better, not just more: Data-centric machine learning for earth observation,” IEEE Geoscience and Remote Sensing Magazine , 2024
work page 2024
-
[14]
Deep occlusion framework for mul- timodal Earth observation data,
B. Ekim and M. Schmitt, “Deep occlusion framework for mul- timodal Earth observation data,” IEEE Geoscience and Remote Sensing Letters, 2024
work page 2024
-
[15]
A novel approach to incomplete multimodal learning for remote sensing data fusion,
Y . Chen, M. Zhao, and L. Bruzzone, “A novel approach to incomplete multimodal learning for remote sensing data fusion,” IEEE Transactions on Geoscience and Remote Sensing , 2024
work page 2024
-
[16]
J. Gawlikowski, S. Saha, J. Niebling, and X. X. Zhu, “Handling unexpected inputs: Incorporating source-wise out-of-distribution detection into SAR-optical data fusion for scene classification,” EURASIP Journal on Advances in Signal Processing, vol. 2023, no. 1, 2023
work page 2023
-
[17]
S. Srivastava, J. E. Vargas-Munoz, and D. Tuia, “Understanding urban landuse from the above and ground perspectives: A deep learning, multimodal solution,” Remote Sensing of Environment, vol. 228, pp. 129–143, 2019
work page 2019
-
[18]
Omnisat: Self-supervised modality fusion for earth observation,
G. Astruc, N. Gonthier, C. Mallet, and L. Landrieu, “Omnisat: Self-supervised modality fusion for earth observation,” in Eu- ropean Conference on Computer Vision . Springer, 2025, pp. 409–427
work page 2025
-
[19]
S. Srivastava, J. E. Vargas Munoz, S. Lobry, and D. Tuia, “Fine- grained landuse characterization using ground-based pictures: A deep learning solution based on globally available data,” International Journal of Geographical Information Science , vol. 34, no. 6, pp. 1117–1136, 2020
work page 2020
-
[20]
Deep multisensor learning for missing-modality all-weather mapping,
Z. Zheng, A. Ma, L. Zhang, and Y . Zhong, “Deep multisensor learning for missing-modality all-weather mapping,” ISPRS Journal of Photogrammetry and Remote Sensing , vol. 174, pp. 254–264, 2021
work page 2021
-
[21]
Decoupling common and unique representa- tions for multimodal self-supervised learning,
Y . Wang, C. M. Albrecht, N. A. A. Braham, C. Liu, Z. Xiong, and X. X. Zhu, “Decoupling common and unique representa- tions for multimodal self-supervised learning,” in18th European Conference on Computer Vision, ECCV 2024 . Springer, 2024
work page 2024
-
[22]
D. Ienco and C. F. Dantas, “DisCoM-KD: Cross-modal knowl- edge distillation via disentanglement representation and adver- sarial learning,” arXiv preprint arXiv:2408.07080 , 2024
-
[23]
MMEarth: Exploring multi-modal pretext tasks for geospatial representation learning,
V . Nedungadi, A. Kariryaa, S. Oehmcke, S. Belongie, C. Igel, and N. Lang, “MMEarth: Exploring multi-modal pretext tasks for geospatial representation learning,” arXiv preprint arXiv:2405.02771, 2024
-
[24]
CropHarvest: A global dataset for crop-type classification,
G. Tseng, I. Zvonkov, C. L. Nakalembe, and H. Kerner, “CropHarvest: A global dataset for crop-type classification,” Proceedings of NIPS Datasets and Benchmarks Track , 2021
work page 2021
-
[25]
S. Ahlswede, C. Schulz, C. Gava, P. Helber, B. Bischke, M. F ¨orster, F. Arias, J. Hees, B. Demir, and B. Kleinschmit, “TreeSatAI benchmark archive: A multi-sensor, multi-label dataset for tree species classification in remote sensing,” Earth System Science Data Discussions , vol. 2022, 2022
work page 2022
-
[26]
Transformer-based incomplete multi-modal learning for land cover classification,
G. Xu, X. Jiang, Y . Zhou, J. Fu, Z. Huang, and X. Liu, “Transformer-based incomplete multi-modal learning for land cover classification,” in IEEE International Geoscience and Remote Sensing Symposium . IEEE, 2024, pp. 7276–7281
work page 2024
-
[27]
Temporal convolu- tional neural network for the classification of satellite image time series,
C. Pelletier, G. I. Webb, and F. Petitjean, “Temporal convolu- tional neural network for the classification of satellite image time series,” Remote Sensing, vol. 11, no. 5, 2019
work page 2019
-
[28]
Deep residual learning for image recognition,
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2016, pp. 770–778
work page 2016
-
[29]
Data-centric machine learning for Earth observation: Necessary and sufficient fea- tures,
H. Najjar, M. Nuske, and A. Dengel, “Data-centric machine learning for Earth observation: Necessary and sufficient fea- tures,” arXiv preprint arXiv:2408.11384 , 2024
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.