pith. sign in

arxiv: 2407.15512 · v2 · pith:3WDGQ5XQnew · submitted 2024-07-22 · 💻 cs.LG · cs.AI· cs.CV

Increasing the Robustness of Model Predictions to Missing Sensors in Earth Observation

Francisco Mena , Diego Arenas , Andreas Dengel This is my paper

Pith reviewed 2026-05-23 22:48 UTC · model grok-4.3

classification 💻 cs.LG cs.AIcs.CV
keywords multi-sensor learningEarth observationmissing sensorsrobustnesssensor dropoutensemble modelstemporal data
0
0 comments X

The pith

Ensemble multi-sensor models maintain the most accuracy when sensors are missing in Earth observation tasks.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests two new approaches for training multi-sensor models so they degrade less when some sensors fail to provide data. Input Sensor Dropout randomly omits sensors during training, while Ensemble Sensor Invariant builds multiple sensor-invariant models and combines them. Experiments across three temporal Earth observation datasets measure how much prediction quality falls at different rates of sensor absence. The results indicate that the ensemble approach limits performance loss more effectively than single models or dropout alone.

Core claim

Input Sensor Dropout and Ensemble Sensor Invariant are introduced to improve robustness, and experiments demonstrate that these methods increase the robustness of model predictions to missing sensors, with ensemble multi-sensor models proving the most robust and the sensor dropout component showing promising results.

What carries the argument

Input Sensor Dropout (ISensD), which adds random sensor omission during training, and Ensemble Sensor Invariant (ESensI), which trains an ensemble of models each invariant to sensor subsets, to reduce accuracy loss under missing sensor conditions.

If this is right

  • Ensemble multi-sensor models show smaller drops in predictive performance than single models when sensors are removed at test time.
  • The sensor dropout component alone contributes measurable robustness gains even without the full ensemble structure.
  • Performance remains more stable across varying percentages of missing sensors when the proposed training procedures are applied.
  • The methods address non-persistent sensors affected by external factors in temporal sequences.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same dropout and ensembling ideas could be tested on multi-modal inputs outside Earth observation, such as medical imaging with variable scanner types.
  • If ensembles dominate, operational systems might default to them when sensor uptime cannot be guaranteed in advance.
  • Combining dropout during training with ensemble inference at test time might produce additive robustness that the current experiments do not yet isolate.

Load-bearing premise

The three multi-sensor temporal datasets used capture the missing-sensor patterns that arise in actual Earth observation deployments.

What would settle it

Running the trained models on a new multi-sensor Earth observation dataset with sensor failure rates and patterns not present in the original three datasets and checking whether the reported robustness ordering still holds.

Figures

Figures reproduced from arXiv: 2407.15512 by Andreas Dengel, Diego Arenas, Francisco Mena.

Figure 1
Figure 1. Figure 1: Illustration of the ISensD method. We show three batch examples and the associated randomly masked sensors based on the SensD technique. Reconstruction. A more sophisticated way to fill in the missing sensors is to learn a function that predicts all the missing sensor features. Such techniques include cross-modal auto-encoders and translation models [22]. The latter method has been applied to EO data by tr… view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of the ESensI method. The main component is the shared prediction head, which makes the ensemble-based a sensor-invariant model. 4 Experiments 4.1 Datasets CropHarvest (CH). We use the cropharvest dataset for multi-sensor crop recog￾nition [25], in a multi-crop version [15]. This involves a classification task in which the crop-type (between 10 crop-type groups including non-crop) at a given l… view at source ↗
Figure 3
Figure 3. Figure 3: PRS of methods at different levels of missing sensors in the CH data. 10 20 30 40 50 60 70 80 90 100 Percentage of missing optical sensor 0.4 0.5 0.6 0.7 0.8 0.9 1.0 PRS value Input ITempD Feature Ensemble 10 20 30 40 50 60 70 80 90 100 Percentage of missing radar sensor ISensD ISensD (NR) ESensI [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: PRS of methods at different levels of missing sensors in the LFMC data. For the predictive performance, we use the F1 Macro (F1) macro in classification and the Coefficient of Determination (R2 ) in regression tasks. 4.4 Results We report the results of the best dropout ratio in the SensD, which is 60% in CH data, 25% in the LFMC data, and 45% in the PM25 data. For the ESensI we include the sensor encoding… view at source ↗
Figure 5
Figure 5. Figure 5: PRS of methods at different levels of missing sensors in the PM25 data [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Predictive performance results of the [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: F1 scores at different levels of missing sensors in the CH data [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: R2 scores of the compared methods at different levels of missing sensors in the LFMC data. 10 20 30 40 50 60 70 80 90 100 Percentage of missing dynamic features 0.0 0.2 0.4 0.6 0.8 R2 score Input ITempD Ensemble 10 20 30 40 50 60 70 80 90 100 Percentage of missing condition features ISensD ISensD (NR) ESensI [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: R2 scores at different levels of missing sensors in the PM25 data. C Alternative encoders In [PITH_FULL_IMAGE:figures/full_fig_p014_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Predictive performance with different encoders in the [PITH_FULL_IMAGE:figures/full_fig_p015_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Predictive performance with different encoders in the [PITH_FULL_IMAGE:figures/full_fig_p015_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Predictive performance with different encoders in the [PITH_FULL_IMAGE:figures/full_fig_p016_12.png] view at source ↗
read the original abstract

Multi-sensor ML models for EO aim to enhance prediction accuracy by integrating data from various sources. However, the presence of missing data poses a significant challenge, particularly in non-persistent sensors that can be affected by external factors. Existing literature has explored strategies like temporal dropout and sensor-invariant models to address the generalization to missing data issues. Inspired by these works, we study two novel methods tailored for multi-sensor scenarios, namely Input Sensor Dropout (ISensD) and Ensemble Sensor Invariant (ESensI). Through experimentation on three multi-sensor temporal EO datasets, we demonstrate that these methods effectively increase the robustness of model predictions to missing sensors. Particularly, we focus on how the predictive performance of models drops when sensors are missing at different levels. We observe that ensemble multi-sensor models are the most robust to the lack of sensors. In addition, the sensor dropout component in ISensD shows promising robustness results.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 0 minor

Summary. The manuscript proposes two methods—Input Sensor Dropout (ISensD) and Ensemble Sensor Invariant (ESensI)—to improve robustness of multi-sensor ML models for Earth Observation tasks against missing sensor data. It evaluates these approaches on three multi-sensor temporal EO datasets, reporting that ensemble multi-sensor models are the most robust and that the sensor dropout component of ISensD yields promising results.

Significance. If the empirical results hold under more detailed validation, the work addresses a practical challenge in EO applications where sensors may be unavailable due to external factors. It extends prior ideas on temporal dropout and sensor-invariant models with multi-sensor-specific techniques and provides an empirical comparison across datasets.

major comments (1)
  1. [Experiments] The central empirical claim depends on the assumption that the three chosen datasets and the simulated missing-sensor patterns are representative of real-world conditions (see abstract and Experiments section). No sensitivity analysis, justification for the missing-pattern simulation procedure, or discussion of potential biases introduced by the simulation is provided, which is load-bearing for the generalization statements.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their insightful comments on our manuscript. We address the major comment point by point below and commit to revisions where appropriate.

read point-by-point responses

  1. Referee: The central empirical claim depends on the assumption that the three chosen datasets and the simulated missing-sensor patterns are representative of real-world conditions (see abstract and Experiments section). No sensitivity analysis, justification for the missing-pattern simulation procedure, or discussion of potential biases introduced by the simulation is provided, which is load-bearing for the generalization statements.

    Authors: We acknowledge that the manuscript lacks explicit justification for the choice of datasets and the missing sensor simulation procedure, as well as sensitivity analysis and discussion of potential biases. This is a valid point that strengthens the paper's claims. In the revised manuscript, we will add a subsection in the Experiments section detailing the rationale for the datasets and simulation method (e.g., based on common EO scenarios like cloud cover or sensor failure), include sensitivity analysis varying the missing patterns (e.g., random vs. correlated missingness), and discuss possible biases such as over- or under-estimation of robustness. We believe this will better support the generalization statements. revision: yes

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper presents an empirical study of two proposed methods (ISensD and ESensI) for improving robustness to missing sensors in multi-sensor EO models, evaluated on three temporal datasets. No mathematical derivation, fitted parameters renamed as predictions, or load-bearing self-citations appear in the abstract or described claims. The central results are experimental performance comparisons under simulated missing-sensor conditions, which do not reduce to self-definition or input fitting by construction. The work is self-contained as an empirical proposal without any derivation chain that collapses to its own inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Review based on abstract only; no specific free parameters, axioms, or invented entities are described or can be extracted.

pith-pipeline@v0.9.0 · 5687 in / 1120 out tokens · 21357 ms · 2026-05-23T22:48:53.724260+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. On What Depends the Robustness of Multi-source Models to Missing Data in Earth Observation?

    cs.LG 2025-03 unverdicted novelty 4.0

    Multi-source EO models' robustness to missing data depends on task nature, source complementarity, and design, sometimes improving when certain sources are removed.

Reference graph

Works this paper leans on

25 extracted references · 25 canonical work pages · cited by 1 Pith paper

  1. [1]

    Photogrammetric Engineering & Re- mote Sensing 75(5), 577–593 (2009)

    Aksoy, S., Koperski, K., Tusk, C., Marchisio, G.: Land cover classification with multi-sensor fusion of partly missing data. Photogrammetric Engineering & Re- mote Sensing 75(5), 577–593 (2009). https://doi.org/10.14358/PERS.75.5.577

    work page doi:10.14358/pers.75.5.577 2009

  2. [2]

    arXiv preprint arXiv:2404.08351 (2024)

    Astruc,G.,Gonthier,N.,Mallet,C.,Landrieu,L.:OmniSat:Self-supervisedmodal- ity fusion for Earth observation. arXiv preprint arXiv:2404.08351 (2024)

    work page arXiv 2024

  3. [3]

    UCI Machine Learning Repository (2017)

    Chen, S.: PM2.5 Data of Five Chinese Cities. UCI Machine Learning Repository (2017). https://doi.org/10.24432/C52K58

    work page doi:10.24432/c52k58 2017

  4. [4]

    IEEE Transactions on Geoscience and Remote Sensing (2024)

    Chen, Y., Zhao, M., Bruzzone, L.: A novel approach to incomplete multimodal learning for remote sensing data fusion. IEEE Transactions on Geoscience and Remote Sensing (2024). https://doi.org/10.1109/TGRS.2024.3387837

    work page doi:10.1109/tgrs.2024.3387837 2024

  5. [5]

    IEEE Transactions on Geoscience and Remote Sensing60, 1–11 (2021)

    Efremova, N., Seddik, M.E.A., Erten, E.: Soil moisture estimation using Sentinel- 1/-2 imagery coupled with cycleGAN for time-series gap filing. IEEE Transactions on Geoscience and Remote Sensing60, 1–11 (2021). https://doi.org/10.1109/TG RS.2021.3134127

    work page doi:10.1109/tg 2021

  6. [6]

    In: Proceedings of IEEE International Geoscience and Remote Sensing Symposium (2024)

    Follath, T., Mickisch, D., Hemmerling, J., Erasmi, S., Schwieder, M., Demir, B.: Multi-modal vision transformers for crop mapping from satellite image time series. In: Proceedings of IEEE International Geoscience and Remote Sensing Symposium (2024)

    work page 2024

  7. [7]

    IEEE Transactions on Geoscience and Remote Sensing (2024)

    Francis, A.: Sensor independent cloud and shadow masking with partial labels and multimodal inputs. IEEE Transactions on Geoscience and Remote Sensing (2024). https://doi.org/10.1109/TGRS.2024.3391625

    work page doi:10.1109/tgrs.2024.3391625 2024

  8. [8]

    In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Guo, X., Lao, J., Dang, B., Zhang, Y., Yu, L., Ru, L., Zhong, L., Huang, Z., Wu, K., Hu, D., et al.: Skysense: A multi-modal remote sensing foundation model towards universal interpretation for Earth observation imagery. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 27672– 27683 (2024)

    work page 2024

  9. [9]

    arXiv preprint arXiv:2401.07782 (2024) 12 F

    Hackstein, J., Sumbul, G., Clasen, K.N., Demir, B.: Exploring masked autoen- coders for sensor-agnostic image retrieval in remote sensing. arXiv preprint arXiv:2401.07782 (2024) 12 F. Mena et al

    work page arXiv 2024

  10. [10]

    Machine Learning (2023)

    Heinrich, R., Scholz, C., Vogt, S., Lehna, M.: Targeted adversarial attacks on wind power forecasts. Machine Learning (2023). https://doi.org/10.1007/s10994-023-0 6396-9

    work page doi:10.1007/s10994-023-0 2023

  11. [11]

    IEEE Transactions on Geoscience and Remote Sensing59, 4340–4354 (2021)

    Hong, D., Gao, L., Yokoya, N., Yao, J., Chanussot, J., Du, Q., Zhang, B.: More diversemeansbetter:Multimodaldeeplearningmeetsremote-sensingimageryclas- sification. IEEE Transactions on Geoscience and Remote Sensing59, 4340–4354 (2021). https://doi.org/10.1109/TGRS.2020.3016820

    work page doi:10.1109/tgrs.2020.3016820 2021

  12. [12]

    Pattern recognition 41(5), 1718–1731 (2008)

    Ko, A.H., Sabourin, R., Britto Jr, A.S.: From dynamic classifier selection to dy- namic ensemble selection. Pattern recognition 41(5), 1718–1731 (2008). https: //doi.org/10.1016/j.patcog.2007.10.015

    work page doi:10.1016/j.patcog.2007.10.015 2008

  13. [13]

    John Wiley & Sons (2002)

    Little, R.J., Rubin, D.B.: Statistical analysis with missing data. John Wiley & Sons (2002)

    work page 2002

  14. [14]

    Ma, M., Ren, J., Zhao, L., Testuggine, D., Peng, X.: Are multimodal trans- formers robust to missing modality? In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 18177–18186 (2022). https: //doi.org/10.1109/CVPR52688.2022.01764

    work page doi:10.1109/cvpr52688.2022.01764 2022

  15. [15]

    In: Pro- ceedings of IEEE International Geoscience and Remote Sensing Symposium (2024)

    Mena, F., Arenas, D., Charfuelan, M., Nuske, M., Dengel, A.: Impact assessment of missing data in model predictions for Earth observation applications. In: Pro- ceedings of IEEE International Geoscience and Remote Sensing Symposium (2024)

    work page 2024

  16. [16]

    arXiv preprint arXiv:2403.16582 (2024)

    Mena, F., Arenas, D., Dengel, A.: In the search for optimal multi-view learn- ing models for crop classification with global remote sensing data. arXiv preprint arXiv:2403.16582 (2024)

    work page arXiv 2024

  17. [17]

    IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing pp

    Mena, F., Arenas, D., Nuske, M., Dengel, A.: Common practices and taxonomy in deep multi-view fusion for remote sensing applications. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing pp. 4797 – 4818 (2024). https://doi.org/10.1109/JSTARS.2024.3361556

    work page doi:10.1109/jstars.2024.3361556 2024

  18. [18]

    1396–1399

    Pathak, D., Miranda, M., Mena, F., Sanchez, C., Helber, P., Bischke, B., et al.: Predicting crop yield with machine learning: An extensive analysis of input modal- ities and models on a field and sub-field level. In: Proceedings of IEEE In- ternational Geoscience and Remote Sensing Symposium. pp. 2767–2770 (2023). https://doi.org/10.1109/IGARSS52108.2023.10282318

    work page doi:10.1109/igarss52108.2023.10282318 2023

  19. [19]

    Remote Sensing of Environment245, 111797 (2020)

    Rao, K., Williams, A.P., Flefil, J.F., Konings, A.G.: SAR-enhanced mapping of live fuel moisture content. Remote Sensing of Environment245, 111797 (2020). https://doi.org/10.1016/j.rse.2020.111797

    work page doi:10.1016/j.rse.2020.111797 2020

  20. [20]

    ISPRS Journal of Photogram- metry and Remote Sensing187, 294–305 (2022)

    Sainte Fare Garnot, V., Landrieu, L., Chehata, N.: Multi-modal temporal attention models for crop mapping from satellite time series. ISPRS Journal of Photogram- metry and Remote Sensing187, 294–305 (2022). https://doi.org/10.1016/j.isprsj prs.2022.03.012

    work page doi:10.1016/j.isprsj 2022

  21. [21]

    Remote Sensing 10(2), 236 (2018)

    Scarpa, G., Gargiulo, M., Mazza, A., Gaetano, R.: A CNN-based fusion method for feature extraction from Sentinel data. Remote Sensing 10(2), 236 (2018). https://doi.org/10.3390/rs10020236

    work page doi:10.3390/rs10020236 2018

  22. [22]

    Advances in Neural Information Processing25 (2012)

    Srivastava, N., Salakhutdinov, R.R.: Multimodal learning with deep Boltzmann machines. Advances in Neural Information Processing25 (2012)

    work page 2012

  23. [23]

    doi: https: //doi.org/10.1016/j.rse.2019

    Srivastava, S., Vargas-Munoz, J.E., Tuia, D.: Understanding urban landuse from the above and ground perspectives: A deep learning, multimodal solution. Remote Sensing of Environment228, 129–143 (2019). https://doi.org/10.1016/j.rse.2019 .04.014

    work page doi:10.1016/j.rse.2019 2019

  24. [24]

    Tseng, G., Cartuyvels, R., Zvonkov, I., Purohit, M., Rolnick, D., Kerner, H.: Lightweight, pre-trained transformers for remote sensing timeseries (2023)

    work page 2023

  25. [25]

    Tseng,G.,Zvonkov,I.,Nakalembe,C.L.,Kerner,H.:CropHarvest:Aglobaldataset for crop-type classification. Proceedings of NIPS Datasets and Benchmarks (2021) Increasing Robustness of Multi-sensor Model to Missing Sensors 13 Supplementary Material A Dataset setting In Table 3 we display the number of features per sensor in each dataset. Re- garding the data pre...

    work page 2021