REVIEW 3 major objections 6 minor 1 cited by
Domain-specific grouped pretraining on 28 lunar channels beats ImageNet, EO models, and vanilla MAE on six Moon tasks.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-12 00:58 UTC pith:4HUCFAO3
load-bearing objection First multi-modal lunar FM benchmark with real domain-specific gains; engineering is careful, label caveats are ordinary, and EO transfer is honestly weak. the 3 major comments →
Moonstone: A Multimodal Foundation Model and Benchmark for Lunar Remote Sensing
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
MG-MAE features pretrained on the assembled 28-channel lunar dataset outperform scratch, ImageNet-pretrained, vanilla MAE, and transferred Earth-observation foundation-model baselines on every one of six fixed downstream tasks, with the largest reported gains on 49-class geology classification and crater segmentation; Earth models transfer only marginally, so domain-specific grouped pretraining is required.
What carries the argument
MG-MAE: a modality-grouped masked autoencoder that uses one multi-channel convolutional tokenizer per physical instrument family, a shared Vision Transformer encoder with key-dimension attention masking for entirely missing groups, coverage-adaptive mask ratios, complementary anchor masking, cross-modal decoder attention, proportional reconstruction loss, InfoNCE alignment, and spectral-continuity regularization on the M3 reflectance bands.
Load-bearing premise
The six constructed labels—especially majority-vote 49-class geologic units on 256-pixel patches and composition targets interpolated from roughly 60 km native resolution onto a 237 m grid—are accurate and scale-matched enough that the measured gains reflect real scientific utility rather than label artifacts.
What would settle it
Re-run the identical linear-probe and finetune protocol after replacing the USGS majority-vote geology labels and the interpolated LP-GRS composition targets with independent higher-resolution ground truth or expert-reviewed maps; if the large MG-MAE gains disappear or reverse, the central claim fails.
If this is right
- New instruments can be added as extra modality groups without redesigning the encoder, because missing-modality masking already handles heterogeneous coverage.
- Polar and other under-covered regions become usable test beds rather than automatic out-of-distribution failures, since the model is trained to remain well-defined when groups contribute zero tokens.
- Few-shot and low-label regimes for geology and age improve substantially, lowering the barrier for resource-prospecting maps that currently rest on a handful of sample-return sites.
- Task-specific lunar models that rely on hand-crafted spectral indices or single-sensor U-Nets are dominated by the multi-modal pretrained features on the shared benchmark.
Where Pith is reading between the lines
- The same coverage-adaptive and ghost-token machinery should transfer to other airless bodies that also have sparse multi-mission archives (Mercury, asteroids) once comparable global stacks exist.
- Because composition linear probing already exceeds finetuning, further gains may come more from better spectral regularization or multi-scale fusion than from simply scaling the ViT.
- The near-zero transfer from Earth models suggests that any future multi-body foundation model will need explicit planetary domain adapters rather than pure continual pretraining on mixed EO–lunar data.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Moonstone, the first multi-modal foundation-model benchmark for lunar remote sensing. It assembles a 28-channel global pretraining corpus at 128 ppd (~237 m) from seven instrument families across five missions, proposes MG-MAE (a modality-grouped masked autoencoder with per-group Conv tokenizers, a shared ViT-Base encoder, key-dimension attention masking for missing modalities, coverage-adaptive masking, complementary masking, proportional reconstruction weighting, InfoNCE alignment, and spectral continuity regularization on M3), and defines six downstream tasks (49-class geology, 5-class age, FeO/TiO2 regression, cross-modal thermal prediction, mare and crater segmentation). Across linear probe and finetune protocols, MG-MAE features outperform scratch, ImageNet ViT-B, vanilla MAE, single-modality, and EO-transfer baselines (SatMAE, Prithvi-EO-2.0, TerraMind with linear adapters), with largest gains on geology (+16.2% Acc) and craters (+14.7 mIoU). Ablations, geographic latitude-band splits, few-shot trials, and public data/code support the claims.
Significance. If the reported margins hold under independent re-runs, this is a substantial and timely contribution: it closes a clear gap relative to EO benchmarks (GEO-Bench, PANGAEA) and Mars-Bench by providing a reusable multi-modal lunar pretraining corpus, a domain-specific MAE design that addresses non-uniform instrument coverage, and a fixed six-task evaluation suite. Strengths that raise the bar include public HuggingFace/GitHub release of data, pipeline, and checkpoint; systematic ablations isolating missing-modality masking, grouping, coverage-adaptive masking, and spectral continuity; geographic splits; few-shot stds; and fair EO-transfer adapters that preserve pretrained patch embeddings. Domain-specific pretraining is shown to matter far more than EO transfer (+1.9% geology for the best EO model vs. +16.2% for MG-MAE), which is a useful negative result for planetary ML.
major comments (3)
- Table 3 reports single-point Acc/R2/mIoU for ImageNet, Scratch, Vanilla MAE, Linear, and Finetune with no multi-seed means or standard deviations (unlike Table 4 few-shot). The central claim of large, consistent margins (e.g., geology 40.1→56.3, craters 0.621→0.768) is load-bearing; without run-to-run variance or at least 3 seeds for the main table, it is hard to judge whether the ranking is stable under training noise. Please add multi-seed statistics for Table 3 (or a compact appendix table) for Scratch, Vanilla MAE, and MG-MAE linear/finetune on at least geology, composition, and craters.
- §3.3 and the composition task definition: LP-GRS FeO/TiO2 have ~60 km effective resolution and are RBF-interpolated onto the 237 m grid, so patch-level R2 largely measures recovery of smooth regional fields rather than fine-scale geochemistry. The paper notes this briefly, but Table 3’s R2=0.924 (linear) is easy to over-interpret as high-resolution composition skill. Please state the resolution mismatch more prominently in the task definition and Discussion (and, if feasible, report a coarse-grid or low-pass evaluation) so absolute R2 is not read as pixel-scale mineral mapping performance. Relative ranking vs. baselines remains informative either way.
- §5.6 / Table 7 EO transfer: the learned linear adapter that maps 28 lunar channels to each EO model’s expected input is a reasonable fairness choice, but the manuscript should state adapter training protocol (frozen vs. joint, data split, whether adapter is trained only on lunar pretraining crops or also on downstream labels) and confirm that EO models are not disadvantaged by an under-trained adapter. A short sensitivity check (e.g., adapter-only vs. full finetune already reported) would make the “EO transfer is near-scratch” claim more airtight.
minor comments (6)
- Fig. 1 caption and §4.1: complementary masking and coverage-adaptive ratios are described in text and Algorithm 1 (supp.), but the main figure’s “60–85%” range should explicitly cite Eq. (4) / mg = 0.75 + α(cg − c̄) so readers can reproduce the per-group visible-token counts in supp. Table 2.
- Eq. (1)–(2): proportional weights wg = ℓg / Σℓg′ are self-balancing; clarify whether ℓg is the current mini-batch loss, EMA, or epoch-level average, and whether gradients flow through wg (stop-grad is standard for such schemes).
- Table 8 ablation “No spectral continuity reg.” improves composition by +1.3 R2 points in the text but the table shows 0.911 vs 0.924; keep wording and table consistent and specify that SCR applies only to the 8 M3 bands.
- §3.2 Mini-RF log(1+x) and Diviner channel naming (diviner_temp_night as rock-free regolith temperature) are important for reusers; consider a one-sentence pointer in the main text to the supp. quality-corrections subsection.
- Typos/clarity: “Wedefinesix” / “WeproposeMG-MAE” spacing glitches in the introduction PDF; “Geographicgeneralization” and similar concatenated headings in Discussion; arXiv line-break artifacts in URLs.
- Related work: Mars-Bench is cited; a sentence on how Moonstone’s multi-modal pretraining + fixed splits differs from evaluating EO/ImageNet models on Mars-only labels would help position the benchmark.
Circularity Check
No significant circularity: empirical ML benchmark with external baselines and ablations; no prediction reduces to a fitted input by construction.
full rationale
Moonstone is a systems/benchmark paper: it assembles a multi-modal lunar dataset, defines MG-MAE (grouped tokenizers, missing-modality key masking, coverage-adaptive mask ratios, spectral continuity L2, proportional reconstruction weights, complementary masking), and reports transfer metrics on six constructed tasks against scratch, ImageNet ViT-B, vanilla MAE, EO models (SatMAE, Prithvi, TerraMind), single-modality baselines, and task-specific methods (U-Net, CNN, Lucey band ratios, RF). The central claim is comparative performance (e.g., geology +16.2% Acc, craters +14.7 mIoU), not a first-principles derivation of a physical constant. Coverage-adaptive masking (mg = 0.75 + α(cg − c̄)) and proportional loss weights (wg = ℓg / Σℓg′) are training hyperparameters; they do not define the reported Acc/R²/mIoU. Ablations (Tab. 8) remove components while holding data fixed; geographic splits, few-shot, and EO-transfer experiments use the same labels for all methods. No self-citation is load-bearing for uniqueness; no uniqueness theorem is invoked; no fitted parameter is renamed as a prediction of a closely related quantity. Label construction (majority USGS unit, LP-GRS interpolation) is an ordinary benchmark design choice, not circularity. Score 0 is appropriate.
Axiom & Free-Parameter Ledger
free parameters (5)
- coverage-adaptive alpha =
0.15
- InfoNCE weight lambda and temperature tau =
lambda=0.1, tau=0.07
- spectral continuity weight mu =
0.01
- base mask ratio and complementary schedule =
0.75 / p=0.5 / 0.90
- selected M3 band subset =
8 bands (750–2857 nm)
axioms (5)
- domain assumption USGS Unified Geologic Map majority labels on 256×256 patches are sufficiently accurate 49-class ground truth for measuring representation quality.
- domain assumption LP-GRS FeO/TiO2 fields at ~60 km native resolution, after RBF interpolation to 237 m, remain valid regression targets for regional geochemical trends.
- ad hoc to paper Key-dimension-only attention masking of ghost tokens is numerically stable and does not discard useful gradient signal.
- ad hoc to paper A learned linear adapter that maps 28 lunar channels onto an EO model’s expected input dimensionality yields a fair transfer comparison while preserving pretrained patch embeddings.
- domain assumption Standard ViT-Base MAE reconstruction plus InfoNCE produces transferable multi-modal features when trained on unlimited random crops of a single global map.
invented entities (2)
-
MG-MAE (modality-grouped MAE with coverage-adaptive masking and spectral continuity regularization)
independent evidence
-
Moonstone 28-channel pretraining corpus and six-task benchmark
independent evidence
read the original abstract
Decades of orbital missions have produced multi-modal remote sensing data for the Moon, spanning optical imagery, spectroscopy, thermal emission, radar, gravity, and elemental composition. Yet these datasets remain fragmented across archives, and no benchmark exists for evaluating machine learning on lunar data. We introduce Moonstone, the first multi-modal foundation model benchmark for lunar remote sensing. Our contributions are: (1) a 28-channel, 128 pixels-per-degree (~237 m) global lunar pretraining dataset from seven instrument families across five missions, (2) MG-MAE, a modality-grouped masked autoencoder with per-group convolutional tokenizers, a shared Vision Transformer encoder, attention masking for missing modalities, coverage-adaptive masking for heterogeneous spatial coverage, and spectral continuity regularization for physically plausible reconstructions, and (3) a benchmark of six downstream tasks covering classification, regression, and segmentation. MG-MAE pretrained features outperform scratch baselines on all tasks and surpass both ImageNet-pretrained and vanilla MAE baselines by large margins. Data and code are available at https://huggingface.co/datasets/ayushprd/Moonstone and https://github.com/ayushprd/Moonstone .
Figures
Forward citations
Cited by 1 Pith paper
-
LunarFM: A Shared Multimodal Representation of the Moon's Surface
A self-supervised multimodal model fuses 18 channels from six lunar instruments into a shared 768-dimensional embedding per 0.5° chip, enabling mineral regression, similarity search, and geological-unit classification...
Reference graph
Works this paper leans on
-
[1]
In: CVPR (2025) 2
Astruc, G., Gonthier, N., Mallet, C., Landrieu, L.: AnySat: One earth observation model for many resolutions, scales, and modalities. In: CVPR (2025) 2
2025
-
[2]
In: ECCV (2022).https://doi.org/10.1007/978-3- 031-19836-6_202, 3, 6, 11
Bachmann, R., Mizrahi, D., Atanov, A., Zamir, A.: MultiMAE: Multi-modal multi- task masked autoencoders. In: ECCV (2022).https://doi.org/10.1007/978-3- 031-19836-6_202, 3, 6, 11
doi:10.1007/978-3- 2022
-
[3]
In: ICLR (2022) 2
Bao, H., Dong, L., Piao, S., Wei, F.: BEiT: BERT pre-training of image transform- ers. In: ICLR (2022) 2
2022
-
[4]
Acta Astronautica226, 375–384 (2025).https://doi.org/10.1016/j
Calzada Diaz, A., Keszthelyi, L.: Descriptive models for lunar high-Ti de- posits. Acta Astronautica226, 375–384 (2025).https://doi.org/10.1016/j. actaastro.2024.10.0315
doi:10.1016/j 2025
-
[5]
In: ICCV (2021) 2
Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging properties in self-supervised vision transformers. In: ICCV (2021) 2
2021
-
[6]
In: NeurIPS (2022) 2, 3, 11, 12
Cong, Y., Khanna, S., Meng, C., Liu, P., Rozi, E., He, Y., Burke, M., Lobell, D., Ermon, S.: SatMAE: Pre-training transformers for temporal and multi-spectral satellite imagery. In: NeurIPS (2022) 2, 3, 11, 12
2022
-
[7]
In: ICLR (2021) 2, 9
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. In: ICLR (2021) 2, 9
2021
-
[8]
In: 51st Lunar and Planetary Science Conference, Abstract #2760 (2020) 4, 5
Fortezzo,C.,Spudis,P.,Harrel,S.:Releaseofthedigitalunifiedglobalgeologicmap of the moon at 1:5,000,000-scale. In: 51st Lunar and Planetary Science Conference, Abstract #2760 (2020) 4, 5
2020
-
[9]
Green, R., Pieters, C., Mouroulis, P., Eastwood, M., Boardman, J., Glavich, T., Isaacson, P., Annadurai, M., Besse, S., Barr, D., et al.: The moon mineralogy mapper (M3) imaging spectrometer for lunar science: Instrument description, cal- ibration, on-orbit measurements, science data calibration and on-orbit validation. J. Geophys. Res.: Planets116(E10), ...
2011
-
[10]
In: CVPR (2024) 2
Guo, X., Lao, J., Dang, B., Zhang, Y., Yu, L., Ru, L., Zhong, L., Huang, Z., Wu, K., Hu, D., et al.: SkySense: A multi-modal remote sensing foundation model towards universal interpretation for earth observation imagery. In: CVPR (2024) 2
2024
-
[11]
He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.: Masked autoencoders are scalable vision learners. In: CVPR. pp. 15979–15988 (2022).https://doi.org/ 10.1109/CVPR52688.2022.015532, 6, 9
-
[12]
In: ICCV (2025) 2, 11, 12
Jakubik, J., Yang, F., Blumenstiel, B., Scheurer, E., Sedona, R., Maurogiovanni, S., Bosmans, J., Dionelis, N., Marsocci, V., Kopp, N., et al.: TerraMind: Large-scale generative multimodality for earth observation. In: ICCV (2025) 2, 11, 12
2025
-
[13]
In: IEEE ITAIC (2020).https://doi.org/10.1109/ ITAIC49862.2020.93391794, 13, 14
Jia, Y., Wan, G., Liu, L., Wu, Y., Zhang, C.: Automated detection of lunar craters using deep learning. In: IEEE ITAIC (2020).https://doi.org/10.1109/ ITAIC49862.2020.93391794, 13, 14
arXiv 2020
-
[14]
In: NeurIPS (2023) 2, 3
Lacoste, A., Lehmann, N., Rodriguez, P., Sherwin, E., Kerner, H., et al.: GEO- Bench: Toward foundation models for earth monitoring. In: NeurIPS (2023) 2, 3
2023
-
[15]
In: ICLR (2019) 8
Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: ICLR (2019) 8
2019
-
[16]
Journal of Geo- physical Research: Planets105(E8), 20297–20305 (2000).https://doi.org/10
Lucey, P., Blewett, D., Jolliff, B.: Lunar iron and titanium abundance algorithms based on final processing of clementine ultraviolet-visible images. Journal of Geo- physical Research: Planets105(E8), 20297–20305 (2000).https://doi.org/10. 1029/1999JE00111713, 14 Moonstone: A Multimodal FM and Benchmark for Lunar Remote Sensing 17
2000
-
[17]
arXiv preprint arXiv:2412.04204 (2024) 2, 3
Marsocci, V., Jia, Y., Le Bellier, G., Kerekes, D., Zeng, L., et al.: PANGAEA: A global and inclusive benchmark for geospatial foundation models. arXiv preprint arXiv:2412.04204 (2024) 2, 3
Pith/arXiv arXiv 2024
-
[18]
In: ICCV (2023) 2
Mendieta, M., Han, B., Shi, X., Zhu, Y., Chen, C.: Towards geospatial foundation models via continual pretraining. In: ICCV (2023) 2
2023
-
[19]
The Planetary Science Journal1(2), 32 (2020).https://doi.org/10.3847/PSJ/ab9a524
Moseley, B., Bickel, V., Burelbach, J., Relatores, N.: Unsupervised learning for thermophysical analysis on the lunar surface. The Planetary Science Journal1(2), 32 (2020).https://doi.org/10.3847/PSJ/ab9a524
-
[20]
In: ECCV (2024) 3
Nedungadi, V., Kariryaa, A., Oehmcke, S., Belongie, S., Igel, C., Lang, N.: MMEarth:Exploringmulti-modalpretexttasksforgeospatialrepresentationlearn- ing. In: ECCV (2024) 3
2024
-
[21]
Space Science Reviews150, 285–302 (2010)
Nozette, S., Spudis, P., Bussey, B., et al.: The LRO miniature radio frequency (Mini-RF) technology demonstration. Space Science Reviews150, 285–302 (2010). https://doi.org/10.1007/s11214-009-9607-54
-
[22]
arXiv preprint arXiv:1807.03748 (2018) 7
van den Oord, A., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018) 7
Pith/arXiv arXiv 2018
-
[23]
TMLR (2024) 2
Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al.: DINOv2: Learning robust visual features without supervision. TMLR (2024) 2
2024
-
[24]
Space Science Reviews150, 125–160 (2010).https://doi.org/10.1007/s11214-009-9529-24
Paige, D., Foote, M., Greenhagen, B., Schofield, J., Calcutt, S., Vasavada, A., Preston, D., Taylor, F., Allen, C., Snook, K., et al.: The lunar reconnaissance orbiter diviner lunar radiometer experiment. Space Science Reviews150, 125–160 (2010).https://doi.org/10.1007/s11214-009-9529-24
-
[25]
Current Science96(4), 500–505 (2009) 1, 4
Pieters, C., Boardman, J., Buratti, B., et al.: The moon mineralogy mapper (M3) on chandrayaan-1. Current Science96(4), 500–505 (2009) 1, 4
2009
-
[26]
In: NeurIPS (2025) 3
Purohit, M., Gajera, B., Malaviya, V., Mehta, I., Kasodekar, K., Adler, J., Lu, S., Rebbapragada, U., Kerner, H.: Mars-Bench: A benchmark for evaluating founda- tion models for mars science tasks. In: NeurIPS (2025) 3
2025
-
[27]
In: ICCV (2023) 2
Reed, C., Gupta, R., Li, S., Brockman, S., Funk, C., Clipp, B., Keutzer, K., Can- dido, S., Uyttendaele, M., Darrell, T.: Scale-MAE: A scale-aware masked autoen- coder for multiscale geospatial representation learning. In: ICCV (2023) 2
2023
-
[28]
crater locations and sizes, comparisons with published databases, and global analysis
Robbins, S.: A new global database of lunar impact craters>1–2 km: 1. crater locations and sizes, comparisons with published databases, and global analysis. J. Geophys. Res.: Planets124(4), 871–892 (2019).https://doi.org/10.1029/ 2018JE0055924
2019
-
[29]
Space Science Reviews150, 81–124 (2010)
Robinson, M., Brylow, S., Tschimmel, M., et al.: Lunar reconnaissance orbiter camera (LROC) instrument overview. Space Science Reviews150, 81–124 (2010). https://doi.org/10.1007/s11214-010-9634-21, 4
-
[30]
Icarus317, 27–38 (2019).https://doi.org/10.1016/j.icarus.2018.06.0224, 13, 14
Silburt, A., Ali-Dib, M., Zhu, C., Jackson, A., Valencia, D., Kissin, Y., Tamayo, D., Menou, K.: Lunar crater identification via deep learning. Icarus317, 27–38 (2019).https://doi.org/10.1016/j.icarus.2018.06.0224, 13, 14
-
[31]
Smith, D., Zuber, M., Neumann, G., Lemoine, F., Mazarico, E., Torrence, M., McGarry, J., Rowlands, D., Head, J., Duxbury, T., et al.: Initial observations from the lunar orbiter laser altimeter (LOLA). Geophys. Res. Lett.37(18) (2010). https://doi.org/10.1029/2010GL0437514
-
[32]
Szwarcman, D., Roy, S., Fraccaro, P., Jakubik, J., et al.: Prithvi-EO-2.0: A versatile multi-temporalfoundationmodelforearthobservationapplications.arXivpreprint arXiv:2412.02732 (2024) 2, 11, 12
arXiv 2024
-
[33]
In: ICML (2025) 2 18 A
Tseng, G., Fuller, A., Reil, M., Herzog, H., Beukema, P., Bastani, F., Green, J., Shelhamer, E., Kerner, H., Rolnick, D.: Galileo: Learning global and local features of many remote sensing modalities. In: ICML (2025) 2 18 A. Prasad and S. Mazumder
2025
-
[34]
USGS Professional Paper 1348, U.S
Wilhelms, D.: The Geologic History of the Moon. USGS Professional Paper 1348, U.S. Government Printing Office (1987) 4
1987
-
[35]
Zuber,M.,Smith,D.,Watkins,M.,etal.:Gravityfieldofthemoonfromthegravity recovery and interior laboratory (GRAIL) mission. Science339(6120), 668–671 (2013).https://doi.org/10.1126/science.12315071, 4 Moonstone: A Multimodal FM and Benchmark for Lunar Remote Sensing 19 Supplementary Material Moonstone: A Multimodal Foundation Model and Benchmark for Lunar Re...
-
[36]
confirms a +3.3% geology accuracy gain from grouped tokenization. 6.2 Dirichlet vs. Complementary Masking The per-channel architecture used Dirichlet(α=1) sampling to allocate a token budget across available channels: for each sample, a random fraction was drawn from a Dirichlet distribution to determine how many tokens each channel re- tained. While math...
arXiv 2000
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.