{"id":"4d2af05a-e9ac-41dd-8c92-2eaaeceebe4a","arxiv_id":"2507.07527","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"MAPEX shows that a modality-conditioned mixture-of-experts vision transformer, pre-trained on six remote sensing modalities and then pruned to keep only the experts for a target modality, can outperform or match larger foundation models on single-modality downstream tasks.","lead":"MAPEX pre-trains one large multi-sensor model whose internal 'experts' are specialized for single data types like radar, infrared, or elevation, then prunes away unused experts to produce small, task-specific models. The paper reports that these pruned models match or beat much larger remote sensing foundation models on flood, wildfire, and land-cover tasks, though the main evaluation set appears to overlap with the pre-training data.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Main land-cover benchmark may be contaminated: ben-ge-8k is a subset of ben-ge, and the paper never states it was held out from pre-training; if not, Tables 1, 4, and the ablations are inflated, weakening the central comparison.","rationale":"The paper's central contribution is the combination of modality-conditioned MoE pre-training and expert pruning to yield compact modality specialists. The flood and wildfire experiments provide external evidence that supports the pipeline, and the architectural design is coherent. The load-bearing weakness is the ben-ge-8k benchmark: because pre-training is performed on ben-ge and ben-ge-8k is described only as 'a subset of ben-ge,' the paper does not rule out train/evaluation overlap. This matters more than the individual numbers because k-NN accuracy especially is sensitive to patch-level memorization, and because all hyperparameter choices (top-k, dropout, routing, shared expert) are tuned on this same set. The issue is not an internal contradiction; it is an omitted condition. If the authors can show disjointness using a standard split or exact sample IDs, the concern disappears. I therefore agree with the reader's conditional verdict and see no need to move it.","tokens_in":13562,"tokens_out":4484,"duration_ms":46797,"concrete_test":"Request the exact split-generation script and sample IDs for ben-ge-8k, and verify disjointness from the pre-training set (e.g., compare tile patch names or coordinates). If any overlap exists, rerun the k-NN and fine-tuning experiments in Tables 1 and 4 with a held-out split (or the official BigEarthNet test split) and report whether MAPEX's margins over SatMAE/Scale-MAE persist; also re-run the top-k and dropout ablations on the clean split.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central quantitative claim rests on ben-ge-8k land-cover results (Tables 1, 4, 5, 6, 7, 8, 9, 10). Section 4 states that ben-ge is used for multi-modal pre-training and that ben-ge-8k is 'a subset of ben-ge' used for downstream evaluation, but it never states that the 8,000 samples are disjoint from the pre-training split. If ben-ge-8k contains pre-training patches, k-NN and fine-tuning accuracies on it are inflated by memorization, which would disproportionately advantage MAPEX over SatMAE/Scale-MAE because MAPEX was pre-trained on the same data. The ablation choices (dropout value, top-k, routing mechanism, shared expert, expert size) are all validated on this same benchmark, so hyperparameter selection would also be compromised. The flood (SEN12-FLOOD) and wildfire (California Fire Perimeters + Sentinel-2) results are external and provide independent support, but the paper's headline comparison for land-cover, as well as the 'specialized models' claim in Figure 6, depend on the overlap-free assumption. This is a missing-support issue rather than a demonstrated inconsistency; it can be settled by releasing the exact split.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces MAPEX, a remote-sensing foundation model that combines masked autoencoding with a mixture-of-modality experts encoder and a router conditioned on modality tokens. After pre-training on BigEarthNet-GeoEnvironment (ben-ge), the authors prune all experts except the top-k routed experts for a target modality, yielding a smaller modality-specialized model that is then evaluated on land-cover classification and segmentation on ben-ge-8k, flood detection on SEN12-FLOOD, and wildfire detection on California Fire Perimeters. The reported results show MAPEX matching or exceeding SatMAE and Scale-MAE on several modalities with fewer parameters, and the paper also presents ablations of modality dropout, shared expert, routing mechanism, and expert count.","tokens_in":13868,"tokens_out":5648,"duration_ms":61925,"significance":"If the ben-ge-8k evaluation were based on samples held out from pre-training, the paper would make a useful practical contribution: converting a large multi-modal MoE into compact per-modality models with competitive accuracy, plus external validation on flood and wildfire tasks. The promise of code release and the use of independent SEN12-FLOOD and California Wildfire benchmarks are strengths. However, the central land-cover comparisons and all ablations rely on ben-ge-8k, a subset of the very dataset used for pre-training, and the manuscript never states that this subset is disjoint from pre-training. This missing-support issue is currently the main barrier to accepting the quantitative claims.","major_comments":[{"comment":"The evaluation benchmark ben-ge-8k is described as “a subset of ben-ge” (§4), while ben-ge is used for pre-training (§4, §5.2); the paper never states that the 8,000 samples were excluded from pre-training. If they were not excluded, the k-NN and fine-tuning numbers in Tables 1, 4, 5, 6, 7, 8, 9, and 10 and Figures 4–6 could reflect memorization rather than transfer, and the comparison with SatMAE and Scale-MAE, which were not pre-trained on ben-ge, would be unfair. This is a load-bearing point for the paper’s central claim. Please report the exact split protocol, provide sample identifiers, and either confirm the holdout or re-run the ben-ge-8k experiments on a genuinely held-out subset.","section":"§4 and §5.1"},{"comment":"Because the routing weights are computed as w_m^r = r([MODALITY]_m), the router input already encodes the modality identity, so the observation that each modality activates a consistent subset of experts is guaranteed by construction. The paper interprets the heatmap in Figure 6 as evidence that “routing successfully instills expert–modality relationships” into the model; this conflates a designed conditioning mechanism with emergent specialization. I would ask the authors to reframe the claim or test whether a token-based router with load balancing produces measurably different routing behavior, so that the “specialization” result is informative rather than circular.","section":"§3.2 and Figure 6"},{"comment":"The pruning rule in Equation (2) is validated mainly on the same ben-ge-8k benchmark, so the choice of top-2 experts and the expert-size trade-offs in Figure 5 inherit the contamination risk described above. In addition, the notation “arg topk(w_m^r)” is ambiguous: w_m^r is introduced as the routing probability for a modality, and the equation would benefit from an explicit definition of the top-k operator over the expert set. Please clarify the notation and, if possible, validate the pruning choice on one of the external benchmarks (SEN12-FLOOD or California Wildfire) as well.","section":"§3.3 and Figure 5"}],"minor_comments":[{"comment":"The phrase “ben-ge-8k consisting of ≈8 000samples” is missing a space; it should be “≈8,000 samples.”","section":"§4"},{"comment":"Equation (2) uses “arg topk” without a definition; please define it as returning the indices of the k experts with the largest routing probabilities for modality m.","section":"§3.3"},{"comment":"For the SEN12-FLOOD and California Wildfire datasets, please specify the exact train/test split and whether the split avoids spatial or temporal leakage (e.g., same-fire-perimeter patches appearing in both training and test).","section":"§5.1, Tables 2 and 3"},{"comment":"The “Supervised FS” column reports fine-tuning accuracy, but the caption does not state this explicitly; please clarify the protocol and architecture for that baseline.","section":"Supplementary Material, Table 9"},{"comment":"The color scale and the “best and worst specialized models” highlights are difficult to read in the printed figure; adding a colorbar and explicit labels or annotations would improve interpretability.","section":"Figure 6"},{"comment":"The discussion of MoMa [29] and DAMEX [23] would be clearer with one sentence explicitly stating what MAPEX adds relative to MoMa’s modality-aware routing, since both use modality-specific experts.","section":"§2, Related Work"}],"recommendation":"major_revision","confidential_remarks":"The main risk is the ben-ge-8k contamination issue: the paper’s headline land-cover results and ablations all depend on a subset of the pre-training corpus, and the manuscript does not establish a held-out split. This is fixable by releasing the exact split and re-running if necessary, but until then the central quantitative comparison is not fully reliable. The external flood and wildfire experiments provide independent support, but they cover only two modalities and do not substitute for a clean land-cover benchmark. I would also encourage the authors to soften the 'specialization' claim, since the router is conditioned on modality identity by design."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The paper has a genuinely useful idea: pre-train one multi-modal mixture-of-experts model with modality-conditioned routing, then prune it into per-modality specialists for downstream tasks. That is a practical answer to a real problem in remote sensing, where foundation models are big and usually single-modality. The routing mechanism (learnable modality tokens as router input) and the pruning recipe are the new bits, and the paper evaluates them across six modalities and three tasks, including a new wildfire dataset. Credit where it is due: the flood and wildfire experiments use data that is external to the pre-training set, and the pruned models are competitive with much larger baselines. Code is promised. This is an honest efficiency contribution, not a hype piece.\n\nThe soft spot is load-bearing. Section 4 calls ben-ge-8k \"a subset of ben-ge,\" and Section 5.1 reports all land-cover results on it, but the paper never states that these 8,000 samples were held out from pre-training. If they were included, then Tables 1, 4, 5, 6, 7, 8, 9, and 10 are inflated by memorization, and the ablation choices (dropout, top-k, routing mechanism) were tuned on the same contaminated set. That would disproportionately advantage MAPEX over SatMAE and Scale-MAE, since MAPEX was pre-trained on the same corpus. This is a missing-support issue rather than a demonstrated contradiction, and it can be settled by releasing the exact split. But until that split is stated and the numbers re-verified, the central quantitative claim for land-cover classification should not be taken at face value.\n\nTwo smaller points. The specialization heatmap in Figure 6 partly confirms the design rather than testing it, because routing is conditioned on the modality label; that is not fatal, but it is weaker evidence than the text implies. And there are no error bars anywhere, which matters for comparing close numbers like the 3% and 1% margins on flood and wildfire.\n\nWho is this for? Remote sensing practitioners who want a single multi-modal pre-trained model that can be cheaply deployed on one or two modalities. The paper deserves a serious referee. If the ben-ge split is clean, it is a solid contribution. If not, the land-cover claims need revision and re-running on a proper holdout. My recommendation: send it to peer review with a clear request for the exact train/val/test split and re-evaluation of the land-cover tables. I would not cite the land-cover numbers until that is resolved, but the external results are worth following up.","headline":"Useful efficiency idea, but the main land-cover benchmark may be contaminated by pre-training overlap; the external flood and wildfire results are the cleaner evidence.","tokens_in":14380,"tokens_out":1649,"would_cite":false,"duration_ms":19867,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MAPEX shows that a mixture-of-modality experts model pre-trained on six remote sensing modalities can be pruned into per-modality specialists that match or beat much larger single-modality foundation models on flood, wildfire, and…","keywords":["mixture-of-experts","modality-aware routing","expert pruning","remote sensing foundation models","self-supervised pre-training","masked autoencoding","flood detection","wildfire detection"],"falsifier":"Check whether any sample identifiers in the ben-ge-8k evaluation subset also appear in the ben-ge pre-training split; if they do, or if the split is not documented, re-run Tables 1, 5, 6, 7, and 10 on a held-out set and compare. A second check would be to train the same pipeline with token-based routing and show that pruned experts no longer concentrate on their intended modality, which would indicate the routing mechanism, not the benchmark, is responsible for specialization.","tokens_in":13363,"feed_emoji":"🛰️","tokens_out":5453,"duration_ms":53291,"temperature":0.7,"pith_summary":"MAPEX aims to resolve a mismatch in remote sensing foundation models: pre-training is usually done on one modality, while real tasks use SAR, SWIR, elevation, or other inputs. The paper's proposal is to pre-train one mixture-of-modality experts transformer on six co-located modalities, with routing that forces input tokens of each modality onto dedicated experts, and then, for each downstream task, prune the network down to the top experts of the task's modality. This produces a small, modality-specialized model that the authors show can be fine-tuned as easily as a single-modality model. The central quantitative claim is that these pruned models outperform or match much larger single-modality foundation models: on flood detection from SAR, MAPEX beats SatMAE by 3% and Scale-MAE by 1% in accuracy, and on wildfire detection from SWIR it beats SatMAE by 2% and matches Scale-MAE while using far fewer parameters. If correct, this means one multi-modal pre-training run can supply many cheap per-modality specialists, including for modalities that currently have no dedicated foundation model.","feed_headline":"Small pruned experts beat larger remote sensing models","feed_subtitle":"One multi-modal model prunes into per-modality specialists that match or beat much larger baselines.","key_machinery":"The central mechanism is the mixture-of-modality experts block, where the router does not look at token content but at a learnable [MODALITY] embedding associated with each input modality; this guarantees that every token of a modality is routed to the same subset of experts, producing explicit expert-to-modality structure. Around this sit a load-balancing loss that keeps expert utilization uniform, a shared expert that processes all modalities to absorb common features, and modality dropout that randomly zeros entire modalities during pre-training. Downstream, pruning simply keeps the top-k experts by routing weight for the modality of interest and drops the rest, turning a 360M-parameter pre-training model into 130M-parameter task models.","core_discovery":"The paper claims that modality-conditioned routing during masked autoencoding pre-training creates experts that are genuinely aligned with modalities, and that this alignment survives pruning. Concretely, the authors report that after keeping only the two experts most strongly routed to a given modality, the resulting sub-network performs best on that modality's data, and that these pruned models reach 75.6% fine-tuned accuracy on RGB land-cover classification, 77.8% on red edge, 79.8% on SWIR, 59.4% on NIR, 73.6% on SAR, and 58.8% on elevation, topping the comparison on NIR, SAR, and elevation. They also report that modality dropout of 50% during pre-training improves downstream k-NN accuracy by about 1.5% absolute on average, indicating that forcing the model to handle missing modalities helps the pruning stage. The paper's stated conclusion is that the architecture combines the benefit of large-scale multi-modal pre-training with the deployment convenience of small specialized models.","pith_inferences":["If the ben-ge-8k evaluation subset overlaps the ben-ge pre-training set, the reported accuracies, especially the k-NN results, would be inflated; the paper does not state that the 8,000 samples were held out, so the cleanest test is re-evaluating on a disjoint split.","The pruning rule is validated only on the same benchmark used to justify it; an independent check would be to pre-train with a random or token-based router and show that top-2 pruning then does not yield the same specialization.","The method is presented as modality-aware, but nothing in the routing stops the shared expert from carrying task information; ablating or shrinking the shared expert could reveal how much of the pruning benefit comes from the modality experts alone.","A natural extension is continual addition of new modalities: because routing is conditioned on an embedding, a new sensor could be inserted with its own [MODALITY] token and fine-tuned while leaving existing experts frozen, though the paper does not run this experiment."],"forward_implications":["A single multi-modal pre-training run can produce one specialized sub-model per modality, so new tasks on under-served modalities such as elevation, NIR, or SAR no longer need a purpose-built pre-trained model.","Downstream fine-tuning becomes cheaper because the pruned model is roughly a third of the pre-training size, and pruning is done before fine-tuning so memory and latency shrink.","Because routing is deterministic per modality, deployment can mix and match: keeping experts for RGB and SAR together produces a multimodal model with no further architecture changes.","The fixed top-2 pruning rule is a practical default: in the size-scaling experiment, most modalities peak at 2 experts per modality, while RGB prefers 5 smaller experts, so a fixed count is a compromise rather than an optimum."],"supporting_citations":[{"why":"Supplies the ben-ge dataset for multi-modal pre-training and the ben-ge-8k subset for land-cover evaluation.","marker":"[32]"},{"why":"SatMAE is the main ViT-base multispectral baseline that MAPEX is compared against on classification and segmentation.","marker":"[10]"},{"why":"Scale-MAE is the larger ViT-large RGB baseline that MAPEX is compared against on classification tasks.","marker":"[37]"},{"why":"SEN12-FLOOD provides the Sentinel-1 SAR data used for the flood detection evaluation.","marker":"[36]"},{"why":"California Fire Perimeters is used to construct the wildfire detection dataset with Sentinel-2 SWIR imagery.","marker":"[6]"},{"why":"Provides the masked image modeling objective and reconstruction setup used for self-supervised pre-training.","marker":"[19]"},{"why":"Defines the mixture-of-experts layer formulation and router that MAPEX adapts for modality-conditioned routing.","marker":"[41]"},{"why":"The ViT base architecture is the backbone that MAPEX builds on.","marker":"[11]"}],"fun_headline_variants":["Two pruned experts per modality beat full models","Modality-aware pruning turns one model into many specialists","Pruned per-modality experts top larger baselines","Prune experts by modality to outdo big baselines","Specialize and shrink: per-modality experts win"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the 8,000 ben-ge samples used for land-cover evaluation were not part of the ben-ge data used for pre-training; the paper describes ben-ge-8k as a subset of ben-ge but never states that it was held out.","fun_headline_variants_meta":{"raw":{"variants":["Two pruned experts per modality beat full models","Modality-aware pruning turns one model into many specialists","Pruned per-modality experts top larger baselines","Prune experts by modality to outdo big baselines","Specialize and shrink: per-modality experts win"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001441,"raw_usage":{"total_tokens":5829,"prompt_tokens":986,"completion_tokens":4843,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":602,"completion_tokens_details":{"reasoning_tokens":4781}},"tokens_in":602,"tokens_out":4843,"duration_ms":34853,"temperature":1.0,"reasoning_tokens":4781,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T18:38:59.354979+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Check whether any sample identifiers in the ben-ge-8k evaluation subset also appear in the ben-ge pre-training split; if they do, or if the split is not documented, re-run Tables 1, 5, 6, 7, and 10 on a held-out set and compare. A second check would be to train the same pipeline with token-based routing and show that pruned experts no longer concentrate on their intended modality, which would indicate the routing mechanism, not the benchmark, is responsible for specialization.","supporting_citations":[{"cited_title":"Ben-ge: Extending BigEarthNet with geographical and environmen- tal data","cited_arxiv_id":null,"evidence_quote":"Supplies the ben-ge dataset for multi-modal pre-training and the ben-ge-8k subset for land-cover evaluation."},{"cited_title":"Scale-MAE: A Scale- Aware Masked Autoencoder for Multiscale Geospatial Rep- resentation Learning","cited_arxiv_id":null,"evidence_quote":"Scale-MAE is the larger ViT-large RGB baseline that MAPEX is compared against on classification tasks."},{"cited_title":"Sen12-flood: a SAR and Multispectral Dataset for Flood Detection","cited_arxiv_id":null,"evidence_quote":"SEN12-FLOOD provides the Sentinel-1 SAR data used for the flood detection evaluation."},{"cited_title":"Cali- fornia Fire Perimeters (all)","cited_arxiv_id":null,"evidence_quote":"California Fire Perimeters is used to construct the wildfire detection dataset with Sentinel-2 SWIR imagery."},{"cited_title":"Masked Autoencoders are Scal- able Vision Learners","cited_arxiv_id":null,"evidence_quote":"Provides the masked image modeling objective and reconstruction setup used for self-supervised pre-training."}],"review_version":1}