{"id":"3c94b75c-45bb-46af-8e8b-cf758e0f4608","arxiv_id":"2507.13812","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A unified multi-modal remote sensing foundation model with adaptive patch merging, modality prompt tokens, mixture of experts, and query-based semantic aggregation contrastive learning outperforms SkySense by 1.8 points on average.","lead":"SkySense V2 is a remote sensing AI model that uses one shared transformer backbone for optical, multispectral, and radar images instead of three separate networks. It reports consistent gains over its predecessor on 16 benchmark datasets while cutting backbone parameters by about half.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Efficiency claim compares the pre-MoE backbone (665M) against SkySense, but the evaluated model includes MoE (1.99B total backbone parameters); the reported 1.8-point gain is not disentangled from MoE scaling.","rationale":"In good faith, the paper is a substantial empirical effort: 16 datasets, 7 tasks, and consistent gains over SkySense, with ablations for each component. The reader's concern about pixel-level geospatial alignment and OSM label noise is legitimate, but it is not the most load-bearing issue because the downstream benchmarks are independent of the pre-training corpus, and the reported gains could not easily arise solely from data-alignment artifacts in pre-training. The more direct threat to the central claim is the parameter-count framing. The paper motivates the unified backbone as eliminating redundancy ('Skysense V2's unified transformer backbone... utilizing only 665 million parameters'), but the deployed model uses MoE (1,994M backbone) and the ablations show MoE contributes ~0.4-0.9 points on the three datasets tested. Since the paper never reports SkySense V2 without MoE on the full 16-benchmark suite, the 1.8-point average gain versus SkySense could be substantially due to the 1.33B extra parameters rather than the unified architecture or QSACL. This is a concrete, checkable confound, not a matter of taste. The verdict should remain conditional, but the conditions should include reporting the no-MoE comparison across all benchmarks and explicitly stating both total and activated parameter counts in the abstract and intro. My concern does not move the verdict because the empirical results could still stand if the no-MoE model is competitive; it changes the framing required for acceptance.","tokens_in":27997,"tokens_out":7125,"duration_ms":78474,"concrete_test":"Re-run the 16 benchmark protocols from Section 4 with the SkySense V2 configuration without MoE (Table 9, 661M backbone), keeping all other settings identical. If the average gain over SkySense is below about 0.5 points or not consistently positive, the parameter-efficiency and unified-design attribution collapses, and the 1.8-point result is attributable to MoE capacity.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central efficiency claim is not supported by the reported experiments. Section 1 states that SkySense V2's unified backbone uses 'only 665 million parameters' versus SkySense's 1.26B three-backbone design, motivating the work as reducing redundancy. However, Section 3.1.3 and Appendix E show that the model actually evaluated in Section 4 includes MoE modules: the backbone has 1,994.10M total parameters (661.40M activated) in the final configuration. All 16-dataset results in Section 4 are for this MoE model; no results are reported for the 665M no-MoE version. The ablation in Appendix F.6 (Table 17) shows that removing MoE drops performance by 0.81 points on RESISC-45, 0.37 on BEN-S2, and 0.86 on BEN-MM (multi-modal). Thus the MoE modules, which add 1.33B parameters relative to SkySense's 1.26B, contribute materially to the reported gains. The headline 'outperforming SkySense by 1.8 points with a 665M backbone' conflates the pre-MoE backbone with the deployed model. Unless the no-MoE model is shown to beat SkySense on the full benchmark suite, the improvement may be attributable to extra MoE capacity rather than to the unified backbone or QSACL.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"SkySense V2 presents a unified multi-modal remote sensing foundation model that replaces SkySense's three separate backbones (Swin-H for HR optical, ViT-L for MS, ViT-L for SAR) with a single shared transformer backbone. The architecture introduces Adaptive Patch Merging (APM) to handle varying ground resolutions across modalities, learnable modality-specific prompt tokens to preserve feature diversity, and a Mixture-of-Experts (MoE) module to scale capacity. Pre-training combines the inherited MGCL and GCPL losses from SkySense with a new Query-based Semantic Aggregation Contrastive Learning (QSACL) loss and dense image-text alignment using OSM labels. The model is evaluated on 16 datasets across 7 tasks, reporting an average improvement of 1.8 points over SkySense and state-of-the-art results on most benchmarks, with additional out-of-sensor generalization experiments in Appendix F.3.","tokens_in":28361,"tokens_out":5398,"duration_ms":56583,"significance":"If the reported results hold, SkySense V2 is a meaningful advance: it demonstrates that a single shared backbone with roughly half of SkySense's activated parameters (661M activated vs 1.26B total for SkySense) can match or exceed its predecessor across diverse single- and multi-modal tasks. The out-of-sensor generalization results (Appendix F.3) and the extensive ablations (MoE, prompts, APM, QSACL queries) are valuable evidence beyond simple benchmark improvements. The paper also provides k-NN convergence curves and t-SNE visualizations. However, the headline efficiency claim is confounded by MoE, the 'state-of-the-art' claim omits relevant unified-backbone baselines mentioned in the paper, and the lack of error bars or significance tests tempers confidence in the small margins. These issues are addressable but require additional experiments or careful reframing.","major_comments":[{"comment":"The efficiency claim conflates the pre-MoE unified backbone with the model actually evaluated. Section 1 states that SkySense V2's unified backbone uses 'only 665 million parameters' versus SkySense's 1.26B three-backbone design, but Section 3.1.3 and Appendix E show that the evaluated model has 1,994.10M total backbone parameters (661.40M activated) because of MoE, and all results in Tables 1-3 are for this MoE model. Appendix F.6 (Table 17) shows that removing MoE decreases accuracy by 0.81 on RESISC-45 (TR=10%), 0.37 on BEN-S2, and 0.86 on BEN-MM, so MoE contributes materially to the reported gains. To support the 'unified backbone reduces parameters' narrative, the paper must either report the full 16-dataset results for the 665M no-MoE model (at least the key comparisons against SkySense), or clearly state that the 1.8-point average gain belongs to the MoE-scaled model with 1.99B total backbone parameters and adjust the efficiency framing accordingly.","section":"Section 1, Section 3.1.3, Appendix E, Appendix F.6"},{"comment":"The paper claims 'state-of-the-art results across various modalities' (Section 4) but does not compare against any of the unified-backbone multi-modal RSFMs it discusses in Section 2.2: AnySat, msGFM, and OFA-Net. These are the most relevant competitors for the central architectural claim. At minimum, the authors should include AnySat and msGFM results on the shared benchmarks (e.g., PASTIS, BEN-MM, fMoW-S2) where public numbers exist, or explicitly justify their exclusion. Without such comparisons, the 'state-of-the-art' claim is overstated and the reader cannot assess whether the unified backbone itself is competitive with existing unified designs.","section":"Section 2.2, Section 4"},{"comment":"All baseline numbers in the comparison tables are taken from prior publications, and the paper reports no error bars, multiple seeds, or significance tests. Several of the reported gains over SkySense are small (e.g., Table 3c: BEN-MM S1-only 86.2 to 86.5; Table 1: BEN-S2 TR=10% 88.67 to 89.13; Table 2a: Dyna.-S2 46.2 to 47.5 for the S2-only row). Since the central claim is an average +1.8-point improvement, the authors should provide variance estimates or at least run the SkySense baseline under identical fine-tuning protocols for a few key datasets to rule out that the gains are within noise or due to tuning differences.","section":"Tables 1-3"},{"comment":"The primary novel pre-training loss, QSACL, is only validated with k-NN accuracy after 20,000 pre-training iterations (Figure 7 and Appendix F.7, Table 18), not with full pre-training followed by downstream fine-tuning. Since QSACL is one of the three loss terms and is central to the paper's claimed novelty, the authors should either report downstream fine-tuning results for QSACL on at least one or two representative tasks (e.g., RESISC-45 and BEN-S2) with the full 600k-iteration pre-training, or clearly acknowledge this as an ablation performed only at reduced scale. This is important because the k-NN proxy may not reflect final transfer performance.","section":"Section 3.2, Figure 7, Appendix F.7"}],"minor_comments":[{"comment":"There are several typos and minor language issues: 'In additional' (Abstract), 'a board range of tasks' (Figure 1 caption), 'RadnomRotate' (Table 5), and 'varing' in the captions of Tables 14, 15, and 18.","section":"Throughout"},{"comment":"The paper states that the OSM labels are used for dense image-text alignment but does not describe any filtering or quality control for OSM label noise. Given that OSM is crowd-sourced, a brief statement on label cleaning or its known limitations would strengthen the reproducibility of the pre-training pipeline.","section":"Section 4.1, Appendix A.2"},{"comment":"The APM mechanism for MS and SAR tokens is described qualitatively as 'applying a linear projection with averaging the weights across the input dimensions.' A concrete formula or diagram would clarify how the shared linear layer is adapted to preserve resolution without training separate weights.","section":"Section 3.1.1, Figure 3"},{"comment":"References [59] and [60] appear to be the same paper ('Mixture-of-experts meets instruction tuning') with conflicting venue and year information; please verify and consolidate.","section":"References"},{"comment":"The parameter comparison table lists an 'Others' category (404.13M for SkySense, 490.49M for SkySense V2) but does not define what it contains. Please specify the components in this category.","section":"Appendix E, Table 9"}],"recommendation":"major_revision","confidential_remarks":"The author group overlaps heavily with SkySense, which may give the new method an advantage in fine-tuning protocol alignment; this makes the absence of re-run baselines for SkySense under identical settings more consequential. The paper is a preprint with no code or model release, which is not unusual but limits reproducibility. The core idea is promising, but the MoE confound in the efficiency claim and the missing comparisons with AnySat/msGFM/OFA-Net are significant and should be resolved before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Thanks for the report. The stress-test note is on target. The paper's central efficiency claim — 665M parameters versus SkySense's 1.26B — describes the backbone before MoE, but every number in Section 4 comes from the MoE model, whose backbone totals 1.99B parameters (661M activated). Table 17 in F.6 shows MoE buys 0.8 points on RESISC-45 and 0.86 on BEN-MM, so the reported gains are partly MoE scaling, not purely the unified backbone or QSACL. The paper never reports no-MoE results on the full benchmark, which makes the contribution of the unified design genuinely hard to disentangle. That said, the stress-test doesn't kill the paper; it just means the efficiency story is oversold. The unified backbone still makes sense for multi-modal activated-parameter efficiency, and the ablations suggest QSACL and the prompts help.\n\nWhat's genuinely new: QSACL is a real, sensible idea. Learnable queries aggregating consistent semantics across augmented views directly addresses the 'different crops show different objects' problem in RS SSL, and the visualizations in Figure 9 give some evidence that different queries specialize. APM is a practical answer to variable GSD in a shared backbone. The evaluation is broad — 16 datasets, 7 tasks — and the out-of-sensor generalization results in F.3 (Gaofen-2, Landsat-8, Gaofen-3) are a nice addition that most RSFM papers skip. The paper is transparent about training cost and hyperparameters, and the ablation coverage is decent.\n\nOn the soft side, besides the MoE conflation: the paper cites AnySat, msGFM, and OFA-Net as unified multi-modal RS baselines but never compares against them. For a paper whose pitch is 'unified backbone,' skipping the closest competitors is a real omission. There are no error bars or significance tests, and baseline numbers are all from prior papers, so small gaps (e.g., 0.3 on some datasets) are within noise. No code or checkpoints are released. Also, the 'average 1.8 points' mixes metrics across tasks (mIoU, mAP, F1, SCS) without weighting, which is a bit loose.\n\nIn sum: this is an incremental but solid follow-up to SkySense. The QSACL idea and the evaluation breadth give it real value for anyone working on multi-modal RS foundation models. The MoE issue doesn't invalidate the results, but it changes the interpretation. A serious referee should push for no-MoE full-benchmark numbers, unified-baseline comparisons, and at least a few repeated runs. I'd send it to review rather than desk-reject. I'd probably not cite it personally until the artifacts and comparisons land.","headline":"The 665M-parameter efficiency claim is misleading because all benchmark results come from the 1.99B MoE model, yet the paper is still a solid incremental RSFM contribution worth reviewing.","tokens_in":28900,"tokens_out":4957,"would_cite":false,"duration_ms":48718,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T45","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"A single shared transformer backbone learns optical, multispectral, and SAR imagery together, beating SkySense by 1.8 points on average across 16 datasets.","keywords":["remote sensing foundation model","multi-modal learning","self-supervised learning","contrastive learning","mixture of experts","unified backbone","adaptive patch merging","modality prompt tokens"],"falsifier":"Re-run the same pre-training and downstream fine-tuning protocol on a publicly released, independently validated subset of the 21-million-set corpus with manually checked geo-alignment, and compare SkySense V2 against SkySense; if the average 1.8-point advantage disappears or shrinks materially on that curated subset, the gains are attributable to data artifacts rather than the unified backbone, APM, prompts, MoE, or QSACL.","tokens_in":27804,"feed_emoji":"🛰️","tokens_out":1531,"duration_ms":20794,"temperature":0.7,"pith_summary":"SkySense V2 claims that a single unified transformer backbone, with all parameters shared across high-resolution optical, Sentinel-2 multispectral, and Sentinel-1 SAR imagery, can outperform SkySense's three separate backbones on 16 datasets across 7 Earth-observation tasks. The paper argues that this unified design eliminates parameter redundancy, speeds up representation learning by aggregating gradients from multiple modalities, and leaves room to scale capacity with a mixture-of-experts module. To adapt self-supervised learning to remote sensing, it introduces Query-based Semantic Aggregation Contrastive Learning (QSACL), where learnable queries aggregate features of the same semantic content across different augmented views before contrastive loss is applied. If the claims hold, a single pre-trained encoder could replace modality-specific backbones, simplifying multi-modal Earth observation models and improving their generalization to unseen sensors.","feed_headline":"One shared backbone beats three separate models on 16 remote-sensing datasets","feed_subtitle":"SkySense V2 unifies optical, multispectral, and SAR processing with fewer parameters and gains 1.8 points on average.","key_machinery":"The central mechanism is the unified transformer encoder with four stages: Swin Transformer V2 blocks in the first two stages and vanilla transformer blocks in the last two, all parameters shared across modalities. Adaptive Patch Merging (APM) selectively downsamples feature resolution for high-resolution optical data while preserving resolution for medium-resolution multispectral and SAR data, maintaining spatial alignment. Learnable modality-specific prompt tokens are inserted in the last two stages to recover feature diversity lost by full parameter sharing. The mixture-of-experts (MoE) module replaces the feed-forward networks in the last six blocks, scaling capacity to 1,994M total (661M activated) parameters with sparse gating (8 experts, top-1 routing). QSACL uses 16 learnable queries in a transformer decoder layer to attend to features from global and local augmented views, producing semantically aggregated features per query that are contrasted across student and teacher branches.","core_discovery":"SkySense V2 establishes that a unified backbone with fully shared parameters across modalities, supplemented by modality-specific prompt tokens and a mixture-of-experts module, yields stronger representations than SkySense's separate Swin-H and two ViT-L backbones. The paper reports that this 665M-parameter (activated) unified backbone outperforms SkySense's 1.26B-parameter three-backbone design by an average of 1.8 points across 16 datasets, including segmentation, detection, classification, change detection, and multi-modal fusion tasks. The key discovery is the combination of Adaptive Patch Merging (APM) to handle varying ground sample distances, learnable modality prompts to preserve feature diversity, and QSACL to make contrastive learning semantically accurate for remote sensing images, which contain multiple subjects per image rather than a single focus.","pith_inferences":["The paper's design suggests that a single shared backbone may also absorb additional modalities beyond optical, multispectral, and SAR, such as thermal or LiDAR, by adding corresponding tokenizers and prompt tokens, without redesigning the core encoder.","The QSACL mechanism of learning semantic queries could be combined with dense image-text alignment more explicitly, where each query corresponds to a semantic class, potentially producing a fully interpretable, class-conditioned representation.","The reported improvement on unseen sensors (Gaofen-2, Landsat-8, Gaofen-3) implies that unified pre-training confers cross-sensor transfer, which could be tested further on more diverse sensor families with a systematic calibration of the alignment quality.","An implicit testable extension is whether the same unified backbone, pre-trained with QSACL, benefits from increased numbers of queries beyond 16 on datasets with more semantic classes, or whether query count should scale with class diversity."],"forward_implications":["If the gains hold, a single pre-trained backbone can serve all three modalities, cutting backbone parameters from 1.26B to 661M and simplifying deployment for downstream Earth-observation tasks.","The unified design aggregates gradients from all modalities, which the paper argues accelerates pre-training convergence compared to separate backbones.","Adding modality-specific prompt tokens recovers feature diversity lost by full parameter sharing, suggesting that small per-modality parameter sets can substitute for entire separate backbones.","QSACL's query-based aggregation of semantics across views could generalize to other multi-subject image domains where standard contrastive learning suffers from semantic mismatch between augmented views.","The MoE-scaling approach indicates that parameter-efficient unified backbones can be scaled up to significantly larger total capacities while keeping activated parameters moderate, yielding further performance gains as expert count grows."],"supporting_citations":[{"why":"The predecessor SkySense, supplying the multi-modal training corpus, the MGCL and GCPL loss components, the multi-modal temporal fusion transformer, and the baseline performance the paper must beat.","marker":"[24]"},{"why":"DINOv2, the teacher-student EMA SSL framework that SkySense V2 adopts as its base pre-training architecture.","marker":"[53]"},{"why":"Mixture of Experts, the foundational technique used to scale up the unified backbone with sparse routing.","marker":"[33]"},{"why":"The Vision Transformer, providing the vanilla transformer blocks used in the final two stages of the unified encoder.","marker":"[18]"},{"why":"Swin Transformer V2 blocks, used in the first two stages to supply locality and translation invariance priors.","marker":"[42]"},{"why":"Vision transformers need registers, the inspiration for learnable tokens later adapted into modality-specific prompt tokens.","marker":"[16]"},{"why":"Visual prompt tuning, the other inspiration cited for the learnable modality prompt token design.","marker":"[35]"}],"fun_headline_variants":["One backbone beats three on multimodal remote sensing by 1.8 points","Unified multimodal backbone outperforms separate models with fewer parameters","Shared backbone gains 1.8 points across 16 remote-sensing datasets","Single transformer for all modalities beats three separate backbones","SkySense V2 unifies modalities with one backbone, gains 1.8 points"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The model is pre-trained on roughly 21 million geographically aligned multi-modal image sets inherited from SkySense, and the whole argument depends on those sets having correct pixel-level alignment across optical, Sentinel-2, and Sentinel-1 images and trustworthy OpenStreetMap labels; systematic misalignment or label noise could inflate the measured gains independently of the new architecture.","fun_headline_variants_meta":{"raw":{"variants":["One backbone beats three on multimodal remote sensing by 1.8 points","Unified multimodal backbone outperforms separate models with fewer parameters","Shared backbone gains 1.8 points across 16 remote-sensing datasets","Single transformer for all modalities beats three separate backbones","SkySense V2 unifies modalities with one backbone, gains 1.8 points"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000562,"raw_usage":{"total_tokens":2661,"prompt_tokens":933,"completion_tokens":1728,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":549,"completion_tokens_details":{"reasoning_tokens":1635}},"tokens_in":549,"tokens_out":1728,"duration_ms":14700,"temperature":1.0,"reasoning_tokens":1635,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:16:20.079646+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same pre-training and downstream fine-tuning protocol on a publicly released, independently validated subset of the 21-million-set corpus with manually checked geo-alignment, and compare SkySense V2 against SkySense; if the average 1.8-point advantage disappears or shrinks materially on that curated subset, the gains are attributable to data artifacts rather than the unified backbone, APM, prompts, MoE, or QSACL.","supporting_citations":[{"cited_title":"Skysense: A multi- modal remote sensing foundation model towards universal interpretation for earth observation imagery","cited_arxiv_id":null,"evidence_quote":"The predecessor SkySense, supplying the multi-modal training corpus, the MGCL and GCPL loss components, the multi-modal temporal fusion transformer, and the baseline performance the paper must beat."},{"cited_title":"Jacobs, Michael I","cited_arxiv_id":null,"evidence_quote":"Mixture of Experts, the foundational technique used to scale up the unified backbone with sparse routing."},{"cited_title":"Swin transformer v2: Scaling up capacity and resolution","cited_arxiv_id":null,"evidence_quote":"Swin Transformer V2 blocks, used in the first two stages to supply locality and translation invariance priors."},{"cited_title":"Vision transformers need registers","cited_arxiv_id":null,"evidence_quote":"Vision transformers need registers, the inspiration for learnable tokens later adapted into modality-specific prompt tokens."},{"cited_title":"Vi- sual prompt tuning","cited_arxiv_id":null,"evidence_quote":"Visual prompt tuning, the other inspiration cited for the learnable modality prompt token design."}],"review_version":1}