{"id":"cab6ee14-36eb-4e3a-942e-906a22040a81","arxiv_id":"2508.18067","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"SegEarth-OV performs annotation-free open-vocabulary segmentation of remote-sensing images by upsampling CLIP features, removing global bias, and distilling optical knowledge into a SAR encoder.","lead":"Remote sensing images can now be segmented into user-defined categories without any pixel or text annotations, using a framework called SegEarth-OV that sharpens coarse CLIP features and subtracts global image bias. It transfers this ability to radar images by distilling optical CLIP knowledge into a SAR encoder, cutting the need for expensive radar-text datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SAR SOTA may be inflated by training AlignEarth on paired data from the same benchmark sources used for evaluation, making the headline SAR generalization claim unverified.","rationale":"The reader identified the fixed lambda and per-dataset prompt engineering as the weakest assumptions. Those are legitimate but not the most load-bearing: lambda is a single scalar whose sensitivity can be probed, and the prompt vocabulary is shared across all baselines, so it does not break the comparison. The SAR train/eval source overlap, by contrast, threatens the validity of the central SOTA claim for the SAR half of the paper. If AlignEarth was trained on the same datasets used for evaluation, the reported gains in Tables 4 and 9 may reflect test-set leakage rather than the claimed ability to enable open-vocabulary segmentation on unseen SAR data. This is directly supported by the manuscript's own dataset lists and citation reuse. The fix is concrete: retrain on a disjoint set and re-evaluate. Because this can be settled by a rerun, the paper should not be rejected outright, but it should be conditional on demonstrating that the SAR results survive the exclusion. The reader's verdict remains CONDITIONAL, but the condition is now more specific and more severe than the original rationale.","tokens_in":27070,"tokens_out":6237,"duration_ms":80306,"concrete_test":"Retrain AlignEarth after excluding every paired source that overlaps an evaluation benchmark: drop FUSAR-Map, WHU-OPT-SAR, DDHR-SK, PIE-RGB-SAR, YESeg-OPT-SAR, and DFC2025 Track1 from the training pool, keeping only SpaceNet6, MSAW, QXS-SAROPT, SAR2Opt, and DFC2023 Track1, then re-run Table 4 with the identical protocol. Also run an image-hash/near-duplicate check between the remaining training pairs and the evaluation SAR images. If average mIoU stays near 34.0 and the DDHR-Korea/PIE-SAR margins persist, the leakage concern is resolved; if performance drops substantially, the current Table 4 overstates generalization.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The most load-bearing weakness is not the fixed lambda in Eq. (9) but an apparent train/evaluation overlap in the SAR experiments. Section 5.1 states that AlignEarth is trained on 65k optical-SAR pairs from SpaceNet6, MSAW, QXS-SAROPT, SAR2Opt, DFC2023 Track1, PIE-RGB-SAR, DFC2025 Track1, WHU-OPT-SAR, DDHR-SK, FUSAR-Map, and YESeg-OPT-SAR. The eight SAR evaluation datasets are PIE-SAR [95], YESeg-SAR [96], FUSAR-Map [97], DDHR-Korea/SD/XA [98], WHU-SAR [99], and OpenEarthMap-SAR [100]. The same references are used for PIE-RGB-SAR/PIE-SAR, YESeg-OPT-SAR/YESeg-SAR, FUSAR-Map, DDHR-SK/DDHR-Korea, WHU-OPT-SAR/WHU-SAR, and DFC2025 Track1/OpenEarthMap-SAR. Thus the AlignEarth SAR encoder is distilled on the same datasets subsequently used for evaluation. Table 4's large gains (e.g., 48.6 vs 16.5 on DDHR-Korea, 51.1 vs 12.1 on PIE-SAR) may reflect transductive adaptation to the test distributions, or even near-duplicate images, rather than the claimed general cross-modal transfer. The baselines are not given this adaptation, so the comparison is not apples-to-apples. The optical experiments do not share this problem because SimFeatUp is trained on Million-AID independently of the evaluation benchmarks, and Global Bias Alleviation requires no training. Therefore the central 'SOTA on SAR' claim is vulnerable if the overlap is real. A fixed lambda=0.3 is a hyperparameter that ablations show to be consistent, and prompt engineering is applied to all baselines, so those are weaker concerns.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"SegEarth-OV is presented as the first annotation-free open-vocabulary segmentation framework for remote sensing. For optical images, it proposes SimFeatUp, a FeatUp-derived upsampler trained once on 16k images from Million-AID and equipped with an image-reconstruction content-retention loss, earlier-stage feature input, a single shared JBU module, and an 11x11 kernel, plus a Global Bias Alleviation operation that subtracts a scaled [CLS] token from patch tokens. For SAR, AlignEarth distills an optical CLIP encoder into a SAR encoder using paired optical-SAR images and a combination of global contrastive, global distillation, and local region-level distillation losses. The paper reports state-of-the-art mIoU on 8 optical semantic-segmentation datasets, 9 single-class extraction datasets, and 8 SAR datasets, with component ablations and plug-and-play experiments on natural images.","tokens_in":27604,"tokens_out":8839,"duration_ms":101397,"significance":"If the empirical claims hold, this is a practically useful contribution: the optical results are broad (17 datasets), SimFeatUp is evaluated as a plug-and-play module under several baselines, and SimFeatUp's training is independent of the optical evaluation benchmarks. The natural-image results in Table 8 also suggest some generality beyond remote sensing, and the authors state that code and models will be released, which would support reproducibility. However, the SAR results—one of the two headline advances—are currently not trustworthy because of an apparent overlap between AlignEarth's training pairs and the SAR evaluation benchmarks. The fixed lambda in Eq. (9) and the lack of variance reporting are secondary but require attention.","major_comments":[{"comment":"AlignEarth's training pairs are taken from SpaceNet6, MSAW, QXS-SAROPT, SAR2Opt, DFC2023 Track1, PIE-RGB-SAR, DFC2025 Track1, WHU-OPT-SAR, DDHR-SK, FUSAR-Map, and YESeg-OPT-SAR. The SAR test sets listed in §5.1 are PIE-SAR [95], YESeg-SAR [96], FUSAR-Map [97], DDHR-Korea/SD/XA [98], WHU-SAR [99], and OpenEarthMap-SAR [100]. By the cited references, PIE-RGB-SAR is [95], YESeg-OPT-SAR is [96], FUSAR-Map is [97], DDHR-SK is [98], WHU-OPT-SAR is [99], and DFC2025 Track1 is [100]. Thus the SAR encoder is distilled on the same benchmarks that are later used for evaluation in Table 4. No train/test split or exclusion of evaluation images is reported. This means the large gains in Table 4 (e.g., 51.1 vs 12.1 on PIE-SAR) may reflect transductive adaptation to the test distributions, and the comparison to baselines that were not given this adaptation is not apples-to-apples. The authors need to ei","section":"§5.1, Table 4"},{"comment":"Global Bias Alleviation subtracts a fixed fraction lambda=0.3 of the [CLS] token from every patch token. The implementation sets lambda=0.3 for all datasets, but no sensitivity analysis is reported; Table 7 only toggles the operation on/off. Since the bias magnitude may depend on scene content, sensor, or category, the optimum of lambda is not guaranteed to be stable, and the claimed universality of GBA across 25 datasets is not established. Please report a lambda sweep on at least one optical dataset and one SAR dataset, and similarly justify gamma, K, and the JBU kernel size.","section":"§3.3, Eq. (9)"},{"comment":"All quantitative claims are single-run numbers with no variance. Several component contributions in Table 7 are small (0.3–1.7 mIoU), and the 'JBU One' and 'Large Kernel' rows are within likely run-to-run noise for a training-based upsampler. Without multiple seeds or a significance test, the incremental ablation story is not fully supported. This also applies to the modest gains in Table 8. I request variance bars or seed information for the main tables, at least for Tables 4 and 7.","section":"§5.4, Table 7"}],"minor_comments":[{"comment":"The phrase 'slight class rename tricks' understates the per-dataset prompt engineering, which includes multiple synonyms and unions. Although the same vocabularies are used for all baselines, the paper should state how these vocabularies were chosen and whether choices were fixed before seeing test labels, to avoid the impression of test-set-specific tuning.","section":"§5.2/Table 1"},{"comment":"Typo: 'a annotation-free OVSS method' should be 'an annotation-free OVSS method'.","section":"Conclusion"},{"comment":"The sentence 'only < 0.3M parameters' is unclear; give the actual parameter counts for FeatUp and SimFeatUp.","section":"§3.2.4"},{"comment":"The relationship between DFC2025 Track1 and OpenEarthMap-SAR should be explicit; currently both names are used (also in Table 4), making the overlap analysis difficult for the reader.","section":"§5.1"},{"comment":"The 896×896 row is missing for WHUSat.II; explain why or provide the number.","section":"Table 3"},{"comment":"The 'Oracle' is described in one sentence as a fully supervised SegFormer with full training data; provide the training protocol and dataset splits to make the upper-bound comparison interpretable.","section":"§5.3.1"}],"recommendation":"major_revision","confidential_remarks":"To the editor: I am not requesting rejection because the optical part is broad and appears sound. The SAR overlap is fixable in revision by reporting disjoint splits or by dropping overlapping test sets. I would also verify the relationship to the authors' CVPR 2025 SegEarth-OV [11]; the manuscript should clearly delineate the new material (AlignEarth, additional optical ablations) from the prior conference publication. The paper would benefit from an explicit statement that no evaluation image or its optical-SAR sibling was used during AlignEarth or SimFeatUp training."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take on arXiv:2508.18067. You should know two things up front: the optical part is a competent engineering follow-up to the authors' CVPR 2025 SegEarth-OV, and the SAR part, which is the genuinely new AlignEarth contribution, has a likely train/evaluation overlap that casts serious doubt on the headline SOTA claim.\n\nWhat is actually new: AlignEarth, a distillation strategy that transfers semantic knowledge from a frozen optical CLIP encoder to a trainable SAR encoder using paired optical-SAR images. That is a real idea and could be useful for modalities where no VLM exists. The optical components—SimFeatUp and Global Bias Alleviation—are modifications of existing FeatUp and the authors' earlier work, but they are applied carefully to remote sensing, and the improvements are consistent across 17 optical datasets. The ablation study in Table 7 is thorough and justifies each design choice, including the image reconstruction loss, the earlier-stage feature upsampling, and the larger kernel. The paper also evaluates against a reasonable set of baselines, both natural-image OVSS methods and remote-sensing CLIPs.\n\nNow the soft spots. The big one is the SAR evaluation. AlignEarth is trained on paired data from SpaceNet6, MSAW, QXS-SAROPT, SAR2Opt, DFC2023, PIE-RGB-SAR, DFC2025, WHU-OPT-SAR, DDHR-SK, FUSAR-Map, and YESeg-OPT-SAR. The evaluation uses PIE-SAR, YESeg-SAR, FUSAR-Map, DDHR-Korea/SD/XA, WHU-SAR, and OpenEarthMap-SAR. These share the same reference numbers—often the same exact dataset. The paper does not state that the training pairs are excluded from the evaluation sets or that a split is enforced. So Table 4's dramatic gains (e.g., 48.6 vs 16.5 on DDHR-Korea, 51.1 vs 12.1 on PIE-SAR) may largely reflect transductive adaptation to the test distributions, or even near-duplicate images. The baselines are not given this adaptation, so the comparison is not apples-to-apples. This is a load-bearing flaw, not a minor caveat.\n\nOther issues are lesser. The paper reports single-run numbers with no variance. It does not ship code or weights. The fixed lambda=0.3 for global bias subtraction is hand-tuned, but the ablation shows it to be consistently helpful, so that is a minor concern. Per-dataset prompt engineering is applied to all baselines, so it is not an unfair advantage in the comparisons.\n\nWho is this for? Researchers working on open-vocabulary segmentation for remote sensing, particularly SAR. The optical part is practically useful and the ablation is worth reading. But the SAR claim needs to be re-evaluated. I would send it to peer review, with the explicit instruction that the authors must clarify the SAR train/test separation and provide evidence of generalization to held-out scenes or datasets. If they cannot, the SAR SOTA claims should be substantially downgraded.","headline":"The optical part is a solid, incremental extension of the authors' own SegEarth-OV; the SAR part, AlignEarth, is the genuinely new contribution but its headline SOTA numbers are undermined by an apparent train/evaluation overlap that must be resolved.","tokens_in":28052,"tokens_out":3067,"would_cite":false,"duration_ms":34095,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SegEarth-OV claims the first annotation-free open-vocabulary segmentation pipeline for remote sensing, using a trained upsampler and a global-bias subtraction to lift mIoU across 17 optical and 8 SAR benchmarks.","keywords":["Semantic segmentation","Open-vocabulary","Remote sensing image","Vision-language model","Annotation-free","Feature upsampling","Global bias alleviation","SAR"],"falsifier":"Measure, per image or per region, the cosine similarity between local patch tokens and the [CLS] token across a scene-diverse remote sensing benchmark, then vary the subtraction strength λ from 0 to 1; if the optimal λ scatters widely rather than clustering near 0.3, the constant-bias assumption is not the operative cause of the reported gains.","tokens_in":27020,"feed_emoji":"🛰️","tokens_out":8181,"duration_ms":87677,"temperature":0.7,"pith_summary":"Open-vocabulary image segmentation—labeling every pixel with classes named on the fly—has worked mainly on natural images, where CLIP's coarse features still carry enough spatial detail. This paper argues that remote sensing imagery breaks that assumption: targets vary over orders of magnitude in scale, fine boundaries matter, and a global image token leaks into every local patch token. To fix this, SegEarth-OV trains a universal upsampler (SimFeatUp) once on unlabeled remote sensing images to restore high-resolution spatial detail, and subtracts the global token from each patch token to sharpen local semantics. For SAR imagery, where no large text-paired model exists, AlignEarth distills optical CLIP semantics into a SAR encoder using paired optical-SAR images, avoiding SAR captions entirely. The paper reports large mIoU gains over prior annotation-free methods on 17 optical and 8 SAR benchmarks, bringing open-vocabulary labeling into reach for Earth observation without dense annotations.","feed_headline":"Satellite images get annotation-free open-vocabulary segmentation","feed_subtitle":"One upsampler plus a fixed global-bias subtraction beats prior methods on optical and SAR remote sensing benchmarks.","key_machinery":"The load-bearing identity is O_hat = O[1:hw+1] − λ O[0]: subtract a fraction (λ=0.3) of the replicated global [CLS] token from every local patch token to remove the global bias that CLIP injects into local features. Around this sits SimFeatUp, a universal upsampler built from a single parameterized joint bilateral upsampling (JBU) module executed repeatedly, guided by spatial distance and feature similarity in the high-resolution RGB image, and regularized by a content retention network that reconstructs the input image from the upsampled features. The third mechanism, AlignEarth, transfers the CLIP semantic space to SAR: a trainable SAR encoder is aligned to a frozen optical encoder by glob","core_discovery":"The paper's central claim is that annotation-free open-vocabulary segmentation of remote sensing images is achievable without task-specific post-training or pixel labels. Three mechanisms carry it. SimFeatUp takes features from an early CLIP block, upsamples them with a single shared parameterized joint bilateral upsampling module run repeatedly, and a content retention network forces the result to reconstruct the input image, preventing semantic drift; one training pass on unlabeled remote sensing images makes the weights universal. Global Bias Alleviation subtracts a scaled copy of the replicated [CLS] token from every patch token (Eq. 9), removing global context that falsely activates unr","pith_inferences":["Inference: The fixed subtraction strength of 0.3 implies a single global-correction constant works across 25 datasets; a per-image or per-region adaptively estimated strength is a natural next test and might push gains further.","Inference: Because AlignEarth uses only paired optical-SAR images, unpaired SAR corpora and synthetic SAR could be added to improve the distilled encoder without any text annotation.","Inference: If the framework generalizes as claimed, pixel-level labeling of novel categories in disaster response or land-cover monitoring could be done on demand from text prompts, with no training labels.","Inference: The natural-image gains suggest SimFeatUp may serve as a general feature refiner for any frozen vision-language model, not only remote sensing encoders."],"forward_implications":["With SimFeatUp and Global Bias Alleviation attached, even the strongest annotation-free OVSS methods gain more: on OpenEarthMap the improvement grows from +3.3 for MaskCLIP to +8.1 for ClearCLIP (Table 6).","The upsampler transfers across modalities without retraining: the same SimFeatUp weights process features from the AlignEarth SAR encoder and from optical CLIP (Table 4).","AlignEarth removes the need for SAR-specific image-text pairs; paired optical-SAR images alone bring every tested OVSS baseline to a much higher mIoU on SAR data.","The full framework reaches 60.2% IoU on flood detection (WBS-SI), suggesting annotation-free OVSS is viable for disaster-relevant water mapping.","SimFeatUp also improves MaskCLIP, SCLIP, and ClearCLIP on natural image benchmarks (Table 8), indicating the upsampler is not remote-sensing-specific."],"supporting_citations":[{"why":"supplies the pre-trained vision-language model whose patch features are upsampled and bias-corrected","marker":"[6]"},{"why":"provides the model-agnostic upsampling paradigm (parameterized JBU plus multi-view consistency) that SimFeatUp adapts","marker":"[39]"},{"why":"established annotation-free MaskCLIP dense-label extraction from CLIP and the attention modification used by the pipeline","marker":"[58]"},{"why":"baseline showing self-self attention improves dense vision-language inference; SegEarth-OV compares against it","marker":"[9]"},{"why":"strongest prior baseline (ClearCLIP) that SegEarth-OV outperforms on the optical benchmarks","marker":"[61]"},{"why":"raw remote sensing image dataset used to train SimFeatUp without any labels","marker":"[101]"},{"why":"paired optical-SAR imagery source used to train the AlignEarth SAR encoder","marker":"[73]"},{"why":"earlier conference version of the SegEarth-OV framework that this paper builds on and extends to SAR","marker":"[11]"}],"fun_headline_variants":["SegEarth-OV: Pixel-free open-vocab mapping for Earth observation","No labels needed: open-vocabulary segmentation for satellite imagery","One upsampler unlocks zero-label segmentation across sensors","Open-vocabulary segmentation without annotations for RS images","SimFeatUp plus bias subtraction enables annotation-free RS segmentation"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The paper assumes that the global context contaminating each local patch feature is a constant fraction of the image's global token, so subtracting the same fixed amount works for every image, dataset, and sensor.","fun_headline_variants_meta":{"raw":{"variants":["SegEarth-OV: Pixel-free open-vocab mapping for Earth observation","No labels needed: open-vocabulary segmentation for satellite imagery","One upsampler unlocks zero-label segmentation across sensors","Open-vocabulary segmentation without annotations for RS images","SimFeatUp plus bias subtraction enables annotation-free RS segmentation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000489,"raw_usage":{"total_tokens":2285,"prompt_tokens":824,"completion_tokens":1461,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":568,"completion_tokens_details":{"reasoning_tokens":1376}},"tokens_in":568,"tokens_out":1461,"duration_ms":12543,"temperature":1.0,"reasoning_tokens":1376,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T16:35:20.296020+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure, per image or per region, the cosine similarity between local patch tokens and the [CLS] token across a scene-diverse remote sensing benchmark, then vary the subtraction strength λ from 0 to 1; if the optimal λ scatters widely rather than clustering near 0.3, the constant-bias assumption is not the operative cause of the reported gains.","supporting_citations":[{"cited_title":"Extract free dense labels from clip,","cited_arxiv_id":null,"evidence_quote":"established annotation-free MaskCLIP dense-label extraction from CLIP and the attention modification used by the pipeline"},{"cited_title":"On creating benchmark dataset for aerial image interpretation: Reviews, guidances and million-aid,","cited_arxiv_id":null,"evidence_quote":"raw remote sensing image dataset used to train SimFeatUp without any labels"},{"cited_title":"Spacenet 6: Multi-sensor all weather mapping dataset,","cited_arxiv_id":null,"evidence_quote":"paired optical-SAR imagery source used to train the AlignEarth SAR encoder"}],"review_version":1}