{"id":"88f8c459-0ca7-49b9-b6c4-0c8bfa04e3eb","arxiv_id":"2505.18022","paper_version":3,"verdict":"REJECT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"RemoteSAM unifies remote sensing classification, detection, segmentation, and grounding through a single referring expression segmentation model trained on 270K VLM-generated image-text-mask triplets.","lead":"RemoteSAM is a model that uses referring expression segmentation to handle many Earth observation tasks with one architecture, trained on a new 270K sample dataset built automatically with vision-language models. It reports strong results on remote sensing benchmarks with far fewer parameters than LLM-based foundation models, but several benchmark numbers may be inflated because the training set includes those benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Benchmark contamination: RemoteSAM-270K is built from the same RRSISD/RisBench/DOTA/iSAID annotations used for evaluation, and Table 14 contradicts Table 3 for the same architecture.","rationale":"I agree with the reader's verdict. The central claim is empirical SOTA, so the evaluation protocol is load-bearing. The paper's own data construction makes contamination likely: it integrates the exact benchmarks it later evaluates on, and no split is specified. The internal inconsistency between Table 14 and Table 3 is additional evidence that the evaluation protocol is either undisclosed or unstable. The lambda tuning on DIOR/UCM further contaminates the image-level results. Unseen datasets (Vaihingen, UDD5, DeepGlobe, SATIN) provide some independent support for generalization, but they do not support the headline SOTA claims on the named benchmarks. The correct resolution is to audit the released dataset for overlap and, if needed, re-evaluate. The reader's REJECT is appropriate; I see no reason to change it.","tokens_in":34916,"tokens_out":5866,"duration_ms":47197,"concrete_test":"Download the released RemoteSAM-270K and, for each source benchmark (RRSISD, RisBench, DOTA, DIOR, iSAID), compare the image IDs in the training triplets against the official test split used in Tables 3, 4, 6, and 7. If any test image appears in training, the SOTA claim is invalid; if the authors claim a held-out split, that split file must be released and used to rerun Table 3's RemoteSAM configuration, which should also resolve the Table 14 (76.21/64.79) vs Table 3 (80.04/71.75) discrepancy.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central SOTA claim rests on benchmark numbers that may not be held out. Section 3.1.1 says RemoteSAM-270K is built by 'directly integrat[ing] existing referring segmentation annotations from RefSegRS, RRSISD, and RisBench' and by converting detection boxes from DOTA and iSAID into masks, yet Tables 3, 4, 6, and 7 report on exactly these benchmarks (RRSISD, RisBench, iSAID, DOTA, DIOR) without disclosing any train/test split. If full benchmark annotations entered training, the reported 71.75/71.46 mIoU, 64.72 iSAID mIoU, and 94/95 multi-label accuracies are not valid held-out results. The paper's own Appendix C, Table 14, aggravates this: the chosen architecture (BERT+Swin-Base) is reported at 76.21 oIoU / 64.79 mIoU on RRSISD, while Table 3 gives RemoteSAM 80.04 oIoU / 71.75 mIoU for the same architecture and dataset; without a disclosed split or protocol difference, the headline numbers are internally inconsistent. Finally, Fig. 22 tunes lambda on DIOR (and UCM) and the same datasets appear in Table 6, i.e., hyperparameters are selected on the evaluation set. The paper must release per-image split provenance and reproduce the headline under one protocol.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RemoteSAM, a referring-expression-segmentation (RES) foundation model for Earth observation, together with a new large-scale dataset RemoteSAM-270K built by integrating existing remote sensing segmentation and detection annotations and by using VLM-generated expressions. The authors claim that a single 180M-parameter model, without task-specific heads, can unify pixel-level (referring and semantic segmentation), region-level (grounding, detection), and image-level (multi-label classification, counting, captioning) tasks, and they report state-of-the-art results on RRSISD, RisBench, iSAID, DIOR, DOTAv2, and Potsdam.","tokens_in":35139,"tokens_out":4483,"duration_ms":40836,"significance":"If the empirical claims were valid, the work would be significant: the RES-centered unification paradigm is conceptually clean, the dataset is substantially larger than prior referring segmentation resources, and the reported parameter efficiency (180M versus 7B VLMs) is attractive. The authors also ship models and data publicly, and they provide a hierarchical remote sensing vocabulary (RSVocab-1K) as a coverage-analysis tool. These are real strengths. However, the central empirical evidence is compromised by apparent train/test overlap between RemoteSAM-270K and the benchmarks used for the headline tables, and by hyperparameter selection on the same evaluation sets. The claimed SOTA results are therefore not currently supported.","major_comments":[{"comment":"The evaluation is not held-out with respect to the training corpus. Section 3.1.1 states that RemoteSAM-270K is built by directly integrating existing referring segmentation annotations from RefSegRS, RRSISD, and RisBench, and by converting detection/instance masks from DOTA, iSAID, DIOR, LoveDA, and Potsdam into mask triplets. Tables 3, 4, 6, and 7 then report performance on RRSISD, RisBench, iSAID, Potsdam, DIOR, and DOTAv2 without disclosing any train/test split. The headline numbers (e.g., 71.75 mIoU on RRSISD, 71.46 mIoU on RisBench, 64.72 mIoU on iSAID, 94%/95% multi-label accuracy on DIOR/DOTAv2) are indistinguishable from recognition of training images. The paper must provide per-image split provenance and re-run the evaluation under a protocol where no benchmark image appears in RemoteSAM-270K.","section":"Section 3.1.1 and Tables 3, 4, 6, 7"},{"comment":"There is an internal inconsistency in the reported performance of the same architecture on the same dataset. Table 14 states that BERT+Swin-Base achieves 76.21 oIoU / 64.79 mIoU on RRSISD, while Table 3 reports RemoteSAM, which uses BERT+Swin-Base per Section 5.1.1, at 80.04 oIoU / 71.75 mIoU on RRSISD. The manuscript does not explain the protocol difference (e.g., training on a subset versus the full RemoteSAM-270K, different epochs, or different evaluation splits). This discrepancy undermines the reproducibility of the main table and must be resolved before the results can be trusted.","section":"Appendix C, Table 14 vs Table 3"},{"comment":"Hyperparameters are tuned on evaluation data. The balance factor lambda in Eq. (3) is selected by experiments on DIOR and UCM shown in Fig. 22, and the resulting model is then evaluated on DIOR and DOTAv2 in Table 6. Likewise, the classification threshold tau_cls in Eq. (4) is set on the same benchmarks. This means the reported multi-label and image classification accuracies partly reflect thresholds fitted to the test set, inflating the measured performance. The authors should either fix these hyperparameters a priori or tune them on a separate validation split, and then report results on a truly disjoint test set.","section":"Section 4.3, Eq. (3), Fig. 22, Table 6"},{"comment":"A large discrepancy between two tables reporting the same task is unexplained. Table 16 reports that the prob-level multi-label classification strategy achieves 94.042% on DIOR and 75.752% on DOTAv2, whereas Table 6 reports RemoteSAM at 94% and 95% on the same datasets. If these numbers came from different protocols (e.g., different label sets, different image subsets, or different thresholds), the paper must say so; otherwise the inconsistency calls both tables into question.","section":"Appendix C, Table 16 vs Table 6"}],"minor_comments":[{"comment":"The vocabulary is named 'RSVocab-1K' in the text but appears as 'RSBocab-1K' in one instance; please fix the typo.","section":"Section 3.2"},{"comment":"The word 'Supplymentary' should be 'Supplementary'.","section":"Section 3.1.2"},{"comment":"The phrase 'lass-wise probability aggregation function' should be 'class-wise probability aggregation function'.","section":"Appendix C"},{"comment":"The right panel of Fig. 22 is labeled 'UCM' but the text does not state which task or metric it reports (e.g., multi-label accuracy or scene classification accuracy); please clarify the axis and the task.","section":"Fig. 22"},{"comment":"The evaluation setup lists DOTA, DIOR, iSAID, Potsdam, and RSVG but omits RRSISD and RisBench, which are used in Table 3 and the appendix; the dataset list should be complete.","section":"Section 5.1.2"},{"comment":"The column '#Attr/Sample' for RemoteSAM-270K is 3.17, but the paper also reports 16 attribute types; please define whether this is the average number of attributes per expression and clarify the counting method.","section":"Table 1"},{"comment":"The 'Pre-trained Data' column for RemoteSAM is 'RemoteSAM-270K', which includes iSAID and Potsdam images; since these are also the evaluation datasets in the same table, a footnote is needed to state that these results are not held-out.","section":"Table 4"}],"recommendation":"reject","confidential_remarks":"The stress-test concern about benchmark contamination is well-founded and is the decisive issue. The dataset is a potentially useful resource, and the task-unification idea is interesting, but the reported SOTA claims are invalid as presented because the evaluation benchmarks overlap with the training corpus and because hyperparameters are tuned on the test benchmarks. Re-running the entire experimental campaign with disjoint splits is a major undertaking, and the current manuscript cannot be accepted without it. If the authors can provide a clean split and consistent tables, a resubmission may be worth considering."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know about this paper is that the core idea—using referring expression segmentation as the unified interface for all vision tasks—is genuinely clever, but the headline numbers are not credible as reported. Section 3.1.1 says the training set is built by directly integrating RRSISD, RisBench, RefSegRS, DOTA, and iSAID annotations. Then Tables 3, 4, 6, and 7 report state-of-the-art results on those exact datasets with no disclosed split. That is a textbook train/test leak, and the paper does nothing to rule it out.\n\nWhat is actually new: the referring-segmentation-centered task unification paradigm, the 270K image-text-mask dataset, and the VLM-driven data engine. The parameter efficiency relative to 7B LLM-based models is appealing. The unseen-dataset results on Vaihingen, UDD5, DeepGlobe, and SATIN are a real bright spot and suggest the model does generalize, at least in part. That independent evidence should not be dismissed.\n\nThe soft spots are serious. First, the contamination: if full benchmark annotations entered training, the reported mIoU and accuracy numbers are inflated. Second, the internal inconsistency: Appendix C, Table 14 reports the same BERT+Swin-Base architecture at 76.21 oIoU on RRSISD, while Table 3 gives 80.04 oIoU. That gap needs an explanation—different split, different training data, or an error. Third, Fig. 22 tunes lambda on DIOR and UCM, the same datasets that later appear in Table 6. These are not minor blemishes; they undermine the central SOTA claim.\n\nThat said, the paradigm and dataset deserve a serious referee. The right outcome is not a desk reject; it is a major revision where the authors release per-image split provenance and re-run all benchmarks under one protocol. The unseen-data results suggest the idea has legs, but the paper as presented cannot support its conclusions.\n\nRecommendation: send to peer review. The evaluation needs to be redone, but the contribution is worth the referees' time.","headline":"Genuinely interesting paradigm, but the benchmark numbers are compromised by train/test overlap and an internal inconsistency.","tokens_in":35796,"tokens_out":2238,"would_cite":false,"duration_ms":20207,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RemoteSAM proposes that a single referring-expression-segmentation model, trained on 270K remote-sensing triplets, can unify pixel-, region-, and image-level perception tasks and beat billion-parameter vision-language models on several…","keywords":["referring expression segmentation","Earth observation foundation model","remote sensing","task unification","image-text-mask dataset","multi-label classification","visual grounding","zero-shot segmentation"],"falsifier":"Train RemoteSAM-270K after deleting every image that overlaps the RRSISD, RisBench, DOTA, DIOR, and iSAID test splits, then rerun the reported benchmarks; if the mIoU gains (71.75 and 71.46) shrink to parity with prior methods, the SOTA claim is explained by train/test leakage. A public check could also compare image hashes between the released dataset and the official test sets.","tokens_in":34599,"feed_emoji":"🛰️","tokens_out":6164,"duration_ms":62708,"temperature":0.7,"pith_summary":"The paper argues that referring expression segmentation—asking a model to paint a pixel mask for whatever a text prompt describes—can serve as the single primitive for all vision-centric Earth observation tasks. RemoteSAM therefore replaces the usual menu of task-specific heads or large language models with one fusion-decoder that outputs masks and converts them into boxes, labels, counts, and captions. To train it, the authors built RemoteSAM-270K, the largest remote-sensing referring-segmentation dataset to date, with 270K image-text-mask triplets covering 297 categories and 16 attribute types, generated by a scalable automatic pipeline using large vision-language models and pseudo-label filtering. If the results hold, a single 180M-parameter architecture can match or beat billion-parameter specialist systems on segmentation, grounding, and multi-label classification while remaining much cheaper to run.","feed_headline":"One 180M model unifies eight Earth-observation vision tasks","feed_subtitle":"RemoteSAM turns pixel masks into detection, grounding, counting, and classification outputs without task-specific heads.","key_machinery":"The load-bearing device is the referring-expression-segmentation interface: input an image and a free-form text prompt, output a binary mask. The paper defines a generalized triplet regime—one-to-one, one-to-many (all instances of a category), and one-to-zero (absent categories, to suppress spurious masks)—which lets one model answer \"segment this thing,\" \"find everything of this class,\" and \"this class is not here.\" Around that mask the authors wrap deterministic converters: mask-to-bbox for grounding, contour refinement plus mask-to-bbox for object detection, per-class mask aggregation for semantic segmentation, and max-plus-average confidence pooling for classification and counting.","core_discovery":"RemoteSAM's central claim is that pixel-level masks are the fundamental output unit of vision, so every region- and image-level task can be derived from a mask by deterministic post-processing. The model is trained only for referring expression segmentation on RemoteSAM-270K, and at inference it turns the mask into a bounding box (mask-to-bbox), a semantic map (aggregating per-class masks), a multi-label prediction (pooling class confidence), an object count, and a caption. On benchmarks, it reports 71.75 mIoU on RRSISD and 71.46 on RisBench, surpassing prior referring-segmentation methods by more than 3 points; it also reports 94% and 95% multi-label accuracy on DIOR and DOTAv2, 74.36 AP50 on RSVG grounding, 62.0% counting accuracy on DIOR, and state-of-the-art zero-shot semantic segmentation on Potsdam (91.80 mF1), all with 180M parameters.","pith_inferences":["Left implicit: the mask-first interface could extend to multispectral, multitemporal, or radar imagery without changing the architecture, since band-specific or time-specific attributes can be expressed in the text prompt.","Testable extension: applying the same mask-conversion pipeline to natural-image referring-segmentation benchmarks would reveal whether task unification is a general vision principle or specific to remote sensing.","An unstated corollary: because image-level outputs are pooled from masks, referring-segmentation mIoU is the single health metric; gains there should propagate upward to detection, counting, and classification, making segmentation accuracy a sufficient optimization target.","The paper's transfer claims would be strengthened by explicitly reporting which images from RRSISD, RisBench, DOTA, DIOR, and iSAID were excluded from RemoteSAM-270K, since the dataset is built directly from those benchmarks."],"forward_implications":["If RemoteSAM is right, a single foundation model can serve pixel-, region-, and image-level remote sensing tasks without per-task decoders or LLM backbones.","The scale and semantic breadth of RemoteSAM-270K is claimed to be the main driver of generalization: more categories and attributes transfer to unseen datasets, as shown by zero-shot classification on SATIN and open-vocabulary segmentation on Vaihingen, UDD5, and DeepGlobe.","Because mask-based outputs are deterministic, the model's behavior on detection, counting, and classification is directly traceable to its segmentation quality; errors in masks become errors in all downstream tasks.","The parameter efficiency claim becomes concrete for deployment: 180M parameters versus 7B for vision-language rivals, which is material when processing high-resolution remote sensing imagery.","The RSVocab-1K vocabulary provides a reusable instrument for measuring whether any remote sensing dataset has enough category and attribute coverage to support foundation-model training."],"supporting_citations":[{"why":"Supplies the RMSIN architecture and the RRSISD benchmark that RemoteSAM builds upon and compares against.","marker":"[43]"},{"why":"Provides the RisBench dataset and CroBIM baseline used in the main referring-segmentation evaluation and as a data source for one-to-one triplets.","marker":"[17]"},{"why":"Provides the RefSegRS dataset and LGCE baseline, which defines the RRSIS task and is integrated into RemoteSAM-270K.","marker":"[79]"},{"why":"The Qwen2.5-VL model used by the automatic data engine to generate attribute-rich referring expressions from remote sensing images.","marker":"[73]"},{"why":"Provides the SAMRS code used to convert detection boxes from DOTA and other detection datasets into instance-level masks.","marker":"[65]"},{"why":"GroundedSAM2 is one of the expert models used to generate pseudo-labels for the attribute-expanded triplets.","marker":"[55]"},{"why":"SigLIP2 is used to compute similarity between mask-related image regions and referring expressions, filtering out unreliable pseudo-labels.","marker":"[61]"},{"why":"GeoChat is a central comparison baseline for multi-label classification and grounding performance.","marker":"[25]"},{"why":"Falcon is a comparison baseline for multi-label classification, object detection, and multi-task unification claims.","marker":"[76]"},{"why":"LHRS-Bot is a comparison baseline for multi-label classification, grounding, and counting performance.","marker":"[50]"}],"fun_headline_variants":["RemoteSAM: 8 tasks, 1 model, 180M params","Pixel masks as universal key: RemoteSAM","RemoteSAM: segment once, infer everything","One 180M model masters eight vision tasks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central numbers assume the RRSISD and RisBench test images never appear in RemoteSAM-270K's training triplets, since the paper builds the dataset from those same benchmarks and reports no held-out split.","fun_headline_variants_meta":{"raw":{"variants":["RemoteSAM: 8 tasks, 1 model, 180M params","Pixel masks as universal key: RemoteSAM","RemoteSAM: segment once, infer everything","One 180M model masters eight vision tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00036,"raw_usage":{"total_tokens":1963,"prompt_tokens":975,"completion_tokens":988,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":591,"completion_tokens_details":{"reasoning_tokens":925}},"tokens_in":591,"tokens_out":988,"duration_ms":10548,"temperature":1.0,"reasoning_tokens":925,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:35:59.430140+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train RemoteSAM-270K after deleting every image that overlaps the RRSISD, RisBench, DOTA, DIOR, and iSAID test splits, then rerun the reported benchmarks; if the mIoU gains (71.75 and 71.46) shrink to parity with prior methods, the SOTA claim is explained by train/test leakage. A public check could also compare image hashes between the released dataset and the official test sets.","supporting_citations":[{"cited_title":"Rotated multi-scale interaction network for referring remote sensing image seg- mentation","cited_arxiv_id":null,"evidence_quote":"Supplies the RMSIN architecture and the RRSISD benchmark that RemoteSAM builds upon and compares against."},{"cited_title":"Rrsis: Referring remote sensing image seg- mentation.IEEE TGRS, 2024","cited_arxiv_id":null,"evidence_quote":"Provides the RefSegRS dataset and LGCE baseline, which defines the RRSIS task and is integrated into RemoteSAM-270K."},{"cited_title":"Qwen2.5 tech- nical report, 2025","cited_arxiv_id":null,"evidence_quote":"The Qwen2.5-VL model used by the automatic data engine to generate attribute-rich referring expressions from remote sensing images."},{"cited_title":"Samrs: Scaling-up re- mote sensing segmentation dataset with segment anything model.NeurIPS, 36:8815–8827, 2023","cited_arxiv_id":null,"evidence_quote":"Provides the SAMRS code used to convert detection boxes from DOTA and other detection datasets into instance-level masks."},{"cited_title":"Geochat: Grounded large vision-language model for remote sensing","cited_arxiv_id":null,"evidence_quote":"GeoChat is a central comparison baseline for multi-label classification and grounding performance."},{"cited_title":"Lhrs-bot: Empowering remote sensing with vgi-enhanced large multimodal language model","cited_arxiv_id":null,"evidence_quote":"LHRS-Bot is a comparison baseline for multi-label classification, grounding, and counting performance."}],"review_version":1}