{"id":"13b7951e-f891-41ed-ae6e-adc78c5fdc7c","arxiv_id":"2507.12857","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"SCORE improves open-vocabulary remote sensing instance segmentation by injecting regional and global scene context from RemoteCLIP into CLIP-based class and text embeddings.","lead":"SCORE adds scene context from a remote-sensing vision-language model to an open-vocabulary instance segmentation pipeline, improving cross-dataset recognition of novel aerial object classes. The method reports state-of-the-art mAP on four remote sensing benchmarks, and the authors release code.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SOTA claim may mostly reflect seen-class recognition: most NWPU/SOTA and iSAID test categories are present in the training set, and no base/novel mAP split is reported.","rationale":"The reader's weakest assumption about Region-Aware Integration is plausible: if mask proposals for novel objects are poor, the regional context pooled in Eq. 5 can be misleading and amplify classification errors. That is a real mechanistic risk, and a per-class analysis would also help probe it. However, the more load-bearing issue is that the benchmark itself does not isolate novel categories. The paper's own category lists show substantial overlap between training and test sets, so the headline 'SOTA on open-vocabulary instance segmentation' could be true only in a weak cross-dataset-transfer sense, not in the standard open-vocabulary sense of recognizing unseen classes. Without a base/novel split, the empirical argument for the paper's core contribution is incomplete. This is not an accusation of dishonesty; the authors may have intended the cross-dataset setting as a form of open-vocabulary evaluation, and the method could still be valuable. The condition I would attach is to report novel-class mAP. The paper has other strengths: clear system design, released code, consistent ablations, and a reasonable comparison to existing OV baselines. The mixed semantic-segmentation extension in the supplement is a secondary concern because the central claim is about instance segmentation. Given these points, the reader's CONDITIONAL verdict remains appropriate, but the condition should explicitly include a base/novel class evaluation.","tokens_in":16501,"tokens_out":5802,"duration_ms":61086,"concrete_test":"Recompute the Table 1 evaluation with per-class AP split into 'base' classes (category name semantically present in the training dataset's category list in Table B) and 'novel' classes (absent from training), for SCORE and the strongest baselines (FC-CLIP and ZoRI) on all eight train/target combinations. If SCORE's average mAP advantage over baselines nearly vanishes on the novel subset, the SOTA claim does not support open-vocabulary generalization; if the advantage persists on novel classes, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To support the central 'open-vocabulary SOTA' claim, the evaluation must show that gains hold for categories absent from training. The paper defines open-vocabulary as Ctrain != Ctest, but the cross-dataset benchmarks do not enforce disjoint class sets. Training on iSAID and testing on NWPU, at least 9 of NWPU's 10 classes (ship, storage tank, baseball diamond, tennis court, basketball court, ground track field, harbor, bridge, vehicle) correspond to classes in iSAID; for SOTA, 14 of 18 categories overlap iSAID; and training on SIOR then testing on iSAID, roughly 12 of 15 iSAID categories overlap SIOR. Thus the overall mAP in Table 1 is dominated by seen classes, and the reported gains of +5.53 and +4.32 over the best baselines could be driven purely by base-class improvements. The paper never reports a base/novel class split, so the open-vocabulary contribution is not isolated. This is a correctness risk in the empirical argument, not an internal contradiction: the method might genuinely improve novel-class prediction, but the current evidence does not demonstrate it.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SCORE, a framework for open-vocabulary remote sensing instance segmentation. It uses a frozen remote-sensing CLIP (RemoteCLIP) to extract two types of scene context: regional context, obtained by pooling patch features inside an adaptively expanded predicted mask, and global context, taken from the image-level [CLS] token. Region-Aware Integration (RAI) refines class embeddings with regional context, while Global Context Adaptation (GCA) injects the global context into frozen CLIP text embeddings via multi-head cross-attention. The method is evaluated on cross-dataset benchmarks—training on iSAID or SIOR and testing on NWPU-VHR-10, SOTA, FAST, and SIOR—where it reports higher average mAP than ODISE, FC-CLIP, MAFT+, and ZoRI. A supplementary experiment extends the framework to remote sensing semantic segmentation.","tokens_in":16759,"tokens_out":6545,"duration_ms":70900,"significance":"If the empirical claims hold, the paper would provide a practical, modular way to exploit scene context for open-vocabulary instance segmentation in remote sensing, an under-explored task with clear application value. The work has several strengths: the design of RAI and GCA is clean and well motivated, the ablation studies cover important design axes (context VLM, context type, injection method), and the code is released. The qualitative results in Figure 4 provide intuitive evidence that regional context can disambiguate visually similar objects such as ships and vehicles. However, the central claim of 'SOTA performance' is currently not isolated from closed-vocabulary recognition: the cross-dataset benchmarks overlap heavily in class labels, and all design choices are selected using the test sets. These issues need to be addressed before the significance can be fully established.","major_comments":[{"comment":"The paper defines open-vocabulary learning as Ctrain != Ctest, but the cross-dataset benchmarks in Table 1 do not enforce class-disjoint splits. For example, when training on iSAID and testing on NWPU-VHR-10, most of the ten NWPU classes (including airplane, ship, storage tank, baseball diamond, tennis court, basketball court, ground track field, harbor, bridge, and vehicle) correspond to iSAID categories; the same issue holds for SOTA, where roughly 14 of 18 classes overlap iSAID, and for the SIOR-to-iSAID direction. As a result, the aggregate mAP values in Table 1 (38.68 and 32.59) are dominated by seen classes and do not isolate open-vocabulary performance. No base/novel class split is reported, so the reported gains of +5.53 and +4.32 over the best baselines could be driven entirely by improvements on seen classes. The qualitative examples in Figure 4 are suggestive but not quantitative. To support the central claim, the authors should report per-class mAP separately for base classes (present in training) and novel classes (absent from training), or restrict evaluation to a held-out subset of unseen classes. This is a load-bearing issue for the paper's main claim.","section":"§3.1, §4.1, Table 1"},{"comment":"The selection of the context VLM (RemoteCLIP vs. CLIP, SkyCLIP, GeoRSCLIP in Table 3) and of the global-context injection method (MHA vs. addition and concatenation in Table 5) is performed by comparing performance on the same test datasets used for the main results. No validation split or cross-validation is reported, and Tables 1, 3, and 5 present single runs without error bars or significance measures. This makes the reported numbers an optimistic estimate of true generalization and weakens the claim that the improvements are systematic. At minimum, the authors should evaluate ablations on a held-out split, report repeated-run statistics, or disclose that the design choices were tuned on the test sets. This is a correctness risk in the empirical argument, not a mathematical inconsistency.","section":"§4.4, Tables 3 and 5"},{"comment":"RAI pools RemoteCLIP patch features from the adaptively expanded predicted mask M'. For novel classes never seen in training, the proposal generator may produce inaccurate masks; in that case, the regional context is pooled from the wrong area and can reinforce an incorrect classification rather than resolve ambiguity. The paper does not analyze this sensitivity, for example by comparing regional context extracted from predicted masks against that extracted from ground-truth masks, or by quantifying proposal quality on novel classes. Since the claimed improvement on novel classes is attributed to RAI, this omission leaves the mechanism insufficiently validated.","section":"§3.4, Eq. (5)"}],"minor_comments":[{"comment":"In the component analysis, the text refers to 'VCA' and 'VPA' when describing the global context module; the consistent name introduced in §3.5 is GCA. Please correct these inconsistencies.","section":"§4.4"},{"comment":"The semantic segmentation extension reports a large drop on Potsdam (26.51 for SCORE vs. 43.29 for GSNet), and the text says the method outperforms 'across three of four benchmarks'. The abstract and main text claim 'achieves SOTA performance' without this caveat; please qualify the claim to match the actual results.","section":"Supplementary, Table A"},{"comment":"Equation (4) computes an integer kernel size k from a continuous learnable parameter δ with clamping. Please state how gradients are handled for the discrete max-pooling kernel size (e.g., straight-through estimator or rounding).","section":"§3.4, Eq. (4)"},{"comment":"The evaluation metric is reported as 'mAP' without specifying the exact protocol (e.g., COCO-style AP with IoU thresholds or PASCAL VOC AP). Please clarify the definition so the numbers are reproducible.","section":"§4.1"},{"comment":"The caption of Figure 4 mentions 'jittering=False' without explaining the setting. Please either provide a brief explanation or remove the phrase.","section":"Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant task and the proposed modules are reasonable, but the main claim of open-vocabulary SOTA is not yet supported because the evaluation does not separate seen and novel classes and the design choices are selected on the test sets. The requested experiments—per-class base/novel mAP and more rigorous ablation selection—are within the manuscript's scope and would considerably strengthen the contribution. I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know about arXiv:2507.12857: it's a solid systems paper for cross-dataset instance segmentation in remote sensing, but its headline 'open-vocabulary SOTA' is not actually supported by the numbers. The evaluation benchmarks share most of their categories with the training sets (NWPU 9/10 overlap iSAID, SOTA 14/18, SIOR roughly 12/15 against iSAID), so the reported +5.53/+4.32 mAP gains could be driven mostly by seen-class improvements. There is no base/novel split anywhere in the paper. That is the load-bearing weakness.\n\nWhat is genuinely new: the authors propose the first cross-dataset open-vocabulary instance segmentation benchmarks for remote sensing, and a method that uses a frozen RS-CLIP to extract regional context from dilated predicted masks (RAI) plus global context injected into text embeddings (GCA), sitting on top of a frozen ConvNeXt CLIP with Mask2Former. The idea is sensible and the ablations are systematic: both modules contribute, RAI more than GCA, and the qualitative results show that context helps disambiguate cases like ship vs. vehicle. The experiments cover multiple datasets and the code is released. That is real work.\n\nThe soft spots, in proportion: the class-overlap problem is the big one. It is fixable—report base/novel mAP, or build a disjoint-class split—but as written the central claim overreaches. Second, the choice of context VLM, context type, and injection method is made by looking at the same test sets, so there's a mild selection effect; the paper itself discloses that RS-CLIPs underperform general CLIP for OV classification, which complicates the 'domain-specific context helps' story. Third, no error bars or seeds. Finally, the semantic segmentation extension is a mixed bag: it gains on FLAIR and FAST but loses badly on Potsdam (26.51 vs 43.29), which the authors disclose but do not explain. That doesn't kill the instance segmentation result, but it should temper the generality claim.\n\nThe reader's take is close to mine. I'd send it to a serious referee—the empirical contribution is substantial enough—but the referee should demand a base/novel evaluation before any acceptance. If the gains hold on truly novel classes, this becomes a useful benchmark for the community. As is, it's a reasonable cross-dataset segmentation method whose 'open-vocabulary' framing is not yet earned.","headline":"Solid cross-dataset RS instance segmentation system, but the 'open-vocabulary' SOTA claim is not demonstrated because the benchmarks mostly share classes with training; no base/novel split.","tokens_in":17262,"tokens_out":2770,"would_cite":false,"duration_ms":30510,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SCORE introduces open-vocabulary remote sensing instance segmentation and claims state-of-the-art cross-dataset results by injecting regional and global scene context into CLIP-based class and text embeddings.","keywords":["open-vocabulary segmentation","remote sensing","instance segmentation","scene context","region-aware integration","global context adaptation","CLIP","cross-dataset generalization"],"falsifier":"Build or select a remote sensing test set where objects appear in atypical environments, such as ships on land or vehicles in water, and compare SCORE against its no-context baseline on that set; if the mAP advantage does not shrink, the reported gains are not actually caused by the environmental priors the paper claims.","tokens_in":16320,"feed_emoji":"🛰️","tokens_out":9445,"duration_ms":94645,"temperature":0.7,"pith_summary":"Open-vocabulary instance segmentation in remote sensing is nearly unexplored: models trained on natural images fail on aerial data, where objects are small, visually ambiguous, and strongly tied to their surroundings. The paper claims that this scene context is the missing signal, and that injecting it into both visual and textual representations closes the gap. Its SCORE framework extracts regional context from a remote-sensing CLIP model's patch embeddings, pools it inside adaptively dilated mask proposals, and refines class embeddings with it; a second module injects global context into frozen CLIP text embeddings. On new cross-dataset benchmarks, the model trained on iSAID reaches 38.68 average mAP versus 33.24 for the best prior method, and trained on SIOR it reaches 32.59 versus 28.27. If true, one model trained on a single annotated dataset can segment unseen object classes in other aerial datasets using only their names.","feed_headline":"Surrounding context lets models segment unseen aerial objects","feed_subtitle":"Trained on one dataset, SCORE recognizes novel objects in unseen aerial scenes using surrounding context.","key_machinery":"The load-bearing mechanism is the pair of context-injection modules, with Region-Aware Integration carrying the main argument. In RAI, adaptive region formation dilates each predicted mask by max-pooling with kernel size $k = 3 + \\mathrm{clamp}(\\delta, 0, 10)$, then pools RemoteCLIP patch embeddings inside the expanded mask into a regional context $F_{\\mathrm{region}}$, which is injected into class embeddings $V$ through $l$ sequential transformer layers with a temperature coefficient $\\lambda$. Global Context Adaptation forms the complementary mechanism: it takes the RemoteCLIP [CLS] token as the query in multi-head cross-attention over text embeddings $T$, producing adapted embeddings $\\hat{T}$. The region-aware class embeddings $\\hat{V}$ and adapted text embeddings $\\hat{T}$ are scored together, combining in-vocabulary and out-of-vocabulary predictions at inference.","core_discovery":"The central discovery is that multi-granularity scene context, taken from a remote-sensing-aware CLIP model, resolves the ambiguity that blocks open-vocabulary instance segmentation in aerial imagery. Region-Aware Integration (RAI) uses each predicted mask as a spatial reference, expands it with a learnable dilation, and mask-pools the RemoteCLIP patch embeddings inside that region to form a regional context vector; the vector is then fused into the class embeddings through stacked transformer layers. Global Context Adaptation (GCA) projects the RemoteCLIP [CLS] token as a query and cross-attends over the frozen text embeddings, producing a classifier adapted to remote sensing while keeping its open-vocabulary generality. With both modules, the paper reports 38.68 average mAP when trained on iSAID and 32.59 when trained on SIOR across the four test datasets, and the same modules also improve open-vocabulary semantic segmentation in the supplementary experiments.","pith_inferences":["The paper does not separate the contribution of the context modules from the contribution of running two frozen CLIP encoders and ensembling their outputs; an ablation that keeps the two encoders but disables only the context fusion would isolate that effect.","The regional-context idea should transfer to other dense geospatial tasks where mask proposals exist, such as open-vocabulary object detection or panoptic segmentation, but the paper does not test those settings.","Because the context branch is a frozen remote-sensing CLIP, replacing it with a larger or better remote-sensing vision-language model would likely improve SCORE without retraining the rest of the system.","If object-environment correlations were deliberately broken, for example by placing ships on land and cars in water, the reported advantage over the no-context baseline should shrink; the paper does not run that stress test."],"forward_implications":["A single model trained on one annotated remote sensing dataset can segment object classes it has never seen in other aerial datasets, provided the class names are supplied at test time.","Objects that are visually similar but live in different environments, such as vehicles and ships, can be separated using the surrounding scene rather than the object alone.","The same context-injection modules transfer beyond instance segmentation, improving open-vocabulary semantic segmentation results reported in the supplement.","The new cross-dataset benchmarks give later work a fixed protocol for measuring open-vocabulary remote sensing instance segmentation."],"supporting_citations":[{"why":"Supplies the RemoteCLIP visual encoder whose [CLS] token and patch embeddings provide the global and regional scene context used by both modules.","marker":"[30]"},{"why":"FC-CLIP is the strongest prior open-vocabulary segmentation model and the source of the frozen ConvNeXt CLIP backbone and out-of-vocabulary scores.","marker":"[63]"},{"why":"iSAID is the large aerial instance segmentation dataset used to train the model in the first benchmark.","marker":"[64]"},{"why":"SAMRS provides the SIOR, SOTA, and FAST test datasets as well as the SIOR training split.","marker":"[48]"},{"why":"ZoRI is the prior zero-shot remote sensing instance segmentation method that SCORE compares against, especially in the SIOR-trained setting.","marker":"[18]"},{"why":"Provides the open-vocabulary protocol of training on one dataset and testing on others that defines the benchmarks.","marker":"[55]"},{"why":"Mask2Former supports the proposal generator that produces the mask proposals and class embeddings for the instance branch.","marker":"[6]"}],"fun_headline_variants":["Scene context lets models segment unseen aerial objects","Context unlocks open-vocabulary segmentation in satellite images","SCORE uses scene context to recognize new aerial objects","Global and local context boost open-vocab aerial segmentation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes the predicted mask proposals from the instance branch are spatially accurate enough to define an object's surrounding region; if a proposal for a novel class is wrong, the regional context pooled from the wrong area can inject misleading information into the class embedding.","fun_headline_variants_meta":{"raw":{"variants":["Scene context lets models segment unseen aerial objects","Context unlocks open-vocabulary segmentation in satellite images","SCORE uses scene context to recognize new aerial objects","Global and local context boost open-vocab aerial segmentation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001115,"raw_usage":{"total_tokens":4671,"prompt_tokens":998,"completion_tokens":3673,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":614,"completion_tokens_details":{"reasoning_tokens":3611}},"tokens_in":614,"tokens_out":3673,"duration_ms":28632,"temperature":1.0,"reasoning_tokens":3611,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:36:08.145379+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build or select a remote sensing test set where objects appear in atypical environments, such as ships on land or vehicles in water, and compare SCORE against its no-context baseline on that set; if the mAP advantage does not shrink, the reported gains are not actually caused by the environmental priors the paper claims.","supporting_citations":[{"cited_title":"Remoteclip: A vision language foundation model for remote sensing.IEEE Transactions on Geoscience and Remote Sensing, 2024","cited_arxiv_id":null,"evidence_quote":"Supplies the RemoteCLIP visual encoder whose [CLS] token and patch embeddings provide the global and regional scene context used by both modules."},{"cited_title":"Convolutions die hard: Open-vocabulary segmentation with single frozen convolutional clip","cited_arxiv_id":null,"evidence_quote":"FC-CLIP is the strongest prior open-vocabulary segmentation model and the source of the frozen ConvNeXt CLIP backbone and out-of-vocabulary scores."},{"cited_title":"isaid: A large-scale dataset for instance segmentation in aerial images, 2019","cited_arxiv_id":null,"evidence_quote":"iSAID is the large aerial instance segmentation dataset used to train the model in the first benchmark."},{"cited_title":"Samrs: Scaling-up remote sensing segmentation dataset with segment anything model.Advances in Neural Information Processing Systems , 36:8815–8827,","cited_arxiv_id":null,"evidence_quote":"SAMRS provides the SIOR, SOTA, and FAST test datasets as well as the SIOR training split."},{"cited_title":"Zori: Towards discriminative zero-shot remote sensing instance segmentation","cited_arxiv_id":null,"evidence_quote":"ZoRI is the prior zero-shot remote sensing instance segmentation method that SCORE compares against, especially in the SIOR-trained setting."},{"cited_title":"A simple baseline for open- vocabulary semantic segmentation with pre-trained vision- language model","cited_arxiv_id":null,"evidence_quote":"Provides the open-vocabulary protocol of training on one dataset and testing on others that defines the benchmarks."},{"cited_title":"Masked-attention mask transformer for universal image segmentation","cited_arxiv_id":null,"evidence_quote":"Mask2Former supports the proposal generator that produces the mask proposals and class embeddings for the instance branch."}],"review_version":1}