{"id":"6e84ce73-94e1-402d-bc02-8d7608cf67aa","arxiv_id":"2412.12798","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"ZoRI combines CLIP text-channel selection, partial fine-tuning, and a pseudo-label cache bank to segment unseen aerial classes, but the cache bank is seeded with the model's own test-set predictions.","lead":"This paper introduces zero-shot remote sensing instance segmentation and a framework, ZoRI, that adapts CLIP with text-channel selection, partial backbone fine-tuning, and a cache bank of visual prototypes. It claims state-of-the-art results on two new benchmarks, but the zero-shot claim is weakened by the use of test-time pseudo-labels to build the cache bank.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PIP's unseen-class cache bank is built from the model's own top-1 predictions on the test set and re-scores those same images, so the zero-shot SOTA comparison is transductive and not a fair inductive benchmark.","rationale":"I agree with the reader's weakest_assumption. I looked for an alternative single point of failure, such as the mismatch between the pretrained CLIP encoder used for cache keys and the KMA-adapted backbone used for queries, or the selection of top-300 channels on test-class text embeddings; while both are worth noting, neither is as load-bearing as the test-time pseudo-label loop. The paper's own supplementary text confirms the loop explicitly, and the component ablation quantifies its contribution. I am not attacking the authors' intent or the internal consistency of the modules: DEC and KMA plausibly provide real gains, since DEC+KMA already exceeds FC-CLIP by a wide margin on unseen mAP (8.64 vs 4.91 on iSAID). But the headline claim is about the full ZoRI pipeline, and the full pipeline uses test-set feedback during inference. That makes the state-of-the-art comparison unfair as a zero-shot claim, and the correct fix is to report the variant without test-time pseudo-labels as the zero-shot method, or to explicitly reframe PIP as transductive with appropriate transductive baselines. Under either fix the paper could be resubmitted; as it stands the reader's REJECT verdict is appropriate.","tokens_in":17284,"tokens_out":5427,"duration_ms":50364,"concrete_test":"Use the released code to rerun the iSAID GZSRI evaluation (Table 5) with the unseen-class entries of the cache bank removed (set P=0 in Eq. (4) while keeping the seen-class prototypes and alpha unchanged). Recompute unseen mAP and HM; if unseen mAP falls to the 8.64 of the DEC+KMA row in Table 1, the 9.30 result is produced by test-set pseudo-labels and the zero-shot SOTA claim fails as stated. If the metric stays near 9.30, the concern is refuted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that ZoRI achieves state-of-the-art zero-shot remote sensing instance segmentation. For this to hold, the evaluation must not use test-set information that inductive baselines do not have. Supplementary B (Testing) violates that condition: \"For unseen classes, we use predictions with the highest probability after incorporating DEC and KMA to get visual samples,\" and \"we only use the top-1 probability prediction as the pseudo visual sample for unseen classes.\" Because unseen-class ground truth is unavailable, these predictions are the model's own outputs on the test images. These pseudo-labeled features are concatenated into the cache bank F in Eq. (4), and Eqs. (5)-(7) then use the mask-pooled feature of the same test segment as query f to compute logits_pip, which is added to the final prediction. The exact test images thus contribute their predicted labels to the classifier that scores them.\n\nThis is a transductive self-training loop, not inductive zero-shot inference. The paper never labels PIP as transductive and provides no transductive baselines. Consequently, the reported margin over FC-CLIP (unseen mAP 9.30 vs 4.91 on iSAID, Table 5) conflates genuine DEC/KMA improvements with test-set feedback. The component analysis makes this visible: DEC+KMA without PIP reaches 8.64 unseen mAP, and adding PIP raises it to 9.30; on NWPU the corresponding jump is 9.32 to 12.26. The increment is the quantity built from test predictions, so the comparison is not a clean zero-shot evaluation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ZoRI, a framework for zero-shot remote sensing instance segmentation. It formulates a new task and establishes benchmarks on iSAID, NWPU-VHR-10, and SIOR. The method combines three components: a Discrimination-Enhanced Classifier (DEC) that selects discriminative text-embedding channels, a Knowledge-Maintained Adaptation (KMA) that partially fine-tunes the CLIP vision backbone while freezing semantic-related channels, and a Prior-Injected Prediction (PIP) that uses a cache bank of visual prototypes to re-weight the CLIP zero-shot classifier. The authors report state-of-the-art results over FC-CLIP and older zero-shot instance segmentation methods under both ZSRI and GZSRI settings, with code and benchmarks released.","tokens_in":17620,"tokens_out":7682,"duration_ms":68476,"significance":"If the evaluations were conducted under a standard inductive zero-shot protocol, the paper would be a meaningful contribution: it is the first to formulate zero-shot instance segmentation for remote sensing, it releases code and benchmarks, and the DEC and KMA components alone yield substantial gains over the FC-CLIP baseline (unseen mAP 8.64 vs 4.91 on iSAID in the GZSRI setting). However, the headline claim is compromised because the PIP module constructs its unseen-class cache bank from the model's own top-1 predictions on the test set and then uses that bank to re-score the same test segments. This is a transductive self-training loop, and the paper does not disclose it or compare with transductive baselines. The feature-space mismatch between the cache keys and the query adds further uncertainty about the source of the reported gains. As evaluated, the contribution does not support the zero-shot claim made in the abstract and title.","major_comments":[{"comment":"The unseen-class cache bank is built from the model's own top-1 predictions on the test set. Supplementary B states: 'For unseen classes, we use predictions with the highest probability after incorporating DEC and KMA to get visual samples' and 'we only use the top-1 probability prediction as the pseudo visual sample for unseen classes.' These pseudo-labeled features are concatenated into the cache bank F in Eq. (4), and Eqs. (5)-(7) then use the mask-pooled feature of the same test segment as the query f to compute logits_pip, which is added to the final score. The exact test images therefore contribute their own predicted labels to the classifier that scores them. This is a transductive self-training loop, not inductive zero-shot inference, and the paper neither discloses it as transductive nor compares with transductive baselines such as test-time adaptation or transductive ZSL methods. The component analysis in Table 1 shows that the PIP increment (iSAID unseen mAP 8.64 to 9.30; NWPU 9.32 to 12.26) is precisely the quantity built from test predictions, so the reported margin over inductive baselines such as FC-CLIP (Table 5) is not a fair zero-shot comparison. I recommend either removing the pseudo-labeled unseen cache bank or explicitly reframing the method as transductive zero-shot and evaluating against transductive baselines.","section":"Prior-Injected Prediction (PIP), Supplementary B (Testing), Eqs. (5)-(7), Table 1"},{"comment":"There is an apparent feature-space mismatch in the cache bank prediction. The cache bank keys F in Eq. (4) are 'obtained using the pretrained CLIP image encoder Evis', while the query f in Eq. (5) is the 'mask-pooled CLIP-extracted backbone feature' from the KMA-adapted backbone used at inference. If Evis is the original frozen CLIP image encoder and the query comes from the partially fine-tuned backbone, the cosine similarity in Eq. (5) compares features from two different encoders, with the KMA trainable/frozen channel split making the two feature spaces differ along at least the 32 trainable channels. The paper should clarify which encoder is used for the unseen-class pseudo samples and for the query, and should justify that the cross-encoder similarity is meaningful; otherwise the PIP gains in Tables 1 and 5 may be an artifact of the encoder mismatch.","section":"Cache Bank Construction, Eq. (4) vs Eq. (5), Supplementary B (Testing)"},{"comment":"Hyperparameters for the final model, including the DEC channel count (300), cache size (4), and α (0.5), are selected using the same iSAID split on which the final results are reported (Tables 2 and 4; Implementation Details). No held-out validation split is described for this selection. Because the reported SOTA numbers are the result of test-set-based model selection, the comparison with baselines is not a clean evaluation of the method's inductive performance. The authors should either use a separate validation split for hyperparameter selection or clearly disclose that the test set was used for model selection.","section":"Experimental Setup and Implementation Details, Tables 2 and 4"}],"minor_comments":[{"comment":"'state-of-art' should be 'state-of-the-art' in the abstract and in the contribution bullet.","section":"Abstract and Introduction"},{"comment":"'ground-level prospective' should be 'ground-level perspective' in both places this phrase appears.","section":"Figure 2 caption and Method (KMA)"},{"comment":"'untilized' should be 'utilized' in the paragraph on zero-shot learning in remote sensing.","section":"Related Work"},{"comment":"'discirminative power' is a typo for 'discriminative power'.","section":"Method (DEC)"},{"comment":"'habor' should be 'harbor' in the captions of Figures 6 and 7.","section":"Figures 6 and 7 captions"}],"recommendation":"reject","confidential_remarks":"The transductive protocol issue is decisive in my assessment. The authors could potentially resubmit by reframing PIP as test-time adaptation and benchmarking against transductive methods, or by reporting an inductive variant without the unseen-class cache. In the current form, the abstract and title overclaim a zero-shot result, and the feature-space mismatch further undermines the PIP attribution. I would not require additional experiments for this version; the central claim needs to be reframed or the evaluation protocol changed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper introduces zero-shot remote sensing instance segmentation as a task, builds two new benchmarks with seen/unseen splits, and proposes a three-module method. The task framing is genuinely new, and the benchmarks look carefully constructed. DEC's channel selection and KMA's partial channel freezing both show real gains in the ablations; the component analysis is thorough and honest about trade-offs. So there is solid material here.\n\nThe problem is the evaluation of PIP. Supplementary B says that for unseen classes, the cache bank is built from the model's own top-1 predictions on the test set, and Eqs. (5)-(7) then score those same test images against that cache bank. That is a transductive self-training loop, not inductive zero-shot inference, and the paper never labels it as such. No transductive baselines are compared. The reported gains over FC-CLIP (unseen mAP 9.30 vs 4.91 on iSAID, 12.26 vs 6.86 on NWPU) therefore conflate genuine DEC/KMA improvements with test-set feedback. The component tables make this visible: on iSAID, PIP added to DEC+KMA gives +0.66 unseen mAP; on NWPU it gives +2.94, and PIP alone gives +1.63 on NWPU but only +0.25 on iSAID. That pattern is exactly what you would expect if the gain is mostly from re-scoring the model's own confident test predictions, which is more pronounced on the smaller, easier NWPU set.\n\nThere is also a lesser feature-space issue: the cache bank is built with the frozen pretrained CLIP encoder, while the query features come from the KMA partially fine-tuned backbone. The paper does not discuss this mismatch, though it may be minor in practice.\n\nIf the authors either remove test-time pseudo-labels and evaluate PIP purely inductively, or explicitly reframe the method as transductive and compare against transductive baselines, the central claim becomes supportable. As written, the zero-shot SOTA claim is not.\n\nWho gets value: readers working on open-vocabulary or zero-shot segmentation for remote sensing will find the task definition and benchmarks useful, and DEC/KMA are worth studying. But the experimental conclusions should be read with caution. This deserves a serious referee because the task is new and the flaw is fixable. My recommendation: send it to review, but make the transductive issue the deciding condition.","headline":"New task and benchmarks, reasonable DEC/KMA ideas, but the headline SOTA claim is compromised by a transductive PIP loop that uses the model's own test-set predictions to re-score the same test images.","tokens_in":18141,"tokens_out":2079,"would_cite":false,"duration_ms":20056,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A new framework, ZoRI, claims state-of-the-art zero-shot instance segmentation for remote sensing by refining CLIP text channels, adapting only part of the image encoder, and injecting aerial visual prototypes.","keywords":["zero-shot instance segmentation","remote sensing","vision-language models","CLIP adaptation","cache bank","channel selection","iSAID","NWPU-VHR-10"],"falsifier":"Re-run the GZSRI evaluation on iSAID with the PIP cache bank built only from seen-class training prototypes and no test-image pseudo-labels for unseen classes; if the unseen-class mAP falls back toward the 4.91 FC-CLIP baseline, the reported 9.30 result would be attributable to self-labeling on the test set rather than to the domain adaptation methods.","tokens_in":17003,"feed_emoji":"🛰️","tokens_out":3405,"duration_ms":30794,"temperature":0.7,"pith_summary":"The paper introduces zero-shot remote sensing instance segmentation as a new task and proposes ZoRI to solve it. It claims that ZoRI segments aerial object classes never seen during training, reaching an unseen-class mAP of 9.30 on iSAID under the generalized zero-shot setting, compared with 4.91 for the FC-CLIP baseline. The method combines three mechanisms: channel selection that sharpens the text classifier, partial adaptation of the vision encoder that preserves pretrained vision-language alignment, and a cache bank of aerial visual prototypes that reweights the final predictions. If the claim holds, it would let vision-language models recognize novel aerial categories without any mask annotations for those categories.","feed_headline":"ZoRI hits 9.30 unseen mAP in zero-shot aerial segmentation","feed_subtitle":"CLIP adaptation plus a visual cache recognize bird's-eye classes never seen in training, no mask labels needed.","key_machinery":"The load-bearing mechanism is a three-part pipeline built on FC-CLIP. DEC (Discrimination-Enhanced Classifier) selects the top-300 text-embedding channels that minimize inter-class similarity and maximize cross-class variance, sharpening the zero-shot classifier. KMA (Knowledge-Maintained Adaptation) freezes the 160 backbone channels most aligned with semantic content and trains only the remaining 32 channels, so the pretrained vision-language alignment survives while features adapt to bird's-eye imagery. PIP (Prior-Injected Prediction) builds a cache bank of CLIP visual prototypes from training instances and top-1 test predictions, then adds a cosine-similarity-based logit to the text classifier output through a weighted sum.","core_discovery":"The central claim is that zero-shot remote sensing instance segmentation can be made effective by adapting a frozen CLIP model to the aerial domain instead of using it directly. On iSAID and NWPU-VHR-10, under both the zero-shot and generalized zero-shot protocols, ZoRI reports higher unseen-class mAP and harmonic-mean mAP than existing zero-shot segmentation methods including FC-CLIP, ZSI, and D2Zero. The authors attribute the gain to three components: a discrimination-enhanced classifier that keeps only the most class-discriminative channels of the text embeddings, a knowledge-maintained adaptation that freezes semantic-related vision channels while training the rest to capture aerial cues, and a prior-injected prediction that merges text-based scores with a cache bank of visual prototypes from seen and pseudo-labeled unseen instances.","pith_inferences":["Beyond the paper, the top-1 pseudo-labels used to build the unseen-class cache bank likely make the method transductive: the cache is created from the very test images being scored, so the reported gains over inductive baselines may shrink if the cache is built only from seen-class training prototypes.","A testable extension would be to replace the pseudo-labeled unseen prototypes with class-description prototypes generated from large language models, keeping the cache training-free while removing any reliance on test predictions.","The same channel-decoupling idea could be transferred to other vision-language backbones that are not Convolutional CLIP, such as transformer-based encoders, to check whether the preservation of semantic channels generalizes beyond the tested architecture.","The benchmark splits themselves are a contribution that invites scrutiny: the paper derives the iSAID split from a detection paper and clusters semantic embeddings for NWPU-VHR-10, so results across future methods may be sensitive to how that clustering was done."],"forward_implications":["Unseen aerial classes such as swimming pools, helicopters, and tennis courts can be segmented without any mask annotations for those classes, using only text prompts and visual prototypes.","The channel-selection criterion used by DEC can be applied to any CLIP-based zero-shot classifier, not just instance segmentation, whenever classes are confusable.","Partially freezing the vision encoder is claimed to outperform both full fine-tuning and visual prompt tuning for adapting CLIP to a specialized domain while retaining zero-shot ability.","The cache bank injection shows that a small number of visual prototypes, here four per class, is enough to improve classification when text embeddings are too generic.","The reported state-of-the-art results establish the first benchmarks for zero-shot remote sensing instance segmentation, so future methods can be compared on the same splits and metrics."],"supporting_citations":[{"why":"Supplies the FC-CLIP baseline and the overall frozen-convolutional-CLIP architecture that ZoRI extends.","marker":"(Yu et al. 2023)"},{"why":"Supplies the CLIP vision-language model whose text embeddings and backbone the paper refines and adapts.","marker":"(Radford et al. 2021)"},{"why":"Supplies the Mask2Former mask generator and the 300-object-query design used for mask prediction.","marker":"(Cheng et al. 2022)"},{"why":"Supplies the RESISC45 prompt templates used to compute text embeddings for remote sensing class names.","marker":"(Cheng, Han, and Lu 2017)"},{"why":"Supplies the training-free cache-bank adaptation idea from Tip-Adapter that PIP extends with visual prototypes.","marker":"(Zhang et al. 2021a)"},{"why":"Supplies the seen/unseen class split for iSAID that the paper adopts, and frames the zero-shot aerial detection setting.","marker":"(Zang et al. 2024)"},{"why":"Supplies the ZSI zero-shot instance segmentation baseline and the Recall@100 plus mAP evaluation protocol.","marker":"(Zheng et al. 2021)"},{"why":"Supplies the D2Zero baseline for comparison and the semantic-debiasing approach the paper positions against.","marker":"(He, Ding, and Jiang 2023b)"}],"fun_headline_variants":["ZoRI cracks zero-shot aerial instance segmentation","Zero-shot aerial segmentation: ZoRI outperforms existing methods","CLIP plus visual cache enables zero-shot aerial masks","ZoRI: State-of-the-art zero-shot instance segmentation aloft","Adaptive CLIP learns unseen aerial objects zero-shot"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The performance gain rests on using the model's own top-1 predictions on the test images to build the cache bank for unseen classes, which assumes this counts as legitimate zero-shot evaluation rather than test-set access.","fun_headline_variants_meta":{"raw":{"variants":["ZoRI cracks zero-shot aerial instance segmentation","Zero-shot aerial segmentation: ZoRI outperforms existing methods","CLIP plus visual cache enables zero-shot aerial masks","ZoRI: State-of-the-art zero-shot instance segmentation aloft","Adaptive CLIP learns unseen aerial objects zero-shot"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000262,"raw_usage":{"total_tokens":1614,"prompt_tokens":979,"completion_tokens":635,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":595,"completion_tokens_details":{"reasoning_tokens":557}},"tokens_in":595,"tokens_out":635,"duration_ms":6254,"temperature":1.0,"reasoning_tokens":557,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:42:57.503775+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the GZSRI evaluation on iSAID with the PIP cache bank built only from seen-class training prototypes and no test-image pseudo-labels for unseen classes; if the unseen-class mAP falls back toward the 4.91 FC-CLIP baseline, the reported 9.30 result would be attributable to self-labeling on the test set rather than to the domain adaptation methods.","supporting_citations":[{"cited_title":"G.; Kirillov, A.; and Girdhar, R","cited_arxiv_id":null,"evidence_quote":"Supplies the Mask2Former mask generator and the 300-object-query design used for mask prediction."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the RESISC45 prompt templates used to compute text embeddings for remote sensing class names."},{"cited_title":"Zero-Shot Aerial Object Detection with Visual Description Regularization","cited_arxiv_id":"2402.18233","evidence_quote":"Supplies the seen/unseen class split for iSAID that the paper adopts, and frames the zero-shot aerial detection setting."},{"cited_title":"Zero-Shot Instance Segmentation","cited_arxiv_id":"2104.06601","evidence_quote":"Supplies the ZSI zero-shot instance segmentation baseline and the Recall@100 plus mAP evaluation protocol."}],"review_version":1}