{"id":"847d9408-2e4d-48aa-a63d-cafd580b39bb","arxiv_id":"2608.10426","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"GeoSeg-OV outperforms prior open-vocabulary remote sensing segmentation methods by 2.5 to 2.7 average mIoU on a new seven-dataset benchmark by repurposing auxiliary vision foundation models as structural guidance rather than as additional visual-text matchers.","lead":"GeoSeg-OV is a new computer vision model for remote sensing that recognizes land cover categories described in plain language, even categories never seen during training. It improves cross-dataset accuracy by using a general-purpose vision model to supply spatial structure clues instead of treating it as an additional text-matching signal.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The +2.5/+2.7 mIoU claim is single-run with no error bars or seeds; if run-to-run variance is typical for cost-aggregation OVSS, the headline gain may be within noise. A multi-seed paired test would settle whether the central claim is robust.","rationale":"Choosing this concern over the reader's 'transferability' assumption is deliberate. The paper's strongest evidence for the design principle is not the mechanism of A_bias but the controlled SGA-vs-AVTM comparison in Table 3, which holds the auxiliary encoder fixed and changes only how its features are used. That experimental design already addresses much of the reader's worry about whether structural features transfer: SGA beats AVTM with two different encoders, and Table 4 shows the full framework is robust across three VFMs. What remains untested is whether any of these comparisons would survive reseeding. Since the headline claim is an average-mIoU advantage of 2.5-2.7 points, and the paper reports a single run per configuration, the difference between a real effect and a favorable run is unresolved. The DeepGlobe row under FLAIR training (GeoSeg-OV 46.9 vs OVRS 48.1) is an explicit case where the method is not uniformly better, so the average gain is carried by the other datasets; this makes the stability question more acute. A five-seed evaluation with paired bootstrap intervals is a straightforward check that would settle it. If the gain is statistically robust, the central claim stands; if not, the paper needs recharacterization regardless of the structural-transfer story. My verdict remains CONDITIONAL, consistent with the reader, but for a slightly different reason. The manuscript should also repair the abstract's overstatement of the qualitative Wuhan/SinoLC-1 comparison, as the reader notes, but that is secondary to the statistical robustness of the quantitative claim.","tokens_in":26456,"tokens_out":10733,"duration_ms":102694,"concrete_test":"Retrain GeoSeg-OV, OVRS, and RSKT-Seg (or the best baseline per setting) from scratch under at least five seeds, separately for the FLAIR and OpenEarthMap training settings, holding splits, iterations, and batch size fixed. For each seed, compute the per-run average mIoU over the same six evaluation datasets; then form paired differences GeoSeg-OV minus best baseline within each seed and report the mean difference with a bootstrap/permutation 95% confidence interval. If the interval excludes zero in both training settings, the headline +2.5/+2.7 gain is statistically supported; if it includes zero, or if the between-seed standard deviation is comparable to the gain, the central empirical claim is not established and the paper should be revised to report uncertainty.","verdict_should_be":"UNCHANGED","load_bearing_attack":"GeoSeg-OV's central claim is an empirical superiority claim: it beats the best prior cost-aggregation methods by +2.5 and +2.7 average mIoU on HRLC (Table 1), and Table 3's SGA-vs-AVTM comparison is the experiment used to attribute those gains to decoupling VFM features from visual-text matching. Every number in Tables 1-5 is from a single training run; no error bars, seed counts, or significance tests are reported. This matters because the headline margin is only about 2.5-2.7 mIoU averaged over six datasets, while per-dataset differences swing far more (Potsdam +5.7 above the second-best method, DeepGlobe -1.2 below OVRS under FLAIR training). Single-seed segmentation runs commonly vary by 1-2 mIoU, which is the same order as the claimed advantage. Without a distribution over runs, the paper has not shown that the +2.5/+2.7 gain is outside noise, and therefore has not shown that the proposed structural-guidance design is responsible for the SOTA result. The transferability concern raised by the reader is real but secondary: the controlled Table 3 comparison would still support the design principle if the underlying numbers were statistically stable. The load-bearing uncertainty is the statistical robustness of the headline gain itself.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GeoSeg-OV, a trainable open-vocabulary remote sensing segmentation framework. It constructs a multi-rotation CLIP cost volume as the sole visual–text matching evidence, uses a frozen auxiliary vision foundation model (Depth Anything V2 by default) to derive pairwise structural biases and multi-scale guidance outside the matching space, introduces Structure-Guided Aggregation (SGA) with text-conditioned class-wise reasoning, and proposes Cost-Aware Decoding (CAD) to adaptively fuse semantic and structural guidance. A new benchmark (HRLC) spanning seven datasets across six continents is introduced. Under FLAIR and OpenEarthMap training, the method reports average mIoU gains of +2.5 and +2.7 over the best prior cost-aggregation methods, with extensive ablations comparing SGA with the AVTM paradigm, three auxiliary encoders, boundary quality, efficiency, and a large-scale zero-shot case study.","tokens_in":26717,"tokens_out":5970,"duration_ms":54323,"significance":"If the reported gains are statistically stable, the paper makes a useful conceptual contribution: it challenges the AVTM paradigm of using auxiliary VFM features as additional matching evidence and instead uses them as structural priors. The experimental design is generally careful: all cost-aggregation baselines share the same CLIP backbone, data splits, iteration counts, batch sizes, and multi-rotation construction where applicable; Table 3 is a clean controlled comparison; Table 4 shows robustness across three VFMs; the HRLC benchmark is a valuable resource; and code/benchmark availability strengthens reproducibility. There is no indication of circularity: training and evaluation splits are disjoint and no target-domain annotations are used. The main unresolved issue is that all headline numbers are single-run, which is load-bearing for the empirical superiority claim.","major_comments":[{"comment":"All results in Tables 1–5 are single training runs, with no error bars, seed counts, or significance tests reported. The average gains of +2.5/+2.7 mIoU are the central claim, while per-dataset differences swing from +5.7 to -1.2 (e.g., DeepGlobe under FLAIR), and single-seed segmentation runs commonly vary by 1–2 mIoU. The paper should report mean ± std over at least three seeds for the main methods and apply a paired significance test (e.g., paired t-test or Wilcoxon signed-rank) for GeoSeg-OV versus OVRS and RSKT-Seg under both training settings. Without this, the 'state-of-the-art' conclusion is not statistically established.","section":"Section 4.2, Tables 1–5"},{"comment":"The key controlled comparison that attributes the gain to the SGA design principle (decoupling VFM features from matching) also rests on single runs. The margins here are smaller (+1.1 to +2.0 mIoU across encoders and settings), so they are especially exposed to training noise. Please provide multi-seed results and a significance test for this comparison as well, and specify the exact AVTM construction (e.g., whether the VFM cost volume is concatenated or averaged with the CLIP cost) to ensure implementation parity.","section":"Section 4.4.2, Table 3"},{"comment":"The mechanism assumes that frozen-VFM pairwise structural biases transfer across the large resolution and geographic shifts in the benchmark. Table 4 shows the method tolerates different VFMs, but it does not directly validate this transferability. I would like to see either a per-dataset analysis of A_bias behavior (e.g., intra-class attention ratio conditioned on resolution gap) or a failure-case analysis for cases where structural guidance could suppress valid CLIP evidence, particularly DeepGlobe under FLAIR where GeoSeg-OV trails OVRS by 1.2 mIoU. This concern is secondary to the statistical one, but it would substantiate the claimed mechanism.","section":"Sections 3.2.1 and 4.4.5"}],"minor_comments":[{"comment":"The caption spells 'A VTM' where 'AVTM' is meant; please correct the typo.","section":"Table 3 caption"},{"comment":"The paper reports parameter counts without specifying whether frozen encoders are included; please state whether these are trainable parameters or total model parameters.","section":"Table 6"},{"comment":"The intra-class attention ratio is reported without a precise definition or the evaluation protocol; please specify how it is computed and over which images or splits it is averaged.","section":"Figure 3(b)"},{"comment":"Boundary IoU is reported as a single number; a short definition or citation would help readers reproduce the metric.","section":"Section 4.4.6"},{"comment":"The abstract states 'large-scale zero-shot case study' as a demonstration of generalization, but the SinoLC-1 comparison is qualitative; the text already notes this, but the abstract and highlights could be read as implying a quantitative evaluation, so please qualify the wording.","section":"Abstract and Highlights"}],"recommendation":"major_revision","confidential_remarks":"The statistical-robustness concern is the only substantive blocker; the paper is otherwise well-executed and the benchmark is a useful contribution. I would be comfortable with acceptance after a revision that adds multi-seed experiments and significance tests for the main comparisons."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague—\n\nHere’s the short version: the paper’s core idea—decoupling an auxiliary VFM from the visual-text matching space and injecting it as a pairwise structural bias into cost aggregation—is genuinely new relative to GSNet and RSKT-Seg, and the controlled AVTM-vs-SGA comparison (Table 3) is the cleanest evidence for it in the OVSS remote sensing literature. The HRLC benchmark is also a real contribution, and the experimental controls (same CLIP backbone, splits, iterations, batch sizes) are better than most.\n\nThe thing to read is Table 3. It holds everything fixed except whether auxiliary features become a second cost volume or a spatial attention bias, and SGA wins by +1.1 to +2.0 mIoU depending on encoder and training set. That is a fair test, and it directly supports the design principle the paper claims.\n\nThe weak spot is statistical. Every number in Tables 1–5 comes from one run. No error bars, no seeds, no significance tests. The headline gain over OVRS/RSKT-Seg is +2.5/+2.7 average mIoU, but per-dataset differences swing from +5.7 (Potsdam) to −1.2 (DeepGlobe under FLAIR). Single-seed segmentation runs commonly vary by 1–2 mIoU, so the headline SOTA claim is not yet shown to be outside noise. The paper is honest about DeepGlobe, which helps, but it does not supply the distribution-over-runs evidence that would settle it. This is a load-bearing issue for the SOTA claim, though not for the design principle—Table 3 would still support that if the underlying numbers were stable.\n\nMinor issues: the abstract says the large-scale case study “demonstrates” generalization, but that comparison is qualitative with no target annotations; it’s a nice demo, not a demonstration. And the transferability premise—that frozen VFM structural features are domain-robust—is plausible but tested only via three VFMs; no failure cases or analysis of when the bias might hurt.\n\nBottom line: this paper is for remote sensing OVSS researchers and anyone building cross-dataset benchmarks. It deserves a serious referee, but the referee should ask for multi-seed runs and a paired significance test before the headline numbers are trusted. I’d conditionally accept in spirit and require the runs.","headline":"Genuinely new VFM-as-structural-prior design with a clean controlled test, but headline gains come from single runs and need multi-seed corroboration.","tokens_in":27336,"tokens_out":2204,"would_cite":true,"duration_ms":19619,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GeoSeg-OV claims that frozen vision foundation model features should steer cost aggregation as structural biases rather than join the visual–text matching process, and reports state-of-the-art cross-dataset open-vocabulary remote sensing…","keywords":["open-vocabulary semantic segmentation","remote sensing","cross-dataset generalization","structural guidance","cost aggregation","vision foundation models","land cover mapping","CLIP"],"falsifier":"Take the HRLC benchmark and replace the structural bias with a constant zero bias, and separately with a bias from a VFM trained only on natural images with no overhead data. If the zero-bias or mismatched-bias variant keeps most of the +2.5/+2.7 average mIoU gain, or if the gain is concentrated in one encoder rather than the paradigm, the transferability premise fails. A second check: compute the intra-class attention ratio per evaluation dataset; if datasets with the largest resolution shift (Potsdam, Vaihingen) do not show higher intra-class attention under SGA than AVTM, then the structural bias is not doing the claimed region-coherence work.","tokens_in":26222,"feed_emoji":"🛰️","tokens_out":11415,"duration_ms":90459,"temperature":0.7,"pith_summary":"GeoSeg-OV is trying to establish that auxiliary vision foundation models (VFMs) should not be used as additional visual–text matchers in open-vocabulary remote sensing segmentation. Instead, their features should stay outside the matching process and act as structural guidance—pairwise biases that control where CLIP matching evidence is allowed to spread during cost aggregation. On a new global benchmark of seven high-resolution land-cover datasets spanning six continents, the resulting system reports 44.2 and 41.6 average mIoU under two training settings, beating the previous best methods by +2.5 and +2.7 mIoU, with the largest gains on datasets whose resolution or geography differs most from training. The broader point is that structure transfers across geographic and resolution shifts better than appearance, and that aggregation guidance is a more robust use of auxiliary encoders than additional matching signals.","feed_headline":"GeoSeg-OV lifts open-vocabulary land-cover mapping by +2.5 mIoU","feed_subtitle":"A frozen vision model steers where matching evidence flows, across seven datasets and six continents.","key_machinery":"The load-bearing object is the pairwise structural bias $A_{\\text{bias}}$, computed from frozen VFM features as $q_s k_s^\\top / \\sqrt{d_h}$ and added to the base attention logits in SGA's spatial aggregation. Because the values propagated are still cost tokens, the VFM shapes where evidence flows without ever being scored against text, which is exactly the decoupling the paper argues for. The second mechanism is CAD's decoder-conditioned gate $\\sigma(P_l \\odot U_l)$, where $P_l$ is a projection of the category-pooled decoder context and $U_l$ is either the semantic or structural guidance stream; this decides how much of each guidance to inject at each decoding scale. The input to both is the multi-rotation cost volume $C_{\\text{clip}}$, built by encoding the image under four rotations, counter-rotating the features, and taking dense cosine similarity with CLIP text embeddings.","core_discovery":"GeoSeg-OV's central claim is that the Auxiliary Visual–Text Matching (AVTM) paradigm—correlating features from a second encoder with CLIP text embeddings to build an extra cost volume—is the wrong use of auxiliary VFMs under geospatial domain shift. Auxiliary encoders lack explicit visual–text alignment, so their similarity scores can disagree with CLIP's matching space; their structure-sensitive representations (region coherence, boundaries, layout) are what degrades least when resolution, platform, and geography change. The system therefore constructs the cost volume from CLIP alone, encoding the image under four rotations for orientation robustness, and uses a frozen VFM to compute a pairwise structural bias $A_{\\text{bias}}$ that is added to the attention logits of Structure-Guided Aggregation (SGA), so cost-token propagation stays within coherent regions. Cost-Aware Decoding (CAD) then refines the semantic and structural guidance streams with gates conditioned on the current decoder state. On the HRLC benchmark, GeoSeg-OV reports 44.2 and 41.6 average mIoU under FLAIR and OpenEarthMap training, +2.5 and +2.7 over the best prior methods, and a controlled comparison shows SGA beats AVTM with the same frozen encoder by +2.0/+1.2 mIoU with Depth Anything V2 and +1.3/+1.1 with RSIB-DINO.","pith_inferences":["The decoupling principle suggests a general recipe for open-vocabulary segmentation under domain shift: keep visual–text matching in a single aligned space and use auxiliary encoders to modulate aggregation. The same recipe could be tested in medical imaging or underwater scenes, where appearance shifts but spatial layout persists.","The paper reports, but does not recommend, the efficiency trade-off: dropping the four-rotation encoding saves 0.11 s/iteration at a cost of about 1.3 mIoU. For large-area zero-shot mapping, the no-rotation variant may be the practical sweet spot.","Because the benchmark's categories are mostly land-cover types with strong spatial structure, an untested boundary of the claim is category sets defined by subtle texture or color, such as crop species; there the structural bias may add little or suppress discriminating appearance evidence.","A testable prediction follows: if structural guidance is category-agnostic, then adding a VFM-derived bias to the aggregation stage of any frozen-CLIP open-vocabulary segmenter should improve cross-domain transfer, not just the CAT-Seg-style pipeline studied here."],"forward_implications":["With both training settings on the HRLC benchmark, GeoSeg-OV reports the top average mIoU (44.2 and 41.6), and its biggest margins come on Potsdam, the evaluation dataset with the largest resolution gap from training.","Using the same frozen encoder, SGA outperforms AVTM by +2.0/+1.2 mIoU with Depth Anything V2 and +1.3/+1.1 with RSIB-DINO, showing the gain comes from the utilization paradigm rather than the specific VFM.","Unseen-class IoU improves by +5.0 on FLAIR→OpenEarthMap and +2.3 on OpenEarthMap→FLAIR over the second-best method, with developed space rising from 1.2 (GSNet) to 24.0 IoU, so structural guidance helps classes never named in training.","Boundary IoU rises to 26.2% and 26.6%, beating the second-best method by +1.7 and +2.6 percentage points, so the structural bias tightens boundaries rather than merely smoothing predictions.","Removing multi-rotation encoding still leaves SGA+CAD at 42.9/40.4 average mIoU, above every prior method, separating the structural-guidance contribution from the rotation augmentation."],"supporting_citations":[{"why":"Supplies CLIP, whose image–text alignment and frozen encoders define the matching space and backbone used to build the cost volume.","marker":"Radford et al. (2021)"},{"why":"Defines the cost aggregation paradigm and the CAT-Seg baseline that all compared remote sensing methods, including GeoSeg-OV, build on.","marker":"Cho et al. (2024)"},{"why":"Introduces the four-rotation cost construction that GeoSeg-OV adapts, and is the strongest prior baseline under FLAIR training.","marker":"Cao et al. (2025)"},{"why":"GSNet, the representative AVTM method pairing CLIP with a DINO-based remote sensing backbone; its paradigm is the one GeoSeg-OV argues against.","marker":"Ye et al. (2025)"},{"why":"RSKT-Seg, an AVTM method with multiple domain-adapted encoders and a strong prior baseline under both training settings.","marker":"Li et al. (2026a)"},{"why":"DINOv2, one of the three frozen auxiliary VFMs tested as structural guidance in the framework generality analysis.","marker":"Oquab et al. (2023)"},{"why":"Depth Anything V2, the default frozen auxiliary VFM whose multi-scale features produce the structural bias and guidance.","marker":"Yang et al. (2024)"},{"why":"SAM 2.1, the third frozen auxiliary VFM tested, supporting the claim that the paradigm is encoder-agnostic.","marker":"Ravi et al. (2024)"},{"why":"FLAIR, one of the two training-source datasets in the HRLC benchmark.","marker":"Garioud et al. (2023)"},{"why":"OpenEarthMap, the other training-source dataset in the HRLC benchmark.","marker":"Xia et al. (2023)"}],"fun_headline_variants":["GeoSeg-OV: +2.5 mIoU via structure, not extra matching","GeoSeg-OV: structure beats extra matching for open-vocab land-cover","GeoSeg-OV: structure-guided aggregation lifts open-vocab mIoU by 2.5","GeoSeg-OV: +2.5 mIoU on global HRLC with structure guidance","Structure, not similarity: GeoSeg-OV lifts open-vocab mIoU"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's gains rest on the premise that a frozen vision foundation model's sense of where object boundaries lie and which regions cohere transfers across large resolution and geographic shifts more reliably than appearance statistics do, so steering the matching evidence with that sense helps rather than hurts.","fun_headline_variants_meta":{"raw":{"variants":["GeoSeg-OV: +2.5 mIoU via structure, not extra matching","GeoSeg-OV: structure beats extra matching for open-vocab land-cover","GeoSeg-OV: structure-guided aggregation lifts open-vocab mIoU by 2.5","GeoSeg-OV: +2.5 mIoU on global HRLC with structure guidance","Structure, not similarity: GeoSeg-OV lifts open-vocab mIoU"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001628,"raw_usage":{"total_tokens":6578,"prompt_tokens":1154,"completion_tokens":5424,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":770,"completion_tokens_details":{"reasoning_tokens":5301}},"tokens_in":770,"tokens_out":5424,"duration_ms":34750,"temperature":1.0,"reasoning_tokens":5301,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:20:37.546481+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the HRLC benchmark and replace the structural bias with a constant zero bias, and separately with a bias from a VFM trained only on natural images with no overhead data. If the zero-bias or mismatched-bias variant keeps most of the +2.5/+2.7 average mIoU gain, or if the gain is concentrated in one encoder rather than the paradigm, the transferability premise fails. A second check: compute the intra-class attention ratio per evaluation dataset; if datasets with the largest resolution shift (Potsdam, Vaihingen) do not show higher intra-class attention under SGA than AVTM, then the structural bias is not doing the claimed region-coherence work.","supporting_citations":[],"review_version":1}