{"id":"7cffdfb3-9612-4717-a3d7-1e9cfaadeca8","arxiv_id":"2608.10801","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"For small-scale PV segmentation with SAM3, hybrid text-plus-box prompting outperforms text-only and box-only prompting, and most gains require only a few hundred labeled samples.","lead":"This paper tests three ways of guiding a vision-language segmentation model, SAM3, to find small solar panels in aerial images: text descriptions, bounding boxes, and both together. It finds that spatial boxes help far more than text, and combining both is the most accurate and stable, using only a few hundred labeled examples.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Prompt type is confounded with which encoders are unfrozen (Sec. III-F): hybrid has two unfrozen encoders, geometric/text only one, so the reported hierarchy may reflect adaptation capacity rather than guidance modality.","rationale":"The reader's weakest_assumption is exactly the confound I identify: prompting strategy is varied together with which modules are unfrozen, so the observed hierarchy could reflect adaptation capacity or optimization dynamics rather than the guidance modality. This is the most load-bearing concern because the paper's headline conclusions are causal statements about prompt type. The concrete test I propose directly de-aliases the two factors by crossing prompt type with unfrozen-module configuration; a single comparison within the hybrid condition (hybrid + geometry-only unfreezing vs the original geometric-only setting) would determine whether the semantic contribution survives when the number of trainable encoders is held constant. I do not see a stronger internal-validity threat: the oracle-box design is acknowledged, the prompt-selection procedure is separated from the evaluation data, and the external datasets support consistency of the ranking, but none of those address the module-unfreezing confound. The verdict should remain CONDITIONAL, since the paper's empirical contribution is substantial but this confound must be resolved before the complementarity claim is accepted as stated.","tokens_in":23046,"tokens_out":2583,"duration_ms":26133,"concrete_test":"Run a 2x2 crossing of prompt type (text-only, box-only, hybrid) with unfrozen modules (text encoder, geometry encoder, both, neither), keeping the mask decoder unfrozen and all hyperparameters fixed. The minimal decisive check is to fine-tune the hybrid prompt with only the geometry encoder unfrozen and compare against the original geometric-only condition (geometry encoder unfrozen). If F1 is statistically indistinguishable from geometric-only, the original hybrid advantage comes from unfreezing both encoders rather than from semantic guidance; if it remains at hybrid level, the semantic contribution survives the control. Repeat for textual-only with the geometry encoder unfrozen to test whether textual performance is limited by the prompt or by the adapters.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III-F defines each prompting pipeline by a different fine-tuning configuration: textual-only unfreezes the text encoder, geometric-only unfreezes the geometry encoder, and hybrid unfreezes both, while the mask decoder is always unfrozen. Thus 'prompting strategy' is perfectly aliased with the set of trainable components. The central conclusion (hybrid > geometric > textual, and semantic and spatial guidance are complementary) depends on attributing performance differences to the prompt modality. However, the hybrid condition is the only one with two trainable encoders; if the geometry encoder is easier to adapt, or if jointly unfreezing both encoders improves optimization dynamics, the hybrid advantage is an artifact of adaptation capacity rather than evidence that semantic guidance adds value beyond spatial guidance. The paper also reports single runs per setting, so the magnitude of differences relative to run-to-run variability is unknown, but the structural confound is the more load-bearing issue because it threatens the causal interpretation of 'prompting strategy emerged as a primary factor' (Conclusion 1) and the complementarity claim (Conclusion 4).","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper systematically evaluates SAM3 (a promptable vision-language foundation model) for pixel-level segmentation of small-scale residential photovoltaic (PV) installations in remote sensing imagery, comparing three prompting strategies: textual only, geometric only (ground-truth bounding boxes), and hybrid (text + boxes). Experiments vary supervision scale (200-700 samples plus zero-shot), training strategy (independent per-image fine-tuning versus sequential transfer learning), spatial resolution (0.25, 0.145, 0.132 m/pixel), and cross-image variability across seven acquisition years in the Northern Negev, with additional validation on three public datasets (France Google, France IGN, Queens NY). The paper reports that prompting strategy is the dominant factor governing model behavior: textual prompting performs worst and is most sensitive to supervision, resolution, and imaging conditions; geometric prompting is substantially better; hybrid prompting achieves the highest and most stable performance, indicating complementarity between semantic and spatial guidance. The authors also report strong data efficiency, with most gains from a few hundred samples, and limited transfer-learning benefit except for textual prompting. The central claim is that the choice of prompt modality is a primary control knob when adapting SAM3 to small-object PV segmentation, and that semantic and spatial guidance are complementary.","tokens_in":23231,"tokens_out":4099,"duration_ms":43347,"significance":"If the central claims hold, this would be a valuable and much-needed systematic benchmark of prompting modalities for a state-of-the-art promptable segmentation foundation model in a challenging, imbalanced remote sensing setting. The study is thoughtfully designed in several respects: held-out validation and test partitions that are fixed across experiments, geographically non-overlapping splits, prompt selection performed on a spatially disjoint subset before the main experiments, external datasets spanning three resolution levels, and appropriate statistical models (fractional logit with HC3 standard errors, Type-II ANOVA with partial eta-squared). The data-efficiency finding (a few hundred annotated samples suffice) has practical importance for mapping PV in off-grid regions. The authors also disclose the use of ground-truth-derived boxes and provide a preliminary detector feasibility experiment, which is an honest limitation statement.","major_comments":[{"comment":"The three prompting pipelines are not just different prompt inputs; they are also different fine-tuning configurations. Textual-only unfreezes the text encoder, geometric-only unfreezes the geometry encoder, and hybrid unfreezes both, with the mask decoder always unfrozen and the vision encoder and DETR components frozen. Thus 'prompting strategy' is perfectly aliased with the set of trainable components. The conclusion that hybrid prompting outperforms geometric prompting because semantic and spatial guidance are complementary (Conclusions 1 and 4) presupposes that the hybrid advantage comes from the text+box input, but it could equally come from having two unfrozen encoders, or from more favorable optimization dynamics when both encoders are jointly adapted. Likewise, the geometric-versus-textual gap could reflect the geometry encoder being easier to adapt than the text encoder rather than a property of the guidance modality. To support the headline claims, this aliasing must be broken. A feasible design is to evaluate each prompt type under multiple fine-tuning configurations (e.g., all pipelines with both encoders unfrozen, or hybrid with only one encoder unfrozen), or to include a matched-capacity control that varies the number of unfrozen components independently of the prompt input. As written, the reported hierarchy is a joint effect of prompt input and trainable components, not an isolated effect of prompting strategy.","section":"III-F"},{"comment":"All experiments appear to be single runs per condition: no seed variation, confidence intervals, or repeated trials are reported. The paper makes load-bearing robustness claims, such as 'lowest sensitivity to changes in supervision scale, training strategy, spatial resolution, and cross-image variability' (Conclusion 4), and 'consistently' observed hierarchies, but with one run per cell the within-condition stochasticity is unmeasured. The Friedman and Wilcoxon tests treat each pooled setting as an observation, which does not quantify run-to-run variability. This is especially important for the reported decline of textual-prompting Recall and IoU beyond 400 training samples (Section IV-C2), which could be within-run noise. I request that the authors add at least three independent seeds for representative conditions (or otherwise quantify variability, e.g., bootstrap confidence intervals on the pooled distributions), and that they explicitly report the number of runs used in each figure and table. Without this, the stability and sensitivity conclusions are not statistically grounded.","section":"IV (Figures 3-5; Conclusions 2-4)"}],"minor_comments":[{"comment":"There are several typographical and formatting issues: 'Isreal' in the author affiliations, inconsistent spelling of the first author's name between the main text ('Blushtein-Livnon') and reference [4] ('Bluestein-Livnon'), and broken LaTeX artifacts such as 'ANOV A' and 'Wald Type-II' spacing in Section IV-D2 and the supplementary tables.","section":"Throughout"},{"comment":"The caption states that predictions correspond to the 'best-performing model configuration' for each prompting strategy and image, but it does not specify how this configuration was selected (e.g., highest validation F1, or an oracle over test sets). Please clarify the selection criterion so that the qualitative examples cannot be perceived as cherry-picked.","section":"Figure 6 caption"},{"comment":"The prompt selection procedure evaluates only M=5 candidate noun phrases. The choice of M is not justified, and although the paper notes that exhaustive search was not the goal, a short discussion of how M affects the reliability of the selected 'solar panels' prompt would strengthen the methodology.","section":"III-D"},{"comment":"All predictions are thresholded at 0.5, but the paper does not report any sensitivity analysis to this threshold. A brief check of thresholds in the range 0.3-0.7 would help confirm that the reported F1/IoU hierarchy is not an artifact of the threshold choice.","section":"III-G"},{"comment":"No code or data availability statement is provided. Given that the primary dataset is private, a public release of the code and the defined train/validation/test splits (or a detailed data description) would materially improve reproducibility for the community.","section":"Code/data availability"}],"recommendation":"major_revision","confidential_remarks":"This is a well-executed empirical study with a transparent experimental design, but the confounding between prompting strategy and fine-tuning configuration is a serious structural issue that directly threatens the paper's central claims. The single-run nature of the experiments compounds the problem. I believe the manuscript can be revised to address these concerns, but only with additional experiments (or, at minimum, a matched-capacity control). If the authors can break the aliasing, the paper would be a strong contribution to the practical use of promptable foundation models in remote sensing. There is no obvious novelty-disclosure concern; the prior letter [5] is disclosed and the present manuscript clearly extends it."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: this is a well-scoped, honestly reported empirical study of how SAM3 behaves under text, box, and hybrid prompting for small PV segmentation, and it will be useful to anyone working on promptable FMs in remote sensing. But the headline conclusion—that prompting strategy is the dominant factor and that semantic and spatial guidance complement each other—is weaker than the paper claims, because prompting strategy is perfectly aliased with which encoders are unfrozen.\n\nThe genuinely new thing: the literature has separate text-guided and geometry-guided SAM adaptations, but not a controlled within-model comparison on the same task. This paper does that systematically: held-out validation/test, prompt selection on a spatially disjoint subset, three supervision scales, two training regimes, three resolutions, and external datasets from France and Queens. The data is real (multi-temporal aerial imagery over ~1,150 km² of off-grid Bedouin settlements, three-expert majority-vote annotations), and the statistical treatment is more careful than most applied RS papers—fractional logit, Type-II ANOVA, Holm corrections. The hierarchy (hybrid > geometric > textual) is consistent across datasets and robust to many changes. That consistency is a genuine empirical contribution.\n\nThe soft spot is structural. In Section III-F, textual-only unfreezes the text encoder, geometric-only unfreezes the geometry encoder, and hybrid unfreezes both; the mask decoder is always unfrozen. So the hybrid condition differs from the other two in two ways: it gets both prompt types and it gets two trainable encoders. The paper never varies the unfrozen modules within a fixed prompt type, so the observed hierarchy could reflect adaptation capacity or optimization dynamics rather than the guidance modality itself. The complementarity conclusion (Conclusion 4) and the 'primary factor' language (Conclusion 1) are overstated. The paper's own limitations section does not mention this confound. It is fixable—a couple of ablation runs where you unfreeze both encoders under each prompt type, or freeze one, would settle it—but as it stands, the causal reading doesn't follow.\n\nMinor: all results are single-run point estimates with no seeds or error bars; the statistically significant differences are across settings, not across runs. And the geometric/hybrid results use oracle bounding boxes, which is fine for isolating prompt contribution but means the practical claims about automated pipelines depend on detector quality; the authors acknowledge this and include preliminary YOLO results, which is good.\n\nWho is this for? Applied remote sensing researchers who want practical guidance on prompting SAM3 for small-object segmentation. It deserves a serious referee, but the confound needs to be addressed before the central claims can be accepted.","headline":"Careful empirical study of SAM3 prompting for small PV segmentation, but the headline hierarchy is confounded with which encoders get fine-tuned.","tokens_in":23751,"tokens_out":2942,"would_cite":false,"duration_ms":29932,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Prompting strategy, not training data volume, is what governs how SAM3 adapts to small solar panel segmentation.","keywords":["foundation models","SAM3","prompting strategies","PV segmentation","remote sensing","small object segmentation","semantic guidance","spatial guidance"],"falsifier":"Run a crossover experiment in which every prompting condition is fine-tuned with the same set of trainable modules—for example, unfreeze both encoders under text-only and geometric-only prompts, and unfreeze only the relevant encoder under hybrid—and check whether the hybrid-over-geometric-over-text ranking and the complementarity effect survive; if they do not, the claim that prompting strategy is the dominant factor would be refuted.","tokens_in":22848,"feed_emoji":"☀️","tokens_out":7369,"duration_ms":63980,"temperature":0.7,"pith_summary":"This paper asks which way of telling a segmentation foundation model what to look for—a text description, a bounding box, or both—matters most when the target is small, sparse photovoltaic panels in aerial imagery. The authors fine-tune SAM3 under all three prompting strategies while varying supervision scale, transfer learning, spatial resolution, and imaging conditions, and they measure the resulting hierarchy in accuracy and stability. Their central claim is that prompting strategy is the primary factor governing model behavior: hybrid prompts perform best, geometric prompts are intermediate, and text-only prompts are worst and least stable. The finding matters because it shows where adaptation effort should go when building PV maps for off-grid regions, and it indicates that most accuracy gains arrive with only a few hundred annotated samples.","feed_headline":"Hybrid prompts beat text-only for tiny solar panel mapping","feed_subtitle":"Prompt modality, not data volume, drives accuracy and robustness for off-grid solar mapping, a SAM3 study finds","key_machinery":"The load-bearing object is SAM3's modular promptable architecture, which encodes text and geometric prompts into a shared embedding space through a unified perception encoder and fuses them with image features via cross-attention; a presence head estimates whether the concept appears, a query-based transformer decoder localizes instances, and a mask decoder produces pixel masks. The paper exploits this modularity to build three fine-tuning pipelines—text-only, with the text encoder trainable; geometric-only, with the geometry encoder trainable; and hybrid, with both trainable, while the mask decoder is always trainable. This setup, together with ground-truth-derived bounding boxes as a controlled spatial prompt, is what lets the authors attribute performance differences to prompting strategy rather than to detector error.","core_discovery":"Working with SAM3, a promptable vision-language foundation model, the paper reports a consistent and statistically significant performance hierarchy across all imaging conditions and datasets: hybrid prompting (text plus bounding box) achieves the highest F1, IoU, and Recall; geometric prompting (bounding box only) is intermediate; textual prompting (noun phrase only) is lowest and most variable. The paper argues that semantic and spatial guidance are complementary rather than interchangeable: text conveys target identity, boxes supply localization, spatial guidance mainly reduces localization uncertainty, and semantic guidance helps recover pixels that would otherwise be missed. It also reports that spatial guidance substantially reduces sensitivity to spatial resolution and cross-image variability, that performance plateaus after roughly 200–300 training samples for spatially guided strategies, and that transfer learning matters mainly for text-only prompting at low supervision. The same hierarchy reproduces on three external aerial datasets, supporting the generality of the findings.","pith_inferences":["If prompting strategy is genuinely the dominant control knob, then prompt-level design—such as learning or optimizing text descriptors and box proposals jointly—could be a more cost-effective investment than architectural modification for this class of small-object remote sensing tasks.","The paper's use of ground-truth boxes leaves open whether a fully automated pipeline with detector-generated boxes preserves the hybrid advantage; testing that chain would determine operational feasibility.","The authors' explanation for the text-only decline at higher supervision (over-specialization to dominant appearances) is plausible but untested; a direct test would compare worst-case and minority-appearance recall as training grows.","The same controlled comparison could be applied to other small and imbalanced targets, and to other promptable foundation models, to see whether the hybrid-over-geometric-over-text hierarchy is a SAM3 property or a general one."],"forward_implications":["For SAM3-based PV mapping, the prompting pipeline should be treated as a design decision on par with model architecture, because it changes accuracy, robustness, and data requirements.","Text-only prompting is the least reliable option for small, sparse targets and needs the most supervision; applications that cannot provide spatial cues should expect lower recall and higher sensitivity to image quality.","Hybrid prompting delivers the highest accuracy and the lowest sensitivity to resolution, image variability, and supervision scale, making it the preferred configuration for heterogeneous multi-campaign imagery.","A few hundred labeled samples are enough to reach most of the achievable performance for spatially guided prompting, so annotation budgets can be kept small.","Transfer learning adds little once explicit spatial guidance is available, so its value is mostly confined to text-driven adaptation under limited supervision."],"supporting_citations":[{"why":"Defines SAM3's concept-conditioned, promptable architecture and training objective used throughout all experiments.","marker":"[8]"},{"why":"Introduces the promptable foundation-model segmentation paradigm that motivates the study.","marker":"[32]"},{"why":"Documents the difficulty of language-guided open-vocabulary segmentation in remote sensing, which the text-only results confirm.","marker":"[36]"},{"why":"Characterizes the small-object and target-background imbalance challenges of residential PV segmentation that define the task.","marker":"[21]"},{"why":"Supplies the multi-expert majority-voting annotation protocol behind the gold-standard masks.","marker":"[6]"},{"why":"Provides the France aerial datasets used for cross-dataset consistency testing.","marker":"[30]"},{"why":"Provides the Queens, New York aerial dataset used for cross-dataset consistency testing.","marker":"[17]"},{"why":"Reports prior prompting-strategy differences that motivated the systematic comparison.","marker":"[5]"}],"fun_headline_variants":["Hybrid prompts beat text-only for tiny solar panel mapping","Spatial guidance boosts SAM3 for small-scale PV segmentation","Prompt type, not data volume, drives SAM3 solar segmentation","Hybrid prompts are most robust for tiny off-grid solar detection","Complementary text+box prompts beat text alone for PV mapping"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison treats prompting strategy as if it were separable from which parts of the model get fine-tuned, but text-only, geometric-only, and hybrid each come with a different fixed set of trainable encoders, so the observed ranking could partly reflect adaptation capacity or optimization dynamics rather than the guidance modality itself.","fun_headline_variants_meta":{"raw":{"variants":["Hybrid prompts beat text-only for tiny solar panel mapping","Spatial guidance boosts SAM3 for small-scale PV segmentation","Prompt type, not data volume, drives SAM3 solar segmentation","Hybrid prompts are most robust for tiny off-grid solar detection","Complementary text+box prompts beat text alone for PV mapping"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001068,"raw_usage":{"total_tokens":4501,"prompt_tokens":999,"completion_tokens":3502,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":615,"completion_tokens_details":{"reasoning_tokens":3417}},"tokens_in":615,"tokens_out":3502,"duration_ms":22265,"temperature":1.0,"reasoning_tokens":3417,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T17:06:14.011380+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a crossover experiment in which every prompting condition is fine-tuned with the same set of trainable modules—for example, unfreeze both encoders under text-only and geometric-only prompts, and unfreeze only the relevant encoder under hybrid—and check whether the hybrid-over-geometric-over-text ranking and the complementarity effect survive; if they do not, the claim that prompting strategy is the dominant factor would be refuted.","supporting_citations":[{"cited_title":"Segment anything, in: Proceedings of the IEEE/CVF international conference on computer vision, pp","cited_arxiv_id":null,"evidence_quote":"Introduces the promptable foundation-model segmentation paradigm that motivates the study."},{"cited_title":"Segearth-ov: Towards training- free open-vocabulary segmentation for remote sensing images, in: Proceedings of the Computer Vision and Pattern Recognition Conference, pp","cited_arxiv_id":null,"evidence_quote":"Documents the difficulty of language-guided open-vocabulary segmentation in remote sensing, which the text-only results confirm."},{"cited_title":"Transpv: Refining photovoltaic panel detection accuracy through a vision transformer- based deep learning model","cited_arxiv_id":null,"evidence_quote":"Characterizes the small-object and target-background imbalance challenges of residential PV segmentation that define the task."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Queens, New York aerial dataset used for cross-dataset consistency testing."},{"cited_title":"On the effectiveness of textual prompting with lightweight fine-tuning for sam3 remote sensing segmentation","cited_arxiv_id":null,"evidence_quote":"Reports prior prompting-strategy differences that motivated the systematic comparison."}],"review_version":1}