pith. sign in

arxiv: 2606.28920 · v1 · pith:7GT3OS7Ynew · submitted 2026-06-27 · 💻 cs.CV

ExACT: Exemplar-Driven Calibrated Refinement for Training-Free Visual Grounding in Remote Sensing Images

Zixiao Zhang , Lingling Li , Pei He , Xu Liu , Licheng Jiao This is my paper

Pith reviewed 2026-06-30 09:44 UTC · model grok-4.3

classification 💻 cs.CV
keywords remote sensing visual groundingtraining-free frameworkone-shot visual promptingmultimodal large language modelspixel-level localizationsegment anything modelcross-modal priors
0
0 comments X

The pith

A single visual example can correct language-based localization errors in satellite images without retraining the model.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper aims to show that providing one visual example of the target object allows a system to refine the rough location guesses from large multimodal models into accurate pixel boundaries in complex aerial scenes. This matters because current models struggle with the difference between vague language descriptions and detailed image features, leading to wrong placements amid clutter. By using the example to pull out matching visual details and then refining prompts for a segmentation model, the approach achieves better results than methods that require training or weak supervision. If successful, it means open-vocabulary grounding in remote sensing can work off-the-shelf with minimal input.

Core claim

The framework uses a vision exemplar-based calibrator to extract fine-grained visual correspondences from the given one-shot exemplar to rectify rough cross-modal priors from frozen multimodal models, suppressing background artifacts, and then a structure-aware refiner employs iterative merge-and-select clustering to consolidate into high-quality prompts for the segment anything model to achieve precise pixel-level predictions.

What carries the argument

The one-shot visual prompting mechanism that supplies discriminative structural guidance to bridge the gap between linguistic semantics and fine-grained visual cues.

If this is right

  • Precise pixel-level localization becomes possible in cluttered remote sensing scenes using only natural language plus one visual example.
  • The approach suppresses background artifacts and outlines target boundaries more accurately than prior training-free methods.
  • It works directly with frozen multimodal models and requires no domain-specific tuning or additional training data.
  • High-quality positive and negative geometric prompts can be generated automatically to guide segmentation models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same exemplar-driven calibration might reduce the need for task-specific fine-tuning in other grounding settings where language and vision misalign.
  • Providing multiple exemplars could be tested to handle greater scene variability without changing the core training-free design.
  • The method's reliance on exemplar quality suggests experiments that vary exemplar similarity to quantify robustness limits.

Load-bearing premise

The one-shot visual exemplar supplies sufficient fine-grained correspondences to reliably rectify cross-modal priors from frozen models and suppress background artifacts in cluttered scenes without introducing new errors.

What would settle it

Testing the method on remote sensing images where the provided exemplar shares no fine-grained visual features with the described target due to changes in angle or lighting, and measuring whether localization accuracy falls below that of baseline multimodal models.

Figures

Figures reproduced from arXiv: 2606.28920 by Licheng Jiao, Lingling Li, Pei He, Xu Liu, Zixiao Zhang.

Figure 1
Figure 1. Figure 1: The motivation of our ExACT. (a) Relying solely [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Effectiveness of our exemplar-driven calibration. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: ExACT comprises two core modules: (1) The Vision Exemplar-Based Calibrator extracts dense feature-level correlations [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Illustration of Structure-Aware Refiner. (a) Text-driven baselines yield noisy proposals that mislead SAM. (b) SAR [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative results on the RRSIS-D dataset. The [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Step-by-step visualization of the internal localization priors. ExACT progressively rectifies noisy attention (a-b) and [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 1
Figure 1. Figure 1: (a) Ablation on semantic peak number 𝐾. (b) Abla￾tion on binarization threshold 𝛼. C More Ablation Studies on RRSIS-D Ablation of Various Geometric Prompts [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Failure cases of our approach on RRSIS-D dataset. [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
read the original abstract

Remote sensing visual grounding (RSVG) aims to locate specific objects in high-resolution RS imagery using free-form natural language descriptions. While recent advances in multimodal large language models (MLLMs) show great potential for such open-vocabulary RSVG, their training-free adaptation is hindered by the modality gap between abstract linguistic semantics and fine-grained visual cues. In cluttered RS scenes, this gap inevitably causes severe localization drift. To bridge this gap, we propose Exemplar-driven Calibrated Refinement (ExACT), a novel training-free framework driven by a one-shot visual prompting mechanism to explicitly provide discriminative structural guidance for precise pixel-level localization. Specifically, we propose a Vision Exemplar-based Calibrator (VEC) that extracts fine-grained visual correspondences from the given exemplar to rectify the rough cross-modal priors from frozen MLLMs, effectively suppressing background artifacts and accurately outlining target boundaries. Subsequently, a Structure-Aware Refiner (SAR) employs an iterative merge-and-select clustering strategy to consolidate the calibrated priors into high-quality positive and negative geometric prompts. These prompts then guide the Segment Anything Model (SAM) to achieve precise pixel-level predictions. Extensive experiments confirm the superiority of ExACT over existing training-free and weakly-supervised methods.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes ExACT, a training-free framework for remote sensing visual grounding (RSVG) that uses a one-shot visual exemplar via Vision Exemplar-based Calibrator (VEC) to rectify cross-modal priors from frozen MLLMs, followed by Structure-Aware Refiner (SAR) that generates geometric prompts to guide SAM for pixel-level localization in cluttered scenes. It claims superiority over training-free and weakly-supervised baselines based on extensive experiments.

Significance. If the central claims hold, ExACT offers a practical training-free alternative for open-vocabulary RSVG by explicitly bridging the modality gap with exemplar-driven calibration, which could reduce reliance on large annotated datasets in remote sensing. The integration of one-shot prompting with SAM and the iterative clustering in SAR are notable strengths; the manuscript also provides empirical validation through comparisons, which strengthens its contribution if the results are robust.

major comments (2)
  1. [§3.2] §3.2 (VEC description): The claim that fine-grained correspondences from a single exemplar reliably rectify MLLM priors and suppress background artifacts rests on the assumption of high visual similarity and robust matching; however, no ablation or analysis is provided on performance under typical RS variations (scale, viewpoint, illumination), which directly bears on whether the rectification step avoids introducing new errors as asserted.
  2. [§4] §4 (experiments): While superiority is claimed over baselines, the absence of error analysis, failure-case breakdowns, or quantitative metrics on correspondence accuracy in VEC makes it difficult to verify that the method consistently outperforms without domain-specific tuning, which is load-bearing for the training-free advantage.
minor comments (2)
  1. [§3] Notation for VEC and SAR components could be clarified with a single overview diagram early in §3 to aid readability.
  2. Some figure captions (e.g., qualitative results) lack explicit mention of the exemplar used, which would help readers trace the one-shot mechanism.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We address the two major comments point by point below, agreeing where additional analysis would strengthen the manuscript.

read point-by-point responses

  1. Referee: [§3.2] §3.2 (VEC description): The claim that fine-grained correspondences from a single exemplar reliably rectify MLLM priors and suppress background artifacts rests on the assumption of high visual similarity and robust matching; however, no ablation or analysis is provided on performance under typical RS variations (scale, viewpoint, illumination), which directly bears on whether the rectification step avoids introducing new errors as asserted.

    Authors: We agree that explicit analysis of robustness to scale, viewpoint, and illumination variations would better substantiate the VEC rectification claims. The current manuscript relies on the one-shot exemplar being visually aligned with the query, but does not quantify sensitivity to these factors. We will add a dedicated ablation study evaluating VEC performance under controlled RS variations in the revised version. revision: yes

  2. Referee: [§4] §4 (experiments): While superiority is claimed over baselines, the absence of error analysis, failure-case breakdowns, or quantitative metrics on correspondence accuracy in VEC makes it difficult to verify that the method consistently outperforms without domain-specific tuning, which is load-bearing for the training-free advantage.

    Authors: We acknowledge that error analysis, failure-case breakdowns, and quantitative correspondence metrics for VEC would provide stronger evidence for consistent outperformance. The existing experiments demonstrate overall superiority on standard benchmarks, but lack these granular diagnostics. In revision we will incorporate quantitative correspondence accuracy metrics, selected failure cases, and error breakdowns to address this. revision: yes

Circularity Check

0 steps flagged

No circularity detected in derivation

full rationale

The paper introduces ExACT as a training-free framework using one-shot visual prompting via VEC to rectify MLLM priors and SAR for SAM guidance. No equations, fitted parameters, or predictions are described in the abstract or provided text. No self-citations appear as load-bearing justifications for uniqueness or ansatzes. The derivation chain relies on external components (frozen MLLMs, SAM) and proposed algorithmic steps without reducing outputs to self-defined inputs by construction. This is a standard non-finding for a methods paper whose claims are evaluated via experiments rather than internal algebraic closure.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract provides no explicit free parameters, axioms, or invented entities; assessment is limited because only the abstract is available.

pith-pipeline@v0.9.1-grok · 5758 in / 1120 out tokens · 23060 ms · 2026-06-30T09:44:57.722339+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

43 extracted references · 8 canonical work pages · 1 internal anchor

  1. [1]

    Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. 2021. Emerging properties in self-supervised vision transformers. InProceedings of the IEEE/CVF international conference on computer vision. 9650–9660

    2021

  2. [2]

    Jinming Chai, Licheng Jiao, Xiaoqiang Lu, Lingling Li, Fang Liu, Long Sun, Xu Liu, Wenping Ma, and Weibin Li. 2026. Like Human Rethinking: Contour Transformer AutoRegression for Referring Remote Sensing Interpretation.IEEE Transactions on Pattern Analysis and Machine Intelligence(2026)

    2026

  3. [3]

    Shengxin Chen, Gen Luo, Yiyi Zhou, Xiaoshuai Sun, Guannan Jiang, and Ron- grong Ji. 2024. Querymatch: A query-based contrastive learning framework for weakly supervised visual grounding. InProceedings of the 32nd ACM International Conference on Multimedia. 4177–4186

    2024

  4. [4]

    Fabio Dell’Acqua and Paolo Gamba. 2012. Remote sensing and earthquake damage assessment: Experiences, limits, and perspectives.Proc. IEEE100, 10 (2012), 2876–2890

    2012

  5. [5]

    Zhe Dong, Yuzhe Sun, Tianzhu Liu, Wangmeng Zuo, and Yanfeng Gu. 2024. Cross-modal bidirectional interaction model for referring remote sensing image segmentation.arXiv preprint arXiv:2410.08613(2024)

    work page arXiv 2024

  6. [6]

    Xin Gu, Heng Fan, Yan Huang, Tiejian Luo, and Libo Zhang. 2024. Context-guided spatio-temporal video grounding. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 18330–18339

    2024

  7. [7]

    Seil Kang, Jinyeong Kim, Junhyeok Kim, and Seong Jae Hwang. 2025. Your large vision-language model only needs a few attention heads for visual grounding. In Proceedings of the Computer Vision and Pattern Recognition Conference. 9339–9350

    2025

  8. [8]

    Dongwon Kim, Namyup Kim, Cuiling Lan, and Suha Kwak. 2023. Shatter and gather: Learning referring image segmentation with text supervision. InProceed- ings of the IEEE/CVF International Conference on Computer Vision. 15547–15557

    2023

  9. [9]

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al

  10. [10]

    InProceedings of the IEEE/CVF international conference on computer vision

    Segment anything. InProceedings of the IEEE/CVF international conference on computer vision. 4015–4026

  11. [11]

    Kartik Kuckreja, Muhammad Sohail Danish, Muzammal Naseer, Abhijit Das, Salman Khan, and Fahad Shahbaz Khan. 2024. Geochat: Grounded large vision- language model for remote sensing. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 27831–27840

    2024

  12. [12]

    Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia

  13. [13]

    InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Lisa: Reasoning segmentation via large language model. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9579–9589

  14. [14]

    Ke Li, Di Wang, Ting Wang, Fuyu Dong, Yiming Zhang, Luyao Zhang, Xiangyu Wang, Shaofeng Li, and Quan Wang. 2025. RSVG-ZeroOV: Exploring a Training- Free Framework for Zero-Shot Open-Vocabulary Visual Grounding in Remote Sensing Images.arXiv preprint arXiv:2509.18711(2025)

    work page arXiv 2025

  15. [15]

    Ke Li, Di Wang, Haojie Xu, Haodi Zhong, and Cong Wang. 2024. Language- guided progressive attention for visual grounding in remote sensing images.IEEE Transactions on Geoscience and Remote Sensing(2024)

    2024

  16. [16]

    Ke Li, Di Wang, Yongshan Zhu, Ting Wang, Weiping Ni, Tao Lei, Quan Wang, and Xinbo Gao. 2026. Training-Free Open-Vocabulary Visual Grounding for Remote Sensing Images and Videos.arXiv preprint arXiv:2606.16124(2026)

    work page arXiv 2026

  17. [17]

    Biao Liu, Xu Liu, Lingling Li, Licheng Jiao, Fang Liu, Xinyu Sun, and Youlin Huang

  18. [18]

    InProceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence

    Language-guided hybrid representation learning for visual grounding on remote sensing images. InProceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. 1557–1566

  19. [19]

    Fang Liu, Yuhao Liu, Yuqiu Kong, Ke Xu, Lihe Zhang, Baocai Yin, Gerhard Hancke, and Rynson Lau. 2023. Referring image segmentation using text supervision. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 22124– 22134

    2023

  20. [20]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual in- struction tuning.Advances in neural information processing systems36 (2023), 34892–34916

    2023

  21. [21]

    Sihan Liu, Yiwei Ma, Xiaoqing Zhang, Haowei Wang, Jiayi Ji, Xiaoshuai Sun, and Rongrong Ji. 2024. Rotated multi-scale interaction network for referring remote sensing image segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 26658–26668

    2024

  22. [22]

    Yufei Liu, Haoke Xiao, Jiaxing Chai, Yongcun Zhang, Rong Wang, Zijie Meng, and Zhiming Luo. 2025. SynPo: Boosting Training-Free Few-Shot Medical Seg- mentation via High-Quality Negative Prompts. InInternational Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 594–603

    2025

  23. [23]

    Yang Liu, Muzhi Zhu, Hengtao Li, Hao Chen, Xinlong Wang, and Chunhua Shen. 2023. Matcher: Segment anything with one shot using all-purpose feature matching.arXiv preprint arXiv:2305.13310(2023)

    work page arXiv 2023

  24. [24]

    Xiaoqiang Lu, Long Sun, Lingling Li, Licheng Jiao, Yuting Yang, Zhongjian Huang, Jinming Chai, Xu Liu, Fang Liu, Wenping Ma, et al . 2025. RRSECS: Referring remote sensing expression comprehension and segmentation.IEEE Geoscience and Remote Sensing Magazine(2025)

    2025

  25. [25]

    Dilxat Muhtar, Zhenshi Li, Feng Gu, Xueliang Zhang, and Pengfeng Xiao. 2024. Lhrs-bot: Empowering remote sensing with vgi-enhanced large multimodal language model. InEuropean Conference on Computer Vision. Springer, 440–457

    2024

  26. [26]

    Chao Pang, Xingxing Weng, Jiang Wu, Jiayu Li, Yi Liu, Jiaxing Sun, Weijia Li, Shuai Wang, Litong Feng, Gui-Song Xia, et al. 2025. Vhm: Versatile and honest vision language model for remote sensing image analysis. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 6381–6388

    2025

  27. [27]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10684–10695

    2022

  28. [28]

    Kosuke Sakurai, Ryotaro Shimizu, and Masayuki Goto. 2025. Vision and Lan- guage Reference Prompt into SAM for Few-shot Segmentation.arXiv preprint arXiv:2502.00719(2025)

    work page arXiv 2025

  29. [29]

    Haozhan Shen, Tiancheng Zhao, Mingwei Zhu, and Jianwei Yin. 2024. Groundvlp: Harnessing zero-shot visual grounding from vision-language pre-training and open-vocabulary object detection. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 4766–4775

    2024

  30. [30]

    Yuxi Sun, Shanshan Feng, Xutao Li, Yunming Ye, Jian Kang, and Xu Huang

  31. [31]

    InProceedings of the 30th ACM International Conference on Multimedia

    Visual grounding in remote sensing images. InProceedings of the 30th ACM International Conference on Multimedia. 404–412

  32. [32]

    Lv Tang, Peng-Tao Jiang, Haoke Xiao, and Bo Li. 2025. Towards training-free open-world segmentation via image prompt foundation models.International Journal of Computer Vision133, 1 (2025), 1–15

    2025

  33. [33]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. 2024. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191(2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  34. [34]

    Marie Weiss, Frédéric Jacob, and Grgory Duveiller. 2020. Remote sensing for agricultural applications: A meta-review.Remote sensing of environment236 (2020), 111402

    2020

  35. [35]

    Size Wu, Sheng Jin, Wenwei Zhang, Lumin Xu, Wentao Liu, Wei Li, and Chen Change Loy. 2025. F-lmm: Grounding frozen large multimodal models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion. 24710–24721

    2025

  36. [36]

    Linhui Xiao, Xiaoshan Yang, Xiangyuan Lan, Yaowei Wang, and Changsheng Xu

  37. [37]

    Towards visual grounding: A survey.IEEE Transactions on Pattern Analysis and Machine Intelligence(2025)

    2025

  38. [38]

    Yang Zhan, Zhitong Xiong, and Yuan Yuan. 2023. Rsvg: Exploring data and models for visual grounding on remote sensing data.IEEE Transactions on Geoscience and Remote Sensing61 (2023), 1–13

    2023

  39. [39]

    Anqi Zhang, Guangyu Gao, Jianbo Jiao, Chi Liu, and Yunchao Wei. 2024. Bridge the points: Graph-based few-shot segment anything semantically.Advances in Neural Information Processing Systems37 (2024), 33232–33261

    2024

  40. [40]

    Jingyi Zhang, Jiaxing Huang, Sheng Jin, and Shijian Lu. 2024. Vision-language models for vision tasks: A survey.IEEE Transactions on Pattern Analysis and Machine Intelligence(2024)

    2024

  41. [41]

    Renrui Zhang, Zhengkai Jiang, Ziyu Guo, Shilin Yan, Junting Pan, Xianzheng Ma, Hao Dong, Peng Gao, and Hongsheng Li. 2023. Personalize segment anything model with one shot.arXiv preprint arXiv:2305.03048(2023)

    work page arXiv 2023

  42. [42]

    Yue Zhou, Mengcheng Lan, Xiang Li, Litong Feng, Yiping Ke, Xue Jiang, Qingyun Li, Xue Yang, and Wayne Zhang. 2024. Geoground: A unified large vision-language model for remote sensing visual grounding.arXiv preprint arXiv:2411.11904(2024)

    work page arXiv 2024

  43. [43]

    The oval ground track field

    Zhe Zhu, Yuyu Zhou, Karen C Seto, Eleanor C Stokes, Chengbin Deng, Steward TA Pickett, and Hannes Taubenböck. 2019. Understanding an urbanizing planet: Strategic directions for remote sensing.Remote Sensing of Environment228 (2019), 164–182. Conference’17, July 2017, Washington, DC, USA Zhang et al. ExACT: Exemplar-Driven Calibrated Refinement for Trainin...

    2019