Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

RGBT-GroundBench: Visual Grounding Beyond RGB in Complex Real-World Scenarios

T0 review · 4 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper introduces RGBT-Ground, the first large-scale paired RGB–thermal visual grounding benchmark, and shows that adding thermal imagery consistently recovers accuracy lost in low light, fog, small-object, and heavy-occlusion conditions

desk verdict First RGB-Thermal grounding benchmark is a real contribution, but annotation validation and split-count inconsistencies need fixing before I'd trust the numbers. read the letter →

arxiv 2512.24561 v2 pith:ESF3XO7G submitted 2025-12-31 cs.CV

Tianyi Zhao , Jiawen Xi , Linhui Xiao , Junnan Li , Xue Yang , Maoxun Yuan , Xingxing Wei This is my paper · ORCID
classification cs.CV
keywords visualgroundingRGB-Thermalbenchmarkmulti-modalreferringexpressionslow-lightrobustnessmodalityadaptationCLIP
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RGBT-GroundBench claims that visual grounding, the task of localizing an object from a natural-language description, is not robustly solved by RGB-only benchmarks because they omit the low-light, adverse-weather, and small-object conditions that break appearance cues. To prove this, the paper builds the first large-scale paired RGB-Thermal grounding benchmark, with 21,535 aligned image pairs and 38,760 instances labeled for scene, lighting, weather, object size, and occlusion. Across 11 adapted models and over 60 configurations, it shows a consistent difficulty ladder—well-lit > low-light, normal-size > small, clear > occluded—and that RGB+TIR fusion beats either uni-modal input in every subset. The proposed baseline, RGBT-VGNet, tops all splits. A sympathetic reader would care because safety-critical applications like autonomous driving and embodied robotics need grounding that works when RGB fails, and this benchmark gives the community a way to measure and close that robustness gap.

What carries the argument

The aligned RGB-TIR image pair is the load-bearing object: thermal imagery is illumination-invariant, so it supplies stable contours exactly where RGB noise and darkness erase appearance. The paper's baseline makes this work via Asymmetric Modality Adaptation (AMA), which gives the thermal encoder a higher-rank LoRA than the RGB encoder, and Language-Aware Visual Synergy (LAVS), which uses the referring expression as a query to weight which modality's features to trust. The evaluation protocol itself—unified data pipeline, three modality settings, identical training—is what converts the dataset into a benchmark rather than a collection.

What would settle it

Take a random sample of 1,000 test instances from RGBT-Ground, have multiple human annotators independently write referring expressions and assign lighting, weather, and occlusion labels, then re-evaluate the trained RGB-only and RGB+TIR models on that human-annotated subset. If the human-based difficulty ordering differs materially from the machine-label-based ordering, or if the RGB-only to RGB+TIR gap shrinks by more than a third, the benchmark's conclusions do not hold.

Watch

Extended reading notes

Core claim

The paper's central claim is that RGB-thermal visual grounding is feasible and consistently more robust than RGB-only grounding under exactly the conditions that matter for real deployment. It constructs RGBT-Ground by re-annotating three public paired RGB-TIR detection datasets, using a large vision-language model to generate referring expressions and multi-level environmental labels, then verifying a random sample. The evaluation shows that RGB-only models degrade sharply in weak light, fog, and heavy occlusion, while RGB+TIR models recover most of the gap. The proposed baseline, RGBT-VGNet, uses asymmetric LoRA adaptation (a higher-rank adapter for the thermal branch to close the RGB-pret

Load-bearing premise

The referring expressions and all lighting, weather, occlusion, and size labels were generated by a large vision-language model and only hierarchically random-sampled for human verification, with no reported inter-annotator agreement or human-checked fraction; if those labels are noisy or biased, the reported difficulty orderings and the size of the thermal advantage could be artifacts.

Editorial extensions

If this is right

  • If correct, RGB-only visual grounding should no longer be considered sufficient for safety-critical deployment; paired thermal input becomes the default recommendation.
  • The released benchmark provides a controlled testbed for future work, with splits isolating lighting, weather, size, and occlusion as independent difficulty axes.
  • The asymmetric LoRA finding suggests a general principle for adapting pre-trained RGB vision encoders to any non-RGB modality.
  • The roughly 10% average improvement from RGB+TIR over the best uni-modal model across all subsets makes multi-modal grounding a clear target for future research.
  • The baseline's code and checkpoints enable direct comparison, so progress can be measured on the same splits.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The difficulty ordering may partly reflect the annotation model's tendency to generate more complex expressions in hard scenes; a human-annotation study on a subset could separate true perceptual difficulty from annotation complexity.
  • The benchmark's value would be strengthened by a per-condition analysis—already present in the appendix—that isolates when thermal alone beats RGB alone; that result is not the paper's headline claim but is directly extractable.
  • The same unified framework could be extended to other paired modalities (e.g., RGB-depth) and to downstream tasks like referring segmentation and VQA, as the appendix suggests.
  • A practical next step is to train a per-region reliability router that decides when to trust RGB versus TIR; the baseline does this implicitly through language-guided attention but never makes it explicit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces RGBT-Ground, a large-scale RGB–Thermal visual grounding benchmark assembled from three existing RGB-T detection datasets (FLIR, M3FD, MFAD), with 21,535 aligned RGB-TIR pairs and 38,760 instances. Each instance is annotated with a referring expression generated by the Qwen-VL LVLM, a bounding box, and multi-level labels for scene type, weather, illumination, object size, and occlusion. The authors also present a unified evaluation framework (RGBT-VG) that adapts several existing RGB-only grounding models to RGB-only, TIR-only, and RGB+TIR inputs, along with a new baseline (RGBT-VGNet) using asymmetric LoRA adaptation and language-aware cross-modal fusion. Extensive experiments across three sub-datasets, multiple splits, and over 60 configurations show that RGB+TIR fusion generally improves accuracy, that low-light and small-object subsets are hardest, and that the proposed baseline achieves the highest Acc@0.5 across all three test subsets.

Significance. If the benchmark is released with reliable annotations and a usable evaluation protocol, it would fill a genuine gap: current visual grounding benchmarks are almost entirely RGB-only and rarely include adverse illumination, weather, or small/occluded objects. The paper provides a plausible resource with multi-level annotations and a broad evaluation of 11 models, which is valuable for future robustness research. The proposed RGBT-VGNet baseline is simple and well-motivated, and the ablation study supports its two main components. The main uncertainty is not the experimental protocol but the quality of the LVLM-generated annotations, since the benchmark's usefulness depends on the referring expressions and environmental labels being accurate and discriminative.

major comments (4)
  1. [§3.2, Appendix D, Table 3] The annotation pipeline generates all referring expressions and all scene/weather/illumination/occlusion labels with the Qwen-VL API, but the described 'hierarchical random sampling strategy' is never quantified. The paper reports no fraction of human-checked instances, no sampling rule, no correction statistics, and no inter-annotator agreement. Table 3 only shows that the resulting distribution of lighting/weather labels is physically plausible, which does not establish per-instance quality. Since the central claim is that RGBT-Ground is a reliable benchmark, this missing evidence is load-bearing. Please report the number/percentage of instances verified by humans, the agreement between annotators and the LVLM, and the types and rates of corrections made.
  2. [Table 2] The reported test-subset counts are internally inconsistent. For RefFLIR, testA+testB+testC = 837+640+986 = 2,463, which exceeds the listed test total of 2,104; for RefM3FD, 1,232+1,094+1,848 = 4,174 > 3,776; for RefMFAD, 789+2,452+2,550 = 5,791 > 4,244. Either the testA/B/C subsets overlap (which would need to be stated explicitly and handled in evaluation), or the counts are erroneous. This must be clarified because every reported Acc@0.5 number for testA/B/C depends on these definitions.
  3. [Abstract, §5.2, Table 4] The abstract states that 'LoRA-based models are more robust in complex scenes', but the comparison does not isolate the effect of LoRA. The two models that use LoRA (HiVG and OneRef, both from the same group) differ from the others in backbone, architecture, and training details, so any robustness difference could be due to these factors rather than the LoRA mechanism. To support this claim, the paper would need a controlled comparison that varies only the adaptation method (e.g., full fine-tuning vs. LoRA on the same architecture). Please either provide such an ablation or soften the claim to 'models with hierarchical fine-grained modulation are more robust' or similar.
  4. [Abstract, §1, §6] The paper repeatedly states that 'resources, annotations, code, checkpoints, and evaluation scripts have been publicly released' but the manuscript contains no URL, repository name, hash, or any concrete pointer. For a benchmark paper, data and code availability is central to reproducibility and to the community's ability to use the benchmark. Please include a stable link and, ideally, a checksum/hash for the annotations, or a clear statement about the release timeline and access procedure.
minor comments (5)
  1. [§5.1] The learning rate is reported as '10e-4', which is nonstandard and ambiguous (10×10⁻⁴ = 1e-3). Please use '1e-3' or '1e-4' consistently.
  2. [Throughout] Several typos and inconsistent notations appear: 'Acc@50' is sometimes used instead of 'Acc@0.5'; 'RGB-Thermal' vs 'RGB-TIR' vs 'RGBT' are used inconsistently; appendix figures contain 'ZTY APPENDIX' watermarks that should be removed; and the appendix text contains typos such as 'omparatively', 'dark-colered', 'Penson', and 'aprked'.
  3. [§2.2] The related work section discusses RGB-D grounding but does not mention any prior RGB-T or infrared visual grounding benchmark. If any such work exists, it should be cited and the 'first' claim in §1 should be qualified accordingly. If none exists, a brief sentence noting this absence would strengthen the novelty claim.
  4. [Table 4/5] In several table rows, values are formatted inconsistently (e.g., '63.7291.52' and '91.5260.97' in the RefMFAD column for HiVG-L). Please ensure proper LaTeX/CSV formatting in the final version, as these are currently unreadable.
  5. [Appendix A] The appendix claims 'consistent ranking across datasets' but reports no variance or statistical analysis across the multiple runs. If only one run was performed, please state this explicitly and discuss the potential for small-sample variability on the tiny scene/weather subsets (e.g., Table 12 shows many cells with only a handful of instances).

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation; benchmark evaluation is external and self-contained.

full rationale

No circularity found. The paper's central contributions are a dataset, an evaluation protocol, and a baseline model; none of these are derived from fitted constants or from self-citations that already contain the conclusion. The Acc@0.5 results in Tables 4 and 5 are measured on fixed benchmark annotations, so they are external comparisons rather than predictions that reduce to their inputs. The baseline RGBT-VGNet is an architectural contribution whose performance is empirically evaluated, not a quantity obtained by construction from the benchmark definition. The main weakness is annotation-quality evidence: referring expressions and scene/weather/lighting/occlusion labels are generated by Qwen-VL and only statistically validated in Table 3, with no reported inter-annotator agreement or human-checked fraction. That is a correctness and reliability risk for the benchmark, but it is not circularity: the evaluation procedure is not equivalent to the label-generation input. Self-citations exist (CLIP-VG, HiVG, OneRef as baselines; the hierarchical design of HiVG adopted in the baseline), but they are not load-bearing in the sense of proving the paper's claims solely through unpublished or self-referential authority. No uniqueness theorem, ansatz, or fitted parameter is disguised as a prediction. Hence the appropriate circularity score is 0.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central benchmark claims rest on the accuracy of LVLM-generated annotations, the alignment of source RGB-TIR data, and curation decisions; no new scientific entities are introduced. The only hand-set quantity central to the benchmark is the 1% small-object area threshold.

free parameters (2)
  • Small-object area threshold (SS/NS boundary) = 0.01 (1% of image area)
    Hand-chosen cutoff in Appendix D.3 defines Small vs Normal Size; directly determines TestC composition and the reported small-object difficulty.
  • Asymmetric LoRA rank pair (r_v, r_t) and scaling factor alpha = not reported
    Eq. (4) sets r_v <= r_t and alpha to adapt CLIP to TIR; exact values are not given, yet they determine RGBT-VGNet's SOTA margin. Treated as tunable hyperparameters.
assumptions (5)
  • domain assumption Qwen-VL-generated expressions and attribute labels are accurate after random human verification.
    Captioning pipeline uses Qwen-VL for referring expressions and scene/weather/lighting/occlusion labels; human verification is only hierarchical random sampling and no accuracy metrics are reported (§3.2, Appendix D).
  • domain assumption RGB and TIR images in the source datasets are sufficiently spatially aligned for grounding.
    The benchmark relies on 'spatially aligned or weakly aligned' pairs from FLIR, M3FD, and MFAD; residual misalignment would corrupt box labels (§3.1, §3.2).
  • domain assumption Source object bounding boxes are correct ground truth.
    The referring expressions and evaluation boxes inherit the source detection datasets' annotations without independent re-verification (§3.2).
  • domain assumption Acc@0.5 is a meaningful evaluation metric for visual grounding.
    The paper uses IoU > 0.5 as the sole metric; this is standard but does not capture fine localization or semantic ambiguity (§5.1).
  • domain assumption The curation/filtering process preserves 'complex real-world' representativeness.
    Excluding extremely small targets, removing long-tail categories, and selecting only the largest instance per category biases the distribution; the paper assumes the remaining data still represents challenging real-world scenarios (§3.2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of RGBT-GroundBench: Visual Grounding Beyond RGB in Complex Real-World Scenarios." pith.science (2026). https://pith.science/paper/ESF3XO7G

@misc{pith2026251224561,
  author       = {Pith},
  title        = {Pith review of: RGBT-GroundBench: Visual Grounding Beyond RGB in Complex Real-World Scenarios},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ESF3XO7G}},
  note         = {Machine review of arXiv:2512.24561}
}
read the original abstract

Visual grounding (VG) localizes target objects in an image from natural-language expressions. In real-world perception, RGB cues often degrade under low illumination and adverse weather, making visual grounding substantially more challenging. However, existing VG benchmarks are largely RGB-only and provide limited, structured coverage of such conditions, hindering systematic robustness evaluation and cross-spectral comparison. We present RGBT-GroundBench, the first large-scale benchmark for RGB-Thermal (TIR) visual grounding in complex environments. It contains over 40K images (21,535 RGB-TIR pairs) and 38,760 object instances with referring expressions, bounding boxes, and fine-grained annotations at three levels: scene types, environmental conditions (illumination and weather), and object properties (size and occlusion). As a benchmark suite, RGBT-GroundBench provides not only curated RGB-TIR grounding annotations but also a unified evaluation protocol supporting RGB-only, TIR-only, and RGB+TIR inputs. Under this protocol, we benchmark 11 representative VG models across diverse scenes and environmental conditions. Our results show that grounding accuracy is strongly correlated with scene complexity, LoRA-based models are more robust in complex scenes, and low-illumination conditions cause significant performance degradation that has been rarely explored. Guided by these observations, we introduce RGBT-VGNet, a simple and reproducible reference baseline under the unified protocol, featuring Asymmetric Modality Adaptation, Language-Aware Visual Synergy, and Tri-Prior Fusion for reliability-aware RGB-TIR integration. Resources, annotations, code, checkpoints, and evaluation scripts have been publicly released.

Figures

Figures reproduced from arXiv: 2512.24561 by the authors.

Figure 1
Figure 1. Comparative overview of the RGBT-Ground benchmark and its evaluation of representative grounding method. (a) Compara [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview and characteristic analysis of the proposed RGBT-Ground benchmark. (a–c) present dataset distributions and corre [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Illustration of the RGBT-Ground data preparation [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: The architecture of the proposed multi-modal RGBT Visual Grounding baseline method (RGBT-VGNet). [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparison of multi-modal RGBT visual grounding results on RGBT-Ground. The three rows correspond to [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Additional qualitative comparison of multi-modal RGBT visual grounding results on RGBT-Ground across diverse real-world [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Qualitative comparison of RGB-only, TIR-only and RGB-TIR grounding results (Figure [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Qualitative comparison of RGB-only, TIR-only and RGB-TIR grounding results (Figure [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Qualitative comparison of RGB-only, TIR-only and RGB-TIR grounding results (Figure [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SpectraDINO: Modality-Conditioned Adaptation of RGB Vision Foundation Models Across Infrared Bands

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    SpectraDINO extends DINOv2 with lightweight per-modality adapters and staged distillation to handle NIR, SWIR, and LWIR in one backbone, but its SWIR gain is weakened by using the evaluation dataset for pretraining.

  2. OmniSch: A Multimodal PCB Schematic Benchmark For Structured Diagram Visual Reasoning

    cs.CV 2026-03 conditional novelty 7.0 of 10

    OmniSch is the first benchmark exposing gaps in LMMs for PCB schematic visual grounding, topology-to-graph parsing, geometric weighting, and tool-augmented reasoning.

Reference graph

Works this paper leans on

70 extracted references · 1 linked inside Pith · cited by 2 Pith papers

  1. [1]

    Qwen-vl: A versatile vision-language model for un- derstanding, localization, text reading, and beyond, 2023

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for un- derstanding, localization, text reading, and beyond, 2023. 5

    2023

  2. [2]

    Uniter: Universal image-text representation learning

    Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. Uniter: Universal image-text representation learning. InECCV, pages 104–120. Springer, 2020. 2

    2020

  3. [3]

    Cops-ref: A new dataset and task on composi- tional referring expression comprehension

    Zhenfang Chen, Peng Wang, Lin Ma, Kwan-Yee K Wong, and Qi Wu. Cops-ref: A new dataset and task on composi- tional referring expression comprehension. InCVPR, pages 10086–10095, 2020. 1

    2020

  4. [4]

    Unit3d: A unified transformer for 3d dense captioning and visual grounding

    Zhenyu Chen, Ronghang Hu, Xinlei Chen, Matthias Nießner, and Angel X Chang. Unit3d: A unified transformer for 3d dense captioning and visual grounding. InICCV, pages 18109–18119, 2023. 3

    2023

  5. [5]

    Advancing visual grounding with scene knowl- edge: Benchmark and method

    Zhihong Chen, Ruifei Zhang, Yibing Song, Xiang Wan, and Guanbin Li. Advancing visual grounding with scene knowl- edge: Benchmark and method. InCVPR, pages 15039– 15049, 2023. 6

    2023

  6. [6]

    Transvg: End-to-end visual ground- ing with transformers

    Jiajun Deng, Zhengyuan Yang, Tianlang Chen, Wengang Zhou, and Houqiang Li. Transvg: End-to-end visual ground- ing with transformers. InICCV, pages 1769–1779, 2021. 2, 5, 7, 12, 22, 23, 24, 25, 26, 27

    2021

  7. [7]

    Bert: Pre-training of deep bidirectional trans- formers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional trans- formers for language understanding. InProceedings of the 2019 conference of the North American chapter of the asso- ciation for computational linguistics: human language tech- nologies, volume 1 (long and short papers), pages 4171– 4186, 2019. 2

    2019

  8. [8]

    D3t: Distinctive dual-domain teacher zigzagging across rgb- thermal gap for domain-adaptive object detection

    Dinh Phat Do, Taehoon Kim, Jaemin Na, Jiwon Kim, Keonho Lee, Kyunghwan Cho, and Wonjun Hwang. D3t: Distinctive dual-domain teacher zigzagging across rgb- thermal gap for domain-adaptive object detection. InCVPR, pages 23313–23322, 2024. 3

    2024

Show all 70 references
  1. [9]

    An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020. 2

    2010 arXiv

  2. [10]

    Large-scale adversarial training for vision- and-language representation learning.NeurIPS, 33:6616– 6628, 2020

    Zhe Gan, Yen-Chun Chen, Linjie Li, Chen Zhu, Yu Cheng, and Jingjing Liu. Large-scale adversarial training for vision- and-language representation learning.NeurIPS, 33:6616– 6628, 2020. 2

    2020

  3. [11]

    Room-and-object aware knowledge reasoning for remote embodied referring expression

    Chen Gao, Jinyu Chen, Si Liu, Luting Wang, Qiong Zhang, and Qi Wu. Room-and-object aware knowledge reasoning for remote embodied referring expression. InCVPR, pages 3064–3073, 2021. 1

    2021

  4. [12]

    The iapr tc-12 benchmark: A new eval- uation resource for visual information systems

    Michael Grubinger, Paul Clough, Henning M ¨uller, and Thomas Deselaers. The iapr tc-12 benchmark: A new eval- uation resource for visual information systems. InInterna- tional workshop ontoImage, 2006. 3

    2006

  5. [13]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 2

    2016

  6. [14]

    Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022. 6

    2022

  7. [15]

    Ei 2 det: Edge-guided illumination-aware interactive learning for visible-infrared object detection.TCSVT, 2025

    Ke Hu, Yudong He, Yuan Li, Jiayu Zhao, Song Chen, and Yi Kang. Ei 2 det: Edge-guided illumination-aware interactive learning for visible-infrared object detection.TCSVT, 2025. 3

    2025

  8. [16]

    Beyond one-to-one: Re- thinking the referring image segmentation

    Yutao Hu, Qixiong Wang, Wenqi Shao, Enze Xie, Zhenguo Li, Jungong Han, and Ping Luo. Beyond one-to-one: Re- thinking the referring image segmentation. InICCV, pages 4067–4077, 2023. 1

    2023

  9. [17]

    Referitgame: Referring to objects in pho- tographs of natural scenes

    Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, and Tamara Berg. Referitgame: Referring to objects in pho- tographs of natural scenes. InEMNLP, pages 787–798, 2014. 1, 3

    2014

  10. [18]

    mplug: Effective and efficient vision-language learning by cross-modal skip-connections

    Chenliang Li, Haiyang Xu, Junfeng Tian, Wei Wang, Ming Yan, Bin Bi, Jiabo Ye, He Chen, Guohai Xu, Zheng Cao, et al. mplug: Effective and efficient vision-language learning by cross-modal skip-connections. InEMNLP, pages 7241– 7259, 2022. 2

    2022

  11. [19]

    Rgb-t semantic segmentation with location, activation, and sharpening.TCSVT, 33(3):1223–1235, 2022

    Gongyang Li, Yike Wang, Zhi Liu, Xinpeng Zhang, and Dan Zeng. Rgb-t semantic segmentation with location, activation, and sharpening.TCSVT, 33(3):1223–1235, 2022. 3

    2022

  12. [20]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, pages 740–755. Springer, 2014. 3

    2014

  13. [21]

    Gres: Gen- eralized referring expression segmentation

    Chang Liu, Henghui Ding, and Xudong Jiang. Gres: Gen- eralized referring expression segmentation. InCVPR, pages 23592–23601, 2023. 1

    2023

  14. [22]

    Refer-it-in-rgbd: A bottom-up ap- proach for 3d visual grounding in rgbd images

    Haolin Liu, Anran Lin, Xiaoguang Han, Lei Yang, Yizhou Yu, and Shuguang Cui. Refer-it-in-rgbd: A bottom-up ap- proach for 3d visual grounding in rgbd images. InCVPR, pages 6032–6041, 2021. 3

    2021

  15. [23]

    Visual instruction tuning.NeurIPS, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.NeurIPS, 36:34892–34916, 2023. 1

    2023

  16. [24]

    Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection

    Jinyuan Liu, Xin Fan, Zhanbo Huang, Guanyao Wu, Risheng Liu, Wei Zhong, and Zhongxuan Luo. Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection. InCVPR, pages 5802–5811, 2022. 3

    2022

  17. [25]

    Cross3dvg: Cross-dataset 3d visual grounding on different rgb-d scans

    Taiki Miyanishi, Daichi Azuma, Shuhei Kurita, and Motoaki Kawanabe. Cross3dvg: Cross-dataset 3d visual grounding on different rgb-d scans. In2024 International Conference on 3D Vision (3DV), pages 717–727. IEEE, 2024. 3

    2024

  18. [26]

    Mod- eling context between objects for referring expression under- standing

    Varun K Nagaraja, Vlad I Morariu, and Larry S Davis. Mod- eling context between objects for referring expression under- standing. InECCV, pages 792–807. Springer, 2016. 1, 3

    2016

  19. [27]

    Flickr30k entities: Collecting region-to-phrase corre- spondences for richer image-to-sentence models

    Bryan A Plummer, Liwei Wang, Chris M Cervantes, Juan C Caicedo, Julia Hockenmaier, and Svetlana Lazeb- nik. Flickr30k entities: Collecting region-to-phrase corre- spondences for richer image-to-sentence models. InICCV, pages 2641–2649, 2015. 1, 3

    2015

  20. [28]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, 9 Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. InICLR, pages 8748–8763. PmLR, 2021. 1, 5

    2021

  21. [29]

    Dynamic mdetr: A dynamic multimodal transformer decoder for visual grounding.IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(2):1181–1198, 2023

    Fengyuan Shi, Ruopeng Gao, Weilin Huang, and Limin Wang. Dynamic mdetr: A dynamic multimodal transformer decoder for visual grounding.IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(2):1181–1198, 2023. 2, 5, 7, 12, 22, 23, 24, 25, 26, 27

    2023

  22. [30]

    Attention is all you need.NeurIPS, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.NeurIPS, 30, 2017. 2

    2017

  23. [31]

    Ofa: Unifying architectures, tasks, and modalities through a simple sequence-to-sequence learning framework

    Peng Wang, An Yang, Rui Men, Junyang Lin, Shuai Bai, Zhikang Li, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. Ofa: Unifying architectures, tasks, and modalities through a simple sequence-to-sequence learning framework. InICLR, pages 23318–23340. PMLR, 2022. 2

    2022

  24. [32]

    Image as a foreign language: Beit pretraining for vision and vision- language tasks

    Wenhui Wang, Hangbo Bao, Li Dong, Johan Bjorck, Zhil- iang Peng, Qiang Liu, Kriti Aggarwal, Owais Khan Mo- hammed, Saksham Singhal, Subhojit Som, et al. Image as a foreign language: Beit pretraining for vision and vision- language tasks. InCVPR, pages 19175–19186, 2023. 1

    2023

  25. [33]

    Clip-vg: Self-paced curriculum adapting of clip for visual grounding.TMM, 26:4334–4347,

    Linhui Xiao, Xiaoshan Yang, Fang Peng, Ming Yan, Yaowei Wang, and Changsheng Xu. Clip-vg: Self-paced curriculum adapting of clip for visual grounding.TMM, 26:4334–4347,

  26. [34]

    Towards visual grounding: A survey

    Linhui Xiao, Xiaoshan Yang, Xiangyuan Lan, Yaowei Wang, and Changsheng Xu. Towards visual grounding: A survey. arXiv preprint arXiv:2412.20206, 2024. 6

    2024

  27. [35]

    Hivg: Hierarchical multimodal fine- grained modulation for visual grounding

    Linhui Xiao, Xiaoshan Yang, Fang Peng, Yaowei Wang, and Changsheng Xu. Hivg: Hierarchical multimodal fine- grained modulation for visual grounding. InProceedings of the 32nd ACM International Conference on Multimedia, pages 5460–5469, 2024. 2, 5, 6, 7, 12, 22, 23, 24, 25, 26, 27

    2024

  28. [36]

    Oneref: Unified one-tower expression grounding and segmentation with mask referring modeling

    Linhui Xiao, Xiaoshan Yang, Fang Peng, Yaowei Wang, and Changsheng Xu. Oneref: Unified one-tower expression grounding and segmentation with mask referring modeling. NeurIPS, 37:139854–139885, 2024. 5, 7, 12, 22, 23, 24, 25, 26, 27

    2024

  29. [37]

    Described object detection: Liberating ob- ject detection with flexible expressions.NeurIPS, 36:79095– 79107, 2023

    Chi Xie, Zhao Zhang, Yixuan Wu, Feng Zhu, Rui Zhao, and Shuang Liang. Described object detection: Liberating ob- ject detection with flexible expressions.NeurIPS, 36:79095– 79107, 2023. 1

    2023

  30. [38]

    Mc-bench: A bench- mark for multi-context visual grounding in the era of mllms

    Yunqiu Xu, Linchao Zhu, and Yi Yang. Mc-bench: A bench- mark for multi-context visual grounding in the era of mllms. InICCV, pages 17675–17687, 2025. 1

    2025

  31. [39]

    Improving one-stage visual grounding by recursive sub-query construction

    Zhengyuan Yang, Tianlang Chen, Liwei Wang, and Jiebo Luo. Improving one-stage visual grounding by recursive sub-query construction. InECCV, pages 387–404. Springer,

  32. [40]

    Unitab: Unifying text and box outputs for grounded vision- language modeling

    Zhengyuan Yang, Zhe Gan, Jianfeng Wang, Xiaowei Hu, Faisal Ahmed, Zicheng Liu, Yumao Lu, and Lijuan Wang. Unitab: Unifying text and box outputs for grounded vision- language modeling. InECCV, pages 521–539. Springer,

  33. [41]

    Vi- sual grounding with multi-modal conditional adaptation

    Ruilin Yao, Shengwu Xiong, Yichen Zhao, and Yi Rong. Vi- sual grounding with multi-modal conditional adaptation. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 3877–3886, 2024. 5, 7, 12, 22, 23, 24, 25, 26, 27

    2024

  34. [42]

    Shifting more attention to visual backbone: Query-modulated refinement networks for end-to-end visual grounding

    Jiabo Ye, Junfeng Tian, Ming Yan, Xiaoshan Yang, Xuwu Wang, Ji Zhang, Liang He, and Xin Lin. Shifting more attention to visual backbone: Query-modulated refinement networks for end-to-end visual grounding. InCVPR, pages 15502–15512, 2022. 2

    2022

  35. [43]

    Modeling context in referring expres- sions

    Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. Modeling context in referring expres- sions. InECCV, pages 69–85. Springer, 2016. 1, 3

    2016

  36. [44]

    C 2former: Calibrated and complementary transformer for rgb-infrared object de- tection.TGRS, 62:1–12, 2024

    Maoxun Yuan and Xingxing Wei. C 2former: Calibrated and complementary transformer for rgb-infrared object de- tection.TGRS, 62:1–12, 2024. 3

    2024

  37. [45]

    Trans- lation, scale and rotation: Cross-modal alignment meets rgb-infrared vehicle detection

    Maoxun Yuan, Yinyan Wang, and Xingxing Wei. Trans- lation, scale and rotation: Cross-modal alignment meets rgb-infrared vehicle detection. InECCV, pages 509–525. Springer, 2022. 1

    2022

  38. [46]

    Improving rgb-infrared object detection with cascade alignment-guided transformer.Information Fusion, 105:102246, 2024

    Maoxun Yuan, Xiaorong Shi, Nan Wang, Yinyan Wang, and Xingxing Wei. Improving rgb-infrared object detection with cascade alignment-guided transformer.Information Fusion, 105:102246, 2024. 1

    2024

  39. [47]

    Unirgb-ir: A unified frame- work for visible-infrared semantic tasks via adapter tuning

    Maoxun Yuan, Bo Cui, Tianyi Zhao, Jiayi Wang, Shan Fu, Xue Yang, and Xingxing Wei. Unirgb-ir: A unified frame- work for visible-infrared semantic tasks via adapter tuning. InACM MM, pages 2409–2418, 2025. 3

    2025

  40. [48]

    Multispectral fusion for object detection with cyclic fuse-and-refine blocks

    Heng Zhang, Elisa Fromont, S ´ebastien Lefevre, and Bruno Avignon. Multispectral fusion for object detection with cyclic fuse-and-refine blocks. InICIP, pages 276–280. IEEE,

  41. [49]

    Abmdrnet: Adaptive-weighted bi-directional modality difference reduc- tion network for rgb-t semantic segmentation

    Qiang Zhang, Shenlu Zhao, Yongjiang Luo, Dingwen Zhang, Nianchang Huang, and Jungong Han. Abmdrnet: Adaptive-weighted bi-directional modality difference reduc- tion network for rgb-t semantic segmentation. InCVPR, pages 2633–2642, 2021. 3

    2021

  42. [50]

    Removal then selection: A coarse-to-fine fusion perspective for rgb-infrared object detection.arXiv e-prints, pages arXiv–2401, 2024

    Tianyi Zhao, Maoxun Yuan, Feng Jiang, Nan Wang, and Xingxing Wei. Removal then selection: A coarse-to-fine fusion perspective for rgb-infrared object detection.arXiv e-prints, pages arXiv–2401, 2024. 3

    2024

  43. [51]

    be- hind the truck

    Tianyi Zhao, Boyang Liu, Yanglei Gao, Yiming Sun, Maoxun Yuan, and Xingxing Wei. Rethinking multi-modal object detection from the perspective of mono-modality fea- ture learning. InICCV, pages 6364–6373, 2025. 3 10 Appendix A . More Evaluation Results on RGBT-Ground 12 A.1 . E...

    2025

  44. [53]

    Person riding a bike, partitially obscured by fog. 2. Bike with visible rear light positioned near road edge, partitially obscured by fog. 3. A black sedan parked on the left of the street under a streetlight. RGB-GT TIR-GT MV-TransVG MV-CLIP-VG RGBT-VGNetMV-D-MDETR MV-MMCA MV...

  45. [54]

    A small silver truck appears ahead near center-left lane in a tunnel. 2. A silver sedan drives ahead through the tunnel’s illuminated corridor

  46. [56]

    Penson in dark clothing stands near a white car, probably holding a phone. 2. A white SUV with a sleek design is parked on a cobblestone street near a residential area, facing forward

  47. [57]

    A dark-colered sedan aprked in a lot near other vhicles. 2. A person in blue shirt and dark pants walks near a residential area

  48. [58]

    A red double-decker bus with advertisements occupies part of the lane near traffic lights under skies. 2. A small white sedan drives along the road, positioned centrally among other vehicles under foggy skies. 3. A small figure wearing yellow clothing stands near the right edg...

  49. [61]

    A red double-decker bus with advertisements occupies part of the lane near traffic lights under skies. 2. A small white sedan drives along the road, positioned centrally among other vehicles under foggy skies. 3. A small figure wearing yellow clothing stands near the right edg...

  50. [64]

    A red double-decker bus with advertisements occupies part of the lane near traffic lights under skies. 2. A small white sedan drives along the road, positioned centrally among other vehicles under foggy skies. 3. A small figure wearing yellow clothing stands near the right edg...

  51. [65]

    Person in dark clothing stands near a library stair, facing away. 2. A dark SUV is parked far from the crowd near the library, facing forward

  52. [66]

    Penson in dark clothing stands near a white car, probably holding a phone. 2. A white SUV with a sleek design is parked on a cobblestone street near a residential area, facing forward. MV-TransVG MV-CLIP-VG RGBT-VGNet MV-D-MDETR MV-MMCA MV-OneRef-B MV-HiVG-B TransVG CLIP-VG GT...

  53. [67]

    near the stairs

    A red double-decker bus with advertisements occupies part of the lane near traffic lights under skies. 2. A small white sedan drives along the road, positioned centrally among other vehicles under foggy skies. 3. A small figure wearing yellow clothing stands near the right edg...

  54. [68]

    Very Weak Light: The image is mostly dark, but some features can be faintly seen

  55. [69]

    Weak Light: The image has low visibility, typical for dawn or dusk. 19

  56. [70]

    Normal Light: The image has normal daylight brightness

  57. [71]

    Please return only one number corresponding to the lighting condition: 0 (very_weak_light), 1 (weak_light), 2 (normal_light), or 3 ( strong_light)

    Strong Light: The image is brightly lit, typically from midday sunlight. Please return only one number corresponding to the lighting condition: 0 (very_weak_light), 1 (weak_light), 2 (normal_light), or 3 ( strong_light). Now, please begin generating the number D.3. Object-Leve...

  58. [72]

    Object Details: Please describe the object in detail, including but not limited to its appearance, color, shape, texture, size, and posture

  59. [73]

    Contextual Relationship: If there is any relationship between the objects within the image or between the object and the background (e.g., relative positioning, interaction), please reflect this in your description

  60. [74]

    Distinguishing Similar Objects: If there are multiple similar objects in the image, differentiate them by comparing their details such as color intensity, position, state, etc

  61. [75]

    small"ifsize_ratio < 0.01else

    Concise and Clear Language: Provide a single, concise sentence that captures the key features without breaking down into different aspects. The description should be rich in information yet simple for later data processing. Please generate only one few words of description for...

  62. [2023]

    2, 5, 7, 12, 22, 23, 24, 25, 26, 27

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.