Pith. sign in

REVIEW 3 major objections 5 minor 87 references

PropVG: End-to-End Proposal-Driven Visual Grounding with Multi-Granularity Discrimination

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A proposal-based visual grounding model can be trained end-to-end without any pre-trained detector, and the paper reports that it outperforms direct-referring and detector-dependent baselines across ten benchmarks.

desk verdict Solid empirical paper on end-to-end proposal-based grounding, but the proposal-recall ceiling is unexamined and the 'first detector-free' claim overlaps with UNINEXT. read the letter →

arxiv 2509.04833 v1 pith:TC7QQOLX submitted 2025-09-05 cs.CV cs.AI

Ming Dai , Wenxuan Cheng , Jiedong Zhuang , Jiang-jiang Liu , Hongshen Zhao , Zhenhua Feng , Wankou Yang This is my paper · ORCID
classification cs.CVcs.AI
keywords visualgroundingreferringexpressioncomprehensionsegmentationgeneralizedend-to-endproposal-basedframeworkcontrastivelearningtargetexistencediscriminationDETR-styledetection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PropVG argues that the old two-stage recipe for visual grounding — propose candidate regions, then match them to the text — can be revived if the proposal generator is trained end-to-end inside the model instead of supplied by a pre-trained detector. The paper's central claim is that learning to propose all prominent foreground objects first, then scoring each proposal's referentiality, matches or beats both modern one-stage direct-referring systems and much larger multimodal language models on classic and generalized grounding benchmarks. Two new mechanisms carry the argument: a contrastive refer-scoring module that compares each proposal to the full sentence and to individual words, and a multi-granularity target-discrimination module that fuses proposal and segmentation evidence to judge whether the expression refers to anything at all. This matters because generalized visual grounding — zero or multiple referents, ambiguous or absent targets — needs reliable object-existence judgment in addition to localization.

What carries the argument

The central object is the learnable proposal query. A DETR-style multi-scale deformable decoder turns a fixed set of queries into foreground boxes, with Hungarian matching providing the query-to-object assignment; those same queries are then reused for referential scoring. The Contrastive-based Refer Scoring module computes a similarity between each proposal and the whole sentence and a max-pooled similarity to each word, then mixes the two with a learned weight. The Multi-granularity Target Discrimination module predicts whether any target exists by applying two score-prior cross-attention blocks — one conditioned on the referring scores, one on the segmentation mask — and combining their output with a top-K average of the segmentation map. The whole pipeline is trained jointly with detection, referring, segmentation, and existence losses.

What would settle it

Run PropVG on a test set constructed so that every referred object has a relative area below 0.05 or an absolute area below 100 pixels. If its localization accuracy stays high, the foreground filter is not a hard ceiling; if accuracy collapses while the same text on larger objects succeeds, the proposal stage's recall is confirmed as the binding constraint.

Watch

Extended reading notes

Core claim

The central discovery claimed is that foreground proposal generation and referential understanding do not need to be separate systems. A single network can emit candidate boxes for all salient objects, then treat the visual grounding task as binary classification over those candidates, and this division of labour improves accuracy and speed over prior two-stage methods because no external detector bottlenecks the pipeline. The paper further claims that training the proposal stage on all foreground objects, not only the referred target, sharpens the model's later judgments and suppresses false positives. Supported this way, the model is said to outperform existing direct-referring specialists and several multimodal-LLM baselines on ten datasets, including generalized benchmarks where expressions can describe zero or multiple objects.

Load-bearing premise

The load-bearing premise is that every referred object is prominent enough to be generated as a foreground proposal; any referent smaller than the filtering cutoff, heavily occluded, or outside the detector-like categories will be missed before the scoring stages can consider it.

Editorial extensions

If this is right

  • Two-stage, proposal-based grounding can be competitive again when the proposal stage is learned end-to-end, so future work does not have to choose between direct referring and detector-dependent pipelines.
  • Because the proposal stage is supervised by all salient foreground objects, the model gains a perception channel for non-referred objects, which direct-referring and MLLM baselines do not explicitly train for.
  • The same architecture handles classic single-target and generalized zero-or-multiple-target settings, suggesting a unified detector-free design can serve both task families.
  • For deployment, the approach offers a small-parameter model that avoids heavyweight LLM inference, making grounding available in latency-sensitive and resource-constrained settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the dataset construction's filter — absolute area above 100 pixels and relative area between 0.05 and 0.8 — effectively defines the model's competence envelope; a benchmark of very small referents would likely expose the proposal stage's recall as the hard ceiling.
  • Inference: because the text conditions the proposal branch early in the encoder, the design could transfer to tasks where the text selects which category of objects counts as foreground, such as open-vocabulary or all-instances-of-X grounding.
  • Inference: the score-prior cross-attention mechanism for existence classification is generic and could be dropped into other detection-based models that need an explicit absent-object prediction.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes PropVG, an end-to-end, detector-free, proposal-based visual grounding framework. The architecture uses a BEiT-3 multimodal encoder with parallel segmentation and detection branches; the detection branch produces N foreground proposals with a multi-scale deformable decoder, and a Contrastive-based Refer Scoring (CRS) module scores each proposal against the expression at sentence and word levels. A Multi-granularity Target Discrimination (MTD) module predicts target existence by combining the maximum refer score, a top-K average segmentation score, and a learned existence score. The model is trained with detection, referring, segmentation, and existence losses and is evaluated on REC/RES, GREC/GRES, Ref-ZOM, and R-RefCOCO/+/g, with the claim of state-of-the-art performance on ten datasets.

Significance. If the results hold, PropVG is a meaningful contribution: it shows that a proposal-based grounding framework can be trained end-to-end without an external detector, provides foreground-level supervision beyond the referred target, and reports a controlled improvement over the same-backbone OneRef method on RefCOCO/+/g. The experimental design is broad (ten benchmarks, module ablations, sensitivity analyses for K and loss weights), and the release of code and models is a practical strength. However, the headline SOTA claims currently rest on two under-supported points: several key comparisons vary the backbone as well as the method, and the proposal-recall ceiling is not examined at all.

major comments (3)
  1. [Appendix C.1 and Eq. (5)] The proposal stage is the ceiling for the whole pipeline, and that ceiling is not measured. The proposal branch is supervised only with COCO foreground boxes that pass the filter (no crowd, absolute area > 100 pixels, relative area in [0.05, 0.8]), and the CRS and MTD stages score only the N decoder queries. By Eq. (5), S_exist is the product of Max(S_ref), TAS(M_seg), and eps_exist, so if the referred object is not among the proposals, Max(S_ref) is near zero and the model cannot recover the target even when the segmentation branch is correct. The paper reports no proposal recall, no size-stratified accuracy, and no failure analysis on any dataset, and the query count N is not stated in the main text or appendix. Please report recall of the referred targets among the proposals, especially for the filtered-out size regime, and discuss how any shortfall affects the reported numbers.
  2. [Section 4.2, Tables 2-4 (and Table 1)] Most headline comparisons pit BEiT3-ViT-B against Swin-B or ResNet baselines (e.g., HDC, ReLA, LAVT, MAttNet), so part of the reported margin may be an encoder/pretraining gain rather than a method gain. The controlled comparison with OneRef, which also uses BEiT3-ViT-B, is the right kind of evidence and shows small positive gains on REC, but no same-backbone specialist baseline is provided for gRefCOCO, Ref-ZOM, or R-RefCOCO/+/g. Please add same-backbone variants of the strongest baselines (at least HDC) for these benchmarks, or explicitly separate encoder gains from method gains in the conclusions.
  3. [Appendix E.4, Fig. 8] The foreground-object filter is justified only by aggregate F1/gIoU curves, which do not show what happens to precisely the objects the filter removes. Since the filter determines the proposal supervision, and small or occluded referents are the ones that fall outside it, the choice R_low = 0.05, R_high = 0.8, min area = 100 needs a per-size analysis. Please include area-stratified recall and accuracy (small/medium/large, or deciles) to demonstrate that the reported results are not confined to prominent foreground objects.
minor comments (5)
  1. [Section 3.2 heading] The heading reads 'Constrastive-based Refer Scoring' and should be 'Contrastive-based Refer Scoring'.
  2. [Appendix F.1, Fig. 9] The text refers to 'our DeRIS' when describing the proposed method; this should read 'PropVG'.
  3. [Appendix D] The number of proposal queries N is never reported; please include it in the implementation details.
  4. [Appendix C.2] The segmentation binarization threshold Thr_m is mentioned but its value is never stated; please report it.
  5. [Section 4.2.1] The claim of '4× faster' than MAttNet is only backed by a single wall-clock number in Table 1; please report the hardware and settings used for the timing comparison.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PropVG's central claims are architectural and empirical, evaluated against external benchmarks, and no prediction or derivation reduces to its own fitted inputs by construction.

full rationale

PropVG is an empirical method paper rather than a derivation chain. The framework has three learned components: foreground proposal generation, contrastive refer scoring, and multi-granularity target discrimination, trained with standard losses (Eq. 6) and evaluated on externally established benchmarks (RefCOCO/+/g, gRefCOCO, Ref-ZOM, R-RefCOCO/+/g). The reported results are comparisons against published numbers from other methods, not quantities derived from the paper's own fitted parameters. Hyperparameters such as K, Thr_p, R_low, and loss weights are tuned on validation splits and then applied to test sets, which is standard practice and does not constitute a fitted input being renamed as a prediction. The foreground-object filter in Appendix C.1 is a design choice about training data (excluding crowd, small, and overly large objects) and is justified by ablation, not derived from the evaluation metrics; it may limit performance on small or occluded referents, but this is a correctness/robustness concern rather than circularity. The self-citations (SimVG, DeRIS) appear as related work and baselines and are not used as load-bearing justification for the proposed design, nor is any author-specific uniqueness theorem invoked to force the framework. Equation 5's dependence of the existence score on Max(S_ref) reflects a real architectural ceiling: if the referred object is absent from the proposals, the model cannot recover it. This is a structural limitation, not a circular reduction, because S_ref and M_seg are learned outputs rather than restatements of the supervision labels. Overall, no step in the paper's reasoning equates a claimed result with its input by definition, and the central SOTA claims stand on external evaluation.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The central claim rests on a standard deep learning training setup. The main external inputs are the pretrained BEiT-3 backbone, standard DETR components, and a handful of hyperparameters tuned on validation splits. No new physical or mathematical entities are introduced.

free parameters (5)
  • K in TAS (TopK Average Score) = 250
    Number of top-scoring segmentation pixels averaged in Eq. (5) for target existence; chosen on validation for best F1score and gIoU (Appendix E.1).
  • Thr_p (post-process threshold) = 0.9
    Threshold for filtering proposals by combined refer score in post-processing; tuned on validation (Appendix E.3, Table 12).
  • Loss weights (lambda_det, lambda_exist, lambda_ref) = 0.1, 0.2, 1.0
    Weights in Eq. (6); default set after ablation in Appendix E.2.
  • Foreground area filter (R_low, R_high, min absolute area) = 0.05, 0.8, 100 pixels
    Filtering of COCO foreground targets for proposal supervision; R_low tuned in Appendix E.4.
  • Scaling factor T in Eq. (2) = learned, init 0.07
    Learned temperature in cosine similarity; initialized to 0.07 and updated during training.
assumptions (3)
  • domain assumption BEiT-3 pretrained multimodal encoder provides strong, transferable visual-linguistic features.
    The entire pipeline depends on BEiT-3 for fused image-text features; the reported gains rely on this pretrained initialization (Section 3.1).
  • standard math Hungarian matching assigns object queries to ground-truth foreground objects, yielding a valid query-to-target assignment set T_ref.
    Standard DETR bipartite matching, used to transfer detection supervision to referring labels (Section 3.1).
  • domain assumption COCO object categories cover the foreground objects relevant to referring expressions in the evaluation benchmarks.
    Foreground supervision is extracted from COCO annotations (Appendix C.1); expressions referring to objects outside COCO categories would not be proposed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PropVG: End-to-End Proposal-Driven Visual Grounding with Multi-Granularity Discrimination." pith.science (2026). https://pith.science/paper/TC7QQOLX

@misc{pith2026250904833,
  author       = {Pith},
  title        = {Pith review of: PropVG: End-to-End Proposal-Driven Visual Grounding with Multi-Granularity Discrimination},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TC7QQOLX}},
  note         = {Machine review of arXiv:2509.04833}
}
read the original abstract

Recent advances in visual grounding have largely shifted away from traditional proposal-based two-stage frameworks due to their inefficiency and high computational complexity, favoring end-to-end direct reference paradigms. However, these methods rely exclusively on the referred target for supervision, overlooking the potential benefits of prominent prospective targets. Moreover, existing approaches often fail to incorporate multi-granularity discrimination, which is crucial for robust object identification in complex scenarios. To address these limitations, we propose PropVG, an end-to-end proposal-based framework that, to the best of our knowledge, is the first to seamlessly integrate foreground object proposal generation with referential object comprehension without requiring additional detectors. Furthermore, we introduce a Contrastive-based Refer Scoring (CRS) module, which employs contrastive learning at both sentence and word levels to enhance the capability in understanding and distinguishing referred objects. Additionally, we design a Multi-granularity Target Discrimination (MTD) module that fuses object- and semantic-level information to improve the recognition of absent targets. Extensive experiments on gRefCOCO (GREC/GRES), Ref-ZOM, R-RefCOCO, and RefCOCO (REC/RES) benchmarks demonstrate the effectiveness of PropVG. The codes and models are available at https://github.com/Dmmm1997/PropVG.

Figures

Figures reproduced from arXiv: 2509.04833 by the authors.

Figure 1
Figure 1. Architecture Comparison. (a) The traditional proposal-based framework relies on pre-trained detectors for pro￾posal generation. (b) The query-based direct referring framework. (c) The proposed PropVG framework, which adopts an end-to￾end, detector-free design. It first generates region candidates in the proposal stage and subsequently establishes associations with the referring expression. hension/Segmentation (GREC… view at source ↗
Figure 2
Figure 2. Overview of PropVG. PropVG utilizes the BEiT-3 [66] architecture to extract multi-modal representations, followed by the SimFPN module for multi-scale feature extraction. The framework then branches into two pathways: one employs a UNet Decoder and SegHead for global referring segmentation, while the other generates foreground proposals using a Multi-scale Deformable Decoder and DetHead. The CRS module computes the … view at source ↗
Figure 3
Figure 3. Architecture of CRS. The architecture begins with a MHCA to facilitate interaction between Q ′ prop and fv. Subse￾quently, sentence- and word-level contrastive learning are applied to compute the referring score. Finally, a learnable weight param￾eter is introduced to balance the contributions of these two levels. textual features ft as follows: \setlength {\abovedisplayskip }{5pt} \setlength {\belowdisplayskip }{5p… view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Architecture of MTD. The MTD comprises two SPCA blocks to integrate object- and semantic-level predictions, using the referring score Sref and segmentation prediction Mseg as at￾tention priors. TAS calculates the average score of the top K pixels in the segmentation pr…
Figure 5
Figure 5. Figure 5: Visualization of predicted proposals, alongside the detection and segmentation results under various textual conditions. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Post-Process Flowchart. We combine the Refer Score and Det Score to form the Proposal Referential Score, which is used to filter the referred target via a threshold T hrp. NMS is optional and not mandatory for DETR-based architectures. on prominent foreground objects w…
Figure 7
Figure 7. Figure 7: Impact of the Hyperparameter K in TopK Average Scoring (TAS) [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Impact of the lower limit on the relative area of the [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Illustration the effect of incorporating foreground supervision. [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Illustration of foreground object prediction results under different texts. [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Visualization of object queries with corresponding detection boxes, detection scores, and referential scores. Red boxes indicate [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: Visualization of RefCOCO/+/g dataset. 7 [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: Visualization of gRefCOCO dataset. Proposals(Pred) refers to the predicted proposal boxes, Refer Box(Pred) refers to the predicted refer boxes, and Refer Mask(Pred) refers to the predicted refer masks. 8 [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 14
Figure 14. Figure 14: Visualization of R-RefCOCO/+/g dataset. 9 [PITH_FULL_IMAGE:figures/full_fig_p020_14.png]
Figure 15
Figure 15. Figure 15: Visualization of Ref-ZOM Dataset. RefZOM (object>5) refers to samples in the Ref-ZOM dataset where the number of objects exceeds five. 10 [PITH_FULL_IMAGE:figures/full_fig_p021_15.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

87 extracted references · 73 canonical work pages

  1. [1]

    Qwen-vl: A versatile vision-language model for un- derstanding, localization, text reading, and beyond.arXiv, 1 (2):3, 2023

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for un- derstanding, localization, text reading, and beyond.arXiv, 1 (2):3, 2023. 6

    2023

  2. [2]

    End- to-end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End- to-end object detection with transformers. InECCV, pages 213–229, 2020. 3, 5, 7

    2020

  3. [3]

    Lion: Empowering multimodal large language model with dual-level visual knowledge

    Gongwei Chen, Leyang Shen, Rui Shao, Xiang Deng, and Liqiang Nie. Lion: Empowering multimodal large language model with dual-level visual knowledge. InCVPR, pages 26540–26550, 2024. 3

    2024

  4. [4]

    Ref-nms: Breaking proposal bottlenecks in two-stage referring expression grounding

    Long Chen, Wenbo Ma, Jun Xiao, Hanwang Zhang, and Shih-Fu Chang. Ref-nms: Breaking proposal bottlenecks in two-stage referring expression grounding. InAAAI, pages 1036–1044, 2021. 2, 6

    2021

  5. [5]

    An efficient and effective transformer decoder-based framework for multi-task visual grounding

    Wei Chen, Long Chen, and Yu Wu. An efficient and effective transformer decoder-based framework for multi-task visual grounding. InECCV, 2024. 3, 5, 6

    2024

  6. [6]

    Sam4mllm: Enhance multi- modal large language model for referring expression seg- mentation

    Yi-Chia Chen, Wei-Hua Li, Cheng Sun, Yu-Chiang Frank Wang, and Chu-Song Chen. Sam4mllm: Enhance multi- modal large language model for referring expression seg- mentation. InECCV, pages 323–340, 2024. 3, 6

    2024

  7. [7]

    Parallel vertex diffusion for unified visual grounding

    Zesen Cheng, Kehan Li, Peng Jin, Siheng Li, Xiangyang Ji, Li Yuan, Chang Liu, and Jie Chen. Parallel vertex diffusion for unified visual grounding. InAAAI, pages 1326–1334,

  8. [8]

    Mask grounding for referring image seg- mentation

    Yong Xien Chng, Henry Zheng, Yizeng Han, Xuchong Qiu, and Gao Huang. Mask grounding for referring image seg- mentation. InCVPR, pages 26573–26583, 2024. 1, 3

    work page 2024

Show all 87 references
  1. [9]

    Simvg: A simple framework for visual grounding with decoupled multi-modal fusion

    Ming Dai, Lingfeng Yang, Yihao Xu, Zhenhua Feng, and Wankou Yang. Simvg: A simple framework for visual grounding with decoupled multi-modal fusion. InNeurIPS,

  2. [10]

    Deris: De- coupling perception and cognition for enhanced referring im- age segmentation through loopback synergy.ICCV, 2025

    Ming Dai, Wenxuan Cheng, Jiang-jiang Liu, Sen Yang, Wenxiao Cai, Yanpeng Sun, and Wankou Yang. Deris: De- coupling perception and cognition for enhanced referring im- age segmentation through loopback synergy.ICCV, 2025. 3

    2025

  3. [11]

    Multi-task visual grounding with coarse- to-fine consistency constraints

    Ming Dai, Jian Li, Jiedong Zhuang, Xian Zhang, and Wankou Yang. Multi-task visual grounding with coarse- to-fine consistency constraints. InAAAI, pages 2618–2626,

  4. [12]

    Transvg: End-to-end visual ground- ing with transformers

    Jiajun Deng, Zhengyuan Yang, Tianlang Chen, Wengang Zhou, and Houqiang Li. Transvg: End-to-end visual ground- ing with transformers. InICCV, pages 1769–1779, 2021. 1, 2, 6

    2021

  5. [13]

    Transvg++: End-to-end visual grounding with lan- guage conditioned vision transformer.TPAMI, 2023

    Jiajun Deng, Zhengyuan Yang, Daqing Liu, Tianlang Chen, Wengang Zhou, Yanyong Zhang, Houqiang Li, and Wanli Ouyang. Transvg++: End-to-end visual grounding with lan- guage conditioned vision transformer.TPAMI, 2023. 2

    2023

  6. [14]

    Vision-language transformer and query generation for refer- ring segmentation

    Henghui Ding, Chang Liu, Suchen Wang, and Xudong Jiang. Vision-language transformer and query generation for refer- ring segmentation. InICCV, pages 16301–16310, 2021. 1, 3, 6, 7

    2021

  7. [15]

    En- coder fusion network with co-attention embedding for refer- ring image segmentation

    Guang Feng, Zhiwei Hu, Lihe Zhang, and Huchuan Lu. En- coder fusion network with co-attention embedding for refer- ring image segmentation. InCVPR, pages 15506–15515,

  8. [16]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Dollar, and Ross Gir- shick. Mask r-cnn. InICCV, 2017. 2

    2017

  9. [17]

    GREC: Generalized referring expression comprehension

    Shuting He, Henghui Ding, Chang Liu, and Xudong Jiang. GREC: Generalized referring expression comprehension. arXiv, 2023. 1, 2, 3, 5, 6, 7

    2023

  10. [18]

    Learning to compose and reason with lan- guage tree structures for visual grounding.IEEE TPAMI,

    Richang Hong, Daqing Liu, Xiaoyu Mo, Xiangnan He, and Hanwang Zhang. Learning to compose and reason with lan- guage tree structures for visual grounding.IEEE TPAMI,

  11. [19]

    Seg- mentation from natural language expressions

    Ronghang Hu, Marcus Rohrbach, and Trevor Darrell. Seg- mentation from natural language expressions. InECCV, pages 108–124, 2016. 1, 2

    2016

  12. [20]

    Natural language object retrieval

    Ronghang Hu, Huazhe Xu, Marcus Rohrbach, Jiashi Feng, Kate Saenko, and Trevor Darrell. Natural language object retrieval. InCVPR, pages 4555–4564, 2016. 1

    2016

  13. [21]

    Modeling relationships in refer- ential expressions with compositional modular networks

    Ronghang Hu, Marcus Rohrbach, Jacob Andreas, Trevor Darrell, and Kate Saenko. Modeling relationships in refer- ential expressions with compositional modular networks. In CVPR, pages 1115–1124, 2017. 1, 2

    2017

  14. [22]

    Beyond one-to-one: Re- thinking the referring image segmentation

    Yutao Hu, Qixiong Wang, Wenqi Shao, Enze Xie, Zhenguo Li, Jungong Han, and Ping Luo. Beyond one-to-one: Re- thinking the referring image segmentation. InICCV, pages 4044–4054, 2023. 2, 3, 5, 6, 7, 1

    2023

  15. [23]

    Densely connected parameter- efficient tuning for referring image segmentation.AAAI,

    Jiaqi Huang, Zunnan Xu, Ting Liu, Yong Liu, Haonan Han, Kehong Yuan, and Xiu Li. Densely connected parameter- efficient tuning for referring image segmentation.AAAI,

  16. [24]

    Referring im- age segmentation via cross-modal progressive comprehen- sion

    Shaofei Huang, Tianrui Hui, Si Liu, Guanbin Li, Yunchao Wei, Jizhong Han, Luoqi Liu, and Bo Li. Referring im- age segmentation via cross-modal progressive comprehen- sion. InCVPR, pages 10485–10494, 2020. 2

    2020

  17. [25]

    Mdetr- modulated detection for end-to-end multi-modal understand- ing

    Aishwarya Kamath, Mannat Singh, Yann LeCun, Gabriel Synnaeve, Ishan Misra, and Nicolas Carion. Mdetr- modulated detection for end-to-end multi-modal understand- ing. InICCV, pages 1780–1790, 2021. 2, 7

    2021

  18. [26]

    Segvg: Transferring object bounding box to segmentation for visual grounding

    Weitai Kang, Gaowen Liu, Mubarak Shah, and Yan Yan. Segvg: Transferring object bounding box to segmentation for visual grounding. InECCV, pages 57–75, 2024. 6

    2024

  19. [27]

    Restr: Convolution-free referring image segmentation using transformers

    Namyup Kim, Dongwon Kim, Suha Kwak, Cuiling Lan, and Wenjun Zeng. Restr: Convolution-free referring image segmentation using transformers. InCVPR, pages 18124– 18133, 2022. 3 9

    2022

  20. [28]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. InICCV, pages 4015–4026, 2023. 6

    2023

  21. [29]

    Lisa: Reasoning segmenta- tion via large language model

    Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisa: Reasoning segmenta- tion via large language model. InCVPR, pages 9579–9589,

  22. [30]

    A survey on benchmarks of multimodal large language models

    Jian Li, Weiheng Lu, Hao Fei, Meng Luo, Ming Dai, Min Xia, Yizhang Jin, Zhenye Gan, Ding Qi, Chaoyou Fu, et al. A survey on benchmarks of multimodal large language models. arXiv preprint arXiv:2408.08632, 2024. 3

    2024 arXiv

  23. [31]

    Grounded language-image pre-training

    Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jian- wei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, et al. Grounded language-image pre-training. InCVPR, pages 10965–10975,

  24. [32]

    Referring transformer: A one- step approach to multi-task visual grounding.NeurIPS, 34,

    Muchen Li and Leonid Sigal. Referring transformer: A one- step approach to multi-task visual grounding.NeurIPS, 34,

  25. [33]

    Bring adaptive binding prototypes to generalized referring expres- sion segmentation.arXiv, 2024

    Weize Li, Zhicheng Zhao, Haochen Bai, and Fei Su. Bring adaptive binding prototypes to generalized referring expres- sion segmentation.arXiv, 2024. 2

    2024

  26. [34]

    Exploring plain vision transformer backbones for object de- tection

    Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He. Exploring plain vision transformer backbones for object de- tection. InECCV, pages 280–296, 2022. 3, 7

    2022

  27. [35]

    Ground- inggpt: Language enhanced multi-modal grounding model

    Zhaowei Li, Qi Xu, Dong Zhang, Hang Song, Yiqing Cai, Qi Qi, Ran Zhou, Junting Pan, Zefeng Li, Vu Tu, et al. Ground- inggpt: Language enhanced multi-modal grounding model. InACL, pages 6657–6678, 2024. 3

    2024

  28. [36]

    Chenxi Liu, Zhe Lin, Xiaohui Shen, Jimei Yang, Xin Lu, and Alan L. Yuille. Recurrent multimodal interaction for referring image segmentation. InICCV, pages 1280–1289,

  29. [37]

    GRES: gen- eralized referring expression segmentation

    Chang Liu, Henghui Ding, and Xudong Jiang. GRES: gen- eralized referring expression segmentation. InCVPR, pages 23592–23601, 2023. 1, 2, 3, 5, 6

    2023

  30. [38]

    Multi-modal mutual attention and iterative interaction for re- ferring image segmentation.TPAMI, 32:3054–3065, 2023

    Chang Liu, Henghui Ding, Yulun Zhang, and Xudong Jiang. Multi-modal mutual attention and iterative interaction for re- ferring image segmentation.TPAMI, 32:3054–3065, 2023. 3

    2023

  31. [39]

    Learning to assemble neural module tree networks for visual grounding

    Daqing Liu, Hanwang Zhang, Feng Wu, and Zheng-Jun Zha. Learning to assemble neural module tree networks for visual grounding. InICCV, pages 4673–4682, 2019. 1, 2, 6

    2019

  32. [40]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. InNeurIPS, 2023. 3

    2023

  33. [41]

    Poly- former: Referring image segmentation as sequential polygon generation

    Jiang Liu, Hui Ding, Zhaowei Cai, Yuting Zhang, Ravi Ku- mar Satzoda, Vijay Mahadevan, and R Manmatha. Poly- former: Referring image segmentation as sequential polygon generation. InCVPR, pages 18653–18663, 2023. 3, 6

    2023

  34. [42]

    Dq-detr: Dual query detection transformer for phrase extraction and grounding

    Shilong Liu, Shijia Huang, Feng Li, Hao Zhang, Yaoyuan Liang, Hang Su, Jun Zhu, and Lei Zhang. Dq-detr: Dual query detection transformer for phrase extraction and grounding. InAAAI, pages 1728–1736, 2023. 2

    2023

  35. [43]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection.arXiv, 2023

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection.arXiv, 2023. 6

    2023

  36. [44]

    CARIS: context-aware re- ferring image segmentation

    Sun’ao Liu, Yiheng Zhang, Zhaofan Qiu, Hongtao Xie, Yongdong Zhang, and Ting Yao. CARIS: context-aware re- ferring image segmentation. InACMMM, pages 779–788,

  37. [45]

    Dara: Domain-and relation-aware adapters make parameter- efficient tuning for visual grounding

    Ting Liu, Xuyang Liu, Siteng Huang, Honggang Chen, Quanjun Yin, Long Qin, Donglin Wang, and Yue Hu. Dara: Domain-and relation-aware adapters make parameter- efficient tuning for visual grounding. InICME, pages 1–6,

  38. [46]

    Mapper: Multimodal prior-guided param- eter efficient tuning for referring expression comprehension

    Ting Liu, Zunnan Xu, Yue Hu, Liangtao Shi, Zhiqiang Wang, and Quanjun Yin. Mapper: Multimodal prior-guided param- eter efficient tuning for referring expression comprehension. InEMNLP, pages 4984–4994, 2024. 2

    2024

  39. [47]

    Improving referring expression grounding with cross-modal attention-guided erasing

    Xihui Liu, Zihao Wang, Jing Shao, Xiaogang Wang, and Hongsheng Li. Improving referring expression grounding with cross-modal attention-guided erasing. InCVPR, pages 1950–1959, 2019. 2

    1950

  40. [48]

    Improving referring expression grounding with cross-modal attention-guided erasing

    Xihui Liu, Zihao Wang, Jing Shao, Xiaogang Wang, and Hongsheng Li. Improving referring expression grounding with cross-modal attention-guided erasing. InCVPR, pages 1950–1959, 2019. 1, 2

    1950

  41. [49]

    Multi-task collaborative network for joint referring expression comprehension and segmentation

    Gen Luo, Yiyi Zhou, Xiaoshuai Sun, Liujuan Cao, Chenglin Wu, Cheng Deng, and Rongrong Ji. Multi-task collaborative network for joint referring expression comprehension and segmentation. InCVPR, pages 10034–10043, 2020. 2, 3, 7

    2020

  42. [50]

    Hdc: Hierarchical semantic decoding with counting assistance for generalized referring expression segmentation.arXiv, 2024

    Zhuoyan Luo, Yinghao Wu, Yong Liu, Yicheng Xiao, Xiao- Ping Zhang, and Yujiu Yang. Hdc: Hierarchical semantic decoding with counting assistance for generalized referring expression segmentation.arXiv, 2024. 5, 6, 7

    2024

  43. [51]

    Generation and comprehension of unambiguous object descriptions

    Junhua Mao, Jonathan Huang, Alexander Toshev, Oana Camburu, Alan L Yuille, and Kevin Murphy. Generation and comprehension of unambiguous object descriptions. In CVPR, pages 11–20, 2016. 5, 1

    2016

  44. [52]

    Mod- eling context between objects for referring expression under- standing

    Varun K Nagaraja, Vlad I Morariu, and Larry S Davis. Mod- eling context between objects for referring expression under- standing. InECCV, pages 792–807, 2016. 6

    2016

  45. [53]

    Kosmos-2: Grounding multimodal large language models to the world.arXiv, 2023

    Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. Kosmos-2: Grounding multimodal large language models to the world.arXiv, 2023. 3

    2023

  46. [54]

    Anwer, Eric Xing, Ming-Hsuan Yang, and Fahad S

    Hanoona Rasheed, Muhammad Maaz, Sahal Shaji, Abdel- rahman Shaker, Salman Khan, Hisham Cholakkal, Rao M. Anwer, Eric Xing, Ming-Hsuan Yang, and Fahad S. Khan. Glamm: Pixel grounding large multimodal model.CVPR,

  47. [55]

    Yolov3: An incremental improvement.arXiv, 2018

    Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement.arXiv, 2018. 2

    2018

  48. [56]

    Faster r-cnn: Towards real-time object detection with region proposal networks.TPAMI, 39(6):1137–1149, 2016

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks.TPAMI, 39(6):1137–1149, 2016. 2

    2016

  49. [57]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. InMICCAI, pages 234–241, 2015. 3, 7

    2015

  50. [58]

    Lqm- former: Language-aware query mask transformer for refer- ring image segmentation

    Nisarg A Shah, Vibashan VS, and Vishal M Patel. Lqm- former: Language-aware query mask transformer for refer- ring image segmentation. InCVPR, pages 12903–12913,

  51. [59]

    Dynamic mdetr: A dynamic multimodal transformer decoder for visual grounding.TPAMI, 2023

    Fengyuan Shi, Ruopeng Gao, Weilin Huang, and Limin Wang. Dynamic mdetr: A dynamic multimodal transformer decoder for visual grounding.TPAMI, 2023. 2

    2023

  52. [60]

    Referring expression comprehension using language adaptive inference.arXiv, 2023

    Wei Su, Peihan Miao, Huanzhang Dou, Yongjian Fu, and Xi Li. Referring expression comprehension using language adaptive inference.arXiv, 2023. 2

    2023

  53. [61]

    Language adaptive weight generation for multi-task visual grounding

    Wei Su, Peihan Miao, Huanzhang Dou, Gaoang Wang, Liang Qiao, Zheyang Li, and Xi Li. Language adaptive weight generation for multi-task visual grounding. InCVPR, pages 10857–10866, 2023. 3, 6

    2023

  54. [62]

    Scan- former: Referring expression comprehension by iteratively scanning

    Wei Su, Peihan Miao, Huanzhang Dou, and Xi Li. Scan- former: Referring expression comprehension by iteratively scanning. InCVPR, pages 13449–13458, 2024. 1, 2

    2024

  55. [63]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv, 2023. 3, 6

    2023

  56. [64]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv, 2023. 6

    2023

  57. [65]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. InNeurIPS, 2017. 3

    2017

  58. [66]

    Image as a foreign language: BEiT pretraining for vision and vision-language tasks

    Wenhui Wang, Hangbo Bao, Li Dong, Johan Bjorck, Zhil- iang Peng, Qiang Liu, Kriti Aggarwal, Owais Khan Mo- hammed, Saksham Singhal, Subhojit Som, and Furu Wei. Image as a foreign language: BEiT pretraining for vision and vision-language tasks. InCVPR, 2023. 3, 4

    2023

  59. [67]

    Towards robust referring image seg- mentation.TIP, 2024

    Jianzong Wu, Xiangtai Li, Xia Li, Henghui Ding, Yunhai Tong, and Dacheng Tao. Towards robust referring image seg- mentation.TIP, 2024. 2, 3, 5, 6, 7, 1

    2024

  60. [68]

    Gsva: Generalized segmentation via multimodal large language models

    Zhuofan Xia, Dongchen Han, Yizeng Han, Xuran Pan, Shiji Song, and Gao Huang. Gsva: Generalized segmentation via multimodal large language models. InCVPR, pages 3858– 3869, 2024. 3, 6, 7

    2024

  61. [69]

    Hivg: Hierarchical multimodal fine- grained modulation for visual grounding.ACMMM, 2024

    Linhui Xiao, Xiaoshan Yang, Fang Peng, Yaowei Wang, and Changsheng Xu. Hivg: Hierarchical multimodal fine- grained modulation for visual grounding.ACMMM, 2024. 6

    2024

  62. [70]

    Oneref: Unified one-tower expression grounding and segmentation with mask referring modeling

    Linhui Xiao, Xiaoshan Yang, Fang Peng, Yaowei Wang, and Changsheng Xu. Oneref: Unified one-tower expression grounding and segmentation with mask referring modeling. NeurIPS, 2024. 2, 5, 6

    2024

  63. [71]

    Universal instance perception as object discovery and retrieval

    Bin Yan, Yi Jiang, Jiannan Wu, Dong Wang, Ping Luo, Ze- huan Yuan, and Huchuan Lu. Universal instance perception as object discovery and retrieval. InCVPR, pages 15325– 15336, 2023. 7

    2023

  64. [72]

    Dynamic graph attention for referring expression comprehension

    Sibei Yang, Guanbin Li, and Yizhou Yu. Dynamic graph attention for referring expression comprehension. InICCV, pages 4644–4653, 2019. 2

    2019

  65. [73]

    A fast and accurate one- stage approach to visual grounding

    Zhengyuan Yang, Boqing Gong, Liwei Wang, Wenbing Huang, Dong Yu, and Jiebo Luo. A fast and accurate one- stage approach to visual grounding. InICCV, pages 4683– 4693, 2019. 2

    2019

  66. [74]

    Improving one-stage visual grounding by recursive sub- query construction

    Zhengyuan Yang, Tianlang Chen, Liwei Wang, and Jiebo Luo. Improving one-stage visual grounding by recursive sub- query construction. InECCV, pages 387–404, 2020. 1, 2

    2020

  67. [75]

    Zhao Yang, Jiaqi Wang, Yansong Tang, Kai Chen, Heng- shuang Zhao, and Philip H. S. Torr. LA VT: language-aware vision transformer for referring image segmentation. In CVPR, pages 18134–18144, 2022. 3, 6, 7

    2022

  68. [76]

    Vi- sual grounding with multi-modal conditional adaptation

    Ruilin Yao, Shengwu Xiong, Yichen Zhao, and Yi Rong. Vi- sual grounding with multi-modal conditional adaptation. In ACMMM, pages 3877–3886, 2024. 3

    2024

  69. [77]

    Shifting more attention to visual backbone: Query-modulated refinement networks for end-to-end visual grounding

    Jiabo Ye, Junfeng Tian, Ming Yan, Xiaoshan Yang, Xuwu Wang, Ji Zhang, Liang He, and Xin Lin. Shifting more attention to visual backbone: Query-modulated refinement networks for end-to-end visual grounding. InCVPR, pages 15502–15512, 2022. 2

    2022

  70. [78]

    Modeling context in referring expres- sions

    Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. Modeling context in referring expres- sions. InECCV, pages 69–85, 2016. 5, 6, 1

    2016

  71. [79]

    Mattnet: Modular at- tention network for referring expression comprehension

    Licheng Yu, Zhe Lin, Xiaohui Shen, Jimei Yang, Xin Lu, Mohit Bansal, and Tamara L Berg. Mattnet: Modular at- tention network for referring expression comprehension. In CVPR, pages 1307–1315, 2018. 1, 2, 5, 6

    2018

  72. [80]

    Rethinking diversified and discriminative proposal generation for visual grounding

    Zhou Yu, Jun Yu, Chenchao Xiang, Zhou Zhao, Qi Tian, and Dacheng Tao. Rethinking diversified and discriminative proposal generation for visual grounding. InIJCAI, pages 1114–1120, 2018

    2018

  73. [81]

    Grounding referring expressions in images by variational context

    Hanwang Zhang, Yulei Niu, and Shih-Fu Chang. Grounding referring expressions in images by variational context. In CVPR, pages 4158–4166, 2018. 2

    2018

  74. [82]

    A real-time global inference network for one-stage referring expression comprehension.TNNLS, 2021

    Yiyi Zhou, Rongrong Ji, Gen Luo, Xiaoshuai Sun, Jinsong Su, Xinghao Ding, Chia-Wen Lin, and Qi Tian. A real-time global inference network for one-stage referring expression comprehension.TNNLS, 2021. 2

    2021

  75. [83]

    Seqtr: A simple yet universal network for visual grounding

    Chaoyang Zhu, Yiyi Zhou, Yunhang Shen, Gen Luo, Xingjia Pan, Mingbao Lin, Chao Chen, Liujuan Cao, Xiaoshuai Sun, and Rongrong Ji. Seqtr: A simple yet universal network for visual grounding. InECCV, pages 598–615, 2022. 2, 3, 6

    2022

  76. [84]

    Deformable detr: Deformable transformers for end-to-end object detection.arXiv, 2020

    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection.arXiv, 2020. 2, 7

    2020

  77. [85]

    Parallel attention: A unified framework for visual object discovery through dialogs and queries

    Bohan Zhuang, Qi Wu, Chunhua Shen, Ian Reid, and Anton Van Den Hengel. Parallel attention: A unified framework for visual object discovery through dialogs and queries. In CVPR, pages 4252–4261, 2018. 2

    2018

  78. [86]

    Falip: Visual prompt as foveal attention boosts clip zero-shot performance

    Jiedong Zhuang, Jiaqi Hu, Lianrui Mu, Rui Hu, Xiaoyu Liang, Jiangnan Ye, and Haoji Hu. Falip: Visual prompt as foveal attention boosts clip zero-shot performance. InECCV, pages 236–253, 2024. 3

    2024

  79. [87]

    St3: Accelerating multimodal large lan- guage model by spatial-temporal visual token trimming

    Jiedong Zhuang, Lu Lu, Ming Dai, Rui Hu, Jian Chen, Qiang Liu, and Haoji Hu. St3: Accelerating multimodal large lan- guage model by spatial-temporal visual token trimming. In AAAI, pages 11049–11057, 2025. 3 11 PropVG: End-to-End Proposal-Driven Visual Grounding with Multi-Gra...

    2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.