Pith. sign in

REVIEW 5 major objections 5 minor 90 references

Advancing Visual Large Language Model for Multi-granular Versatile Perception

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A compact visual language model unifies word- and sentence-based perception in a single architecture, answering with both bounding boxes and segmentation masks.

desk verdict A solid unified perception model with strong RES/REC numbers, but the 'all four combinations' claim is missing its word-based box leg. read the letter →

arxiv 2507.16213 v1 pith:GF4FEH3X submitted 2025-07-22 cs.CV cs.AI

Wentao Xiang , Haoxian Tan , Cong Wei , Yujie Zhong , Dengjie Li , Yujiu Yang This is my paper
classification cs.CVcs.AI
keywords visuallargelanguagemodelmulti-granularitydecoderpanopticsegmentationreferringexpressiongroundingdynamicqueryselectionchain-of-thoughtdatacurationopen-vocabulary
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Visual perception tasks come in four varieties, depending on whether the user gives a single word or a full sentence and wants a bounding box or a pixel mask. Existing models typically cover only one or two of these combinations. This paper claims that a single small visual language model, with 1.3 billion parameters, can handle all four at once while retaining competitive accuracy on each. It does so by combining a chain-of-thought-style data recipe that turns many box- and mask-annotated datasets into one instruction-tuning set, a decoder that emits boxes and masks together, and a query-construction step that uses the language model's own summary token plus instruction-guided visual features. If the claim holds, one compact model could replace separate specialists for detection, grounding, panoptic segmentation, and referring segmentation.

What carries the argument

The load-bearing mechanism is a two-part query construction. First, the LLM generates a special <PER> token as part of its answer, and the hidden state of that token is projected into N base query vectors. In parallel, the instruction text, a category word list or a referring sentence, is embedded and used to select the top-N visual features from the image encoder's multi-scale feature map by similarity; those features are added to the base queries. The resulting N dynamic queries go into a multi-granularity decoder, adapted from OpenSeeD, whose shared heads output boxes, masks, and cross-modal similarity scores. Around this, a chain-of-thought-inspired data unification converts five datasets, COCO-Panoptic, Objects365, RefCOCO series, GoldG, and captions, into one instruction-tuning set, with the response formatted as '[image caption]. The perception result is <PER>.' The caption-before-summary format is the 'thinking' half of the paradigm, and the paper's ablation shows it slightly helps all metrics.

What would settle it

Train the full model but replace the instruction-guided visual feature selection with an equal number of randomly chosen visual features while keeping the LLM-based base queries; if referring-segmentation accuracy does not drop relative to the selected-feature version, the language-guided selection mechanism is not responsible for the reported gains. The same comparison should be run across all four task groups rather than the two reported in the paper's ablation.

Watch

Extended reading notes

Core claim

The paper's central claim is that the four task groups, word-based box prediction (detection and grounding), word-based mask prediction (panoptic and semantic segmentation), sentence-based box prediction (referring expression comprehension), and sentence-based mask prediction (referring segmentation), can be solved simultaneously by one VLLM-based architecture, MVP-LM, without sacrificing specialist-level accuracy. Concretely, MVP-LM reports 56.1 PQ and 66.8 mIoU on COCO-Panoptic, 83.6 cIoU on RefCOCO val and 75.6 cIoU on RefCOCOg val for referring segmentation, 93.5% accuracy on RefCOCO val for referring comprehension, and substantial open-set gains over the strongest VLLM baseline, such as 5.7 PQ on ADE20K. These scores come from a 1.3B model with a Swin-B image encoder, the Phi-1.5 language model, and a multi-granularity decoder, and the paper attributes the gains to joint training across all four task groups, a thinking-then-perceiving response format, and a decoder whose queries are built from the language model's hidden state and instruction-selected visual features.

Load-bearing premise

The argument depends on the claim that combining the <PER> summary token's hidden state with the top-N visual features chosen by similarity to the instruction produces better decoder queries than fixed learned queries, even though the paper's own ablation shows this selection hurts referring segmentation by 2.6 cIoU while helping panoptic segmentation by about a point.

Editorial extensions

If this is right

  • One 1.3B model can serve as a single perception backend for all four task groups, replacing separate detectors, grounders, panoptic segmenters, and referring segmenters in a deployment stack.
  • Joint training across the four groups lifts sentence-based perception by 6.0 cIoU on RefCOCO val while leaving word-based panoptic scores roughly unchanged, so the unification transfers across instruction types.
  • The caption-then-perceive response format, enabled by multi-caption auto-labeling and data cleaning, improves both closed-set and open-set benchmarks and is a reusable recipe for other VLLM perception models.
  • Because the same decoder answers boxes and masks, the model can be trained end-to-end with a combined loss covering language generation, classification, boxes, and masks, pointing toward a single objective for multi-task visual grounding.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The mixed query-selection ablation hints at a hybrid decoder that uses dynamic queries for word-based tasks and fixed queries for sentence-based tasks; the paper does not test this, but the numbers suggest it could beat either variant.
  • Since box annotations are much cheaper than masks, a natural extension is to scale the word-box path on massive box-only data and transfer to mask prediction; the paper's joint-training results suggest the two prediction types reinforce each other.
  • The caption-first format is effectively a learned 'think before you point' behavior; one could test whether a perception-specific reinforcement-learning stage, which the authors mention as future work, compounds the gain.
  • Unifying the four groups in one model also invites a curriculum question: whether ordering datasets by instruction complexity, words before sentences and boxes before masks, changes the final accuracy, something the paper does not ablate.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The manuscript presents MVP-LM, a compact visual large language model that combines a Swin-B image encoder, a Phi-1.5 LLM, and an OpenSeeD-style multi-granularity decoder. The proposed architecture is designed to handle four combinations of prediction type (box vs. mask) and instruction type (word vs. sentence). The training pipeline uses a chain-of-thought-inspired data unification strategy that prepends image captions to the response, and a dynamic query generation mechanism that selects visual features as decoder queries. Experiments report results on COCO-Panoptic and open-set segmentation (word-based mask), RefCOCO-series referring segmentation (sentence-based mask), and referring expression comprehension (sentence-based box), with ablations on joint training, response format, query count, and query selection.

Significance. If the full claim of covering all four perception combinations is substantiated, MVP-LM would be a compact (1.3B) unified generalist model, which is valuable for efficient multi-task perception. The paper includes useful design elements: joint training across box and mask datasets, a query selection mechanism, and a unified SFT format. The REC results at 1.3B are strong relative to the listed baselines, and the joint-training ablation is a concrete demonstration that adding more data can improve sentence-based perception by 6.0 cIoU. The paper also reports reproducible benchmark numbers and provides a clear ablation structure.

major comments (5)
  1. [Sec. 4.1, Tables 3–5] The central claim (title, Abstract, Sec. 2.4, Table 1) is that MVP-LM covers all four combinations of box/mask and word/sentence instructions. The experiments, however, never evaluate word-based box prediction: Table 3 reports mask-only tasks (COCO-Panoptic, ADE-OV, Cityscapes-OV, PC59-OV, PAS20-OV), Table 4 reports sentence-based mask tasks (RefCOCO/gRefCOCO), and Table 5 reports sentence-based box tasks (REC). No detection mAP, LVIS, or phrase-grounding result is given for category-name prompts, despite training on Objects365 and GoldG. Please add a word-based box benchmark (e.g., COCO detection, LVIS, or OV-COCO) to substantiate the fourth combination; without it, the headline claim is unverified.
  2. [Table 9] The ablation of the proposed dynamic query selection is mixed. Removing query selection and using fixed learnable queries improves RefCOCO val cIoU from 75.7 to 78.3, while only reducing COCO-Panoptic PQ from 55.6 to 54.4. The paper's explanation ('RefCOCO's simpler targets suit fixed queries, and COCO's complexity needs more flexible query selection') is post-hoc and not supported by any further analysis. Since query enhancement is one of the three stated contributions, the authors should either provide a more principled study (e.g., analyzing query selection by object size, category, or instruction complexity) or moderate the contribution claim.
  3. [Table 3] The 'open-set' segmentation benchmarks used in Table 3 (ADE-20K, Cityscapes, PC-59, PAS-20) share many categories with the training data (COCO-Panoptic and Objects365). For instance, Cityscapes and PAS-20 categories largely overlap with COCO categories. Without a split into base and novel categories, or a per-category analysis, the results do not demonstrate open-set generalization as claimed. Please report novel-category performance or otherwise control for training-category overlap.
  4. [Table 5] The REC comparison in Table 5 is selective: it includes Shikra, MiniGPT-v2, and Octopus, but omits several strong recent VLLM baselines (e.g., Qwen-VL, Ferret, mPLUG-DocOwl, and variants). The sentence 'achieves metrics that are second only to DeepSeek-VL2' is therefore not justified by the presented comparison. Please include a broader, more representative set of baselines or soften the claim.
  5. [Sec. 3.2 and Table 7] The 'CoT-inspired dataset unification' is described as a 'thinking-then-perceiving' paradigm, but the actual implementation prepends an externally generated image caption to the response; this is not the model's own chain-of-thought reasoning. The ablation in Table 7 shows only marginal gains over the no-caption baseline (RefCOCO cIoU: 75.7 vs. 75.6; COCO PQ: 55.6 vs. 55.3; mIoU: 66.2 vs. 65.7). The contribution should be either better substantiated or relabeled to reflect that it is a caption-conditioning strategy rather than chain-of-thought.
minor comments (5)
  1. [Section 2 headings and Fig. 3 caption] There are typos: 'Preception' should be 'Perception' in the Section 2 headings, and 'segmetation' should be 'segmentation' in the Fig. 3 caption.
  2. [Table 3] The entry 'PSALM [32]' points to reference [32], which is LISA, not PSALM; the correct reference is [88]. Please fix this citation.
  3. [Table 4 caption] The caption mentions 'Gray numbers denote the method using gRefCOCO for training,' but no gray numbers appear in the table. Please clarify or remove this phrase.
  4. [Datasets (Sec. 4)] Please clarify whether the gRefCOCO test images overlap with the COCO training images used for panoptic segmentation. If they are not explicitly excluded, this is a potential image-level leakage that should be discussed.
  5. [Reference [30]] The GoldG dataset is referenced as [30], but that reference is a spoken-language dataset paper; GoldG typically refers to the grounding dataset from GLIP or its constituent datasets (e.g., Flickr30K Entities and Visual Genome). Please correct the reference.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the central results are external benchmark numbers, and the self-citations to HyperSeg and LaSagnA appear only as baseline rows rather than as load-bearing authority; the missing word-based box evaluation is an evidentiary gap, not a circular step.

full rationale

The derivation chain in this paper is empirical rather than definitional. The model is trained on standard datasets (COCO-Panoptic, RefCOCO, O365, GoldG) and evaluated on external benchmarks (COCO-Panoptic, ADE-OV, Cityscapes-OV, PC59-OV, PAS20-OV, RefCOCO series, gRefCOCO, and REC). The composite loss L = L_llm + lambda_word L_word + lambda_sent L_sent + L_mask + L_box is a standard supervised objective, and no equation in the paper defines a predicted benchmark value as a function of the training constants or of a fitted parameter that is later called a prediction. Ablations in Tables 6-9 compare trained configurations; they are not fitted parameters renamed as predictions. The only self-citations are HyperSeg [71] and LaSagnA [70], which appear in Table 1 and Tables 3-4 as comparison baselines, not as justification for the architecture, the loss, or the four-combination coverage claim, so they are not load-bearing. The reviewer's concern that no benchmark evaluates word-based box prediction is a legitimate evidentiary gap about the 'all four combinations' claim in Sec. 2.4, since the reported word-based results are all mask-based; however, this is an unverified claim, not a circularity, because it is not forced by construction, by fitted inputs, or by a self-citation chain. Under the stated rules, the honest finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The central claim rests on several domain assumptions and one ad hoc pooling assumption. None are formalized or separately validated, but most are testable via ablations, and the paper provides partial evidence.

free parameters (4)
  • Composite loss weights for L_LLM, L_word/L_sent, L1, GIoU, BCE, Dice = 1, 2, 5, 2, 5, 5
    Set by hand in Table 2 with no sensitivity analysis; the balance between language loss, detection losses, and segmentation losses affects every reported number.
  • Decoder query count N = 100 (plus 100 learnable stuff queries for panoptic)
    Chosen as default after Table 8 ablation; RefCOCO cIoU ranges from 76.3 (30 queries) to 72.4 (300 queries), and COCO PQ varies by 2.5 points across settings.
  • Training-step budget = 80,000 for main models, 9,000 for ablations
    Ablation studies in Tables 6-9 use 9k iterations, so their conclusions may not match the 80k-iteration final model; this is a hand-selected budget.
  • Caption filtering thresholds for length and entropy = Not specified
    Data Refinement in Sec. 3.2 uses thresholds that are not reported, so the automatic caption pipeline cannot be exactly reproduced.
assumptions (5)
  • domain assumption Mask annotations can be converted to box annotations without loss of information.
    Invoked in Sec. 1 to justify joint training of box and mask losses; converting masks to boxes discards shape information, so the assumption is only approximately true.
  • ad hoc to paper Average pooling of LLM token embeddings for each category word yields an instruction embedding that is comparable to visual features via an MLP.
    Defined in Sec. 3.2; no experiment validates the pooling choice against alternatives such as last-token or attention-weighted embeddings.
  • domain assumption The hidden state of the <PER> summary token captures enough instruction-aware context to serve as a base query for the decoder.
    Core premise of Dynamic Query Generation in Sec. 3.1; supported only indirectly by mixed ablations (Table 9 shows fixed queries beat dynamic selection on RefCOCO).
  • domain assumption Auto-generated captions from VILA-3B, VILA-13B, InternVL2-8B, and InternVL2-26B are accurate enough for training after length/entropy filtering.
    Sec. 3.2 Multi-caption Auto-labeling relies on external caption generators whose errors propagate into the training targets; filtering keywords like 'may' and 'might' is a heuristic.
  • domain assumption The evaluation datasets for 'open-set' results are sufficiently disjoint from training categories to measure generalization.
    Table 3 uses ADE20K, Cityscapes, PC59, and PAS20 without reporting category overlap with COCO/Objects365; many classes are shared, so open-set numbers may partly reflect seen categories.
invented entities (1)
  • <PER> special summary token
    purpose: Aggregate LLM context into a query-initializing vector for the multi-granularity decoder.
    A trainable special token whose hidden state is used to initialize base queries; the paper provides ablations (Tables 7-9) but no external falsifiable handle.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Advancing Visual Large Language Model for Multi-granular Versatile Perception." pith.science (2026). https://pith.science/paper/GF4FEH3X

@misc{pith2026250716213,
  author       = {Pith},
  title        = {Pith review of: Advancing Visual Large Language Model for Multi-granular Versatile Perception},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GF4FEH3X}},
  note         = {Machine review of arXiv:2507.16213}
}
read the original abstract

Perception is a fundamental task in the field of computer vision, encompassing a diverse set of subtasks that can be systematically categorized into four distinct groups based on two dimensions: prediction type and instruction type. Notably, existing researches often focus solely on a limited subset of these potential combinations, which constrains their applicability and versatility across various contexts. In response to this challenge, we present MVP-LM, a Multi-granular and Versatile Perception framework incorporating Visual Large Language Model. Our framework is designed to integrate both word-based and sentence-based perception tasks alongside box and mask predictions within a single architecture. MVP-LM features an innovative multi-granularity decoder in conjunction with a CoT-inspired dataset unification strategy, enabling seamless supervised fine-tuning across a wide spectrum of tasks, including but not limited to panoptic segmentation, detection, grounding, and referring expression segmentation. Furthermore, we introduce a query enhancement strategy aimed at harnessing the decoding and generative capabilities inherent in VLLMs. Extensive experiments conducted across a range of benchmarks in both word-based and sentence-based perception tasks substantiate the efficacy of our framework. The code will be available at https://github.com/xiangwentao666/MVP-LM.

Figures

Figures reproduced from arXiv: 2507.16213 by the authors.

Figure 1
Figure 1. Overview of MVP-LM. MVP-LM implements perception by integrating a multi-granularity decoder into the existing VLLM framework. We utilize a unified prompt template to construct the input sequence for the LLM across different tasks. The base query is derived from the summary token of the generated response. Concurrently, we extract the instruction embeddings from the input sequence (denoted by the same color) and sele… view at source ↗
Figure 2
Figure 2. Dynamic query selection. MVP-LM expands the em￾bedding (hidden states) of the summary token into N base query vectors. Concurrently, multi-scale visual features are flattened, and their similarity with the instruction embedding from the in￾put sequence is computed. The top N similar features are then selected and integrated with the corresponding N base query vec￾tors, resulting in N final queries tailored for multi… view at source ↗
Figure 3
Figure 3. Visualization of the segmetation results on COCO, ADE20K and gRefCOCO datasets. We plot the predicted mask with a confidence larger than 0.5 for a clear visualization. The text above each image in the gRefCOCO denotes the corresponding reference text [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

90 extracted references · 61 canonical work pages

  1. [1]

    Qwen-vl: A frontier large vision-language model with versatile abilities

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966, 2023. 1, 2, 3

    arXiv 2023

  2. [2]

    End-to- end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to- end object detection with transformers. In European confer- ence on computer vision, pages 213–229. Springer, 2020. 2

    2020

  3. [3]

    See-through-text grouping for referring image segmentation

    Ding-Jie Chen, Songhao Jia, Yi-Chen Lo, Hwann-Tzong Chen, and Tyng-Luh Liu. See-through-text grouping for referring image segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 7454–7463, 2019. 3

    2019

  4. [4]

    Hybrid task cascade for instance seg- mentation

    Kai Chen, Jiangmiao Pang, Jiaqi Wang, Yu Xiong, Xiaox- iao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jianping Shi, Wanli Ouyang, et al. Hybrid task cascade for instance seg- mentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4974–4983,

  5. [5]

    Shikra: Unleashing multi- modal llm’s referential dialogue magic

    Keqin Chen, Zhao Zhang, Weili Zeng, Richong Zhang, Feng Zhu, and Rui Zhao. Shikra: Unleashing multi- modal llm’s referential dialogue magic. arXiv preprint arXiv:2306.15195, 2023. 2, 3

    arXiv 2023

  6. [6]

    Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 24185–24198, 2024. 1, 2, 3

    2024

  7. [7]

    Masked-attention mask transformer for universal image segmentation

    Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexan- der Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1290–1299, 2022. 6

    2022

  8. [8]

    The cityscapes dataset for semantic urban scene understanding

    Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. 6, 7

    2016

Show all 90 references
  1. [9]

    Phraseclick: toward achieving flexible interactive segmenta- tion by phrase and click

    Henghui Ding, Scott Cohen, Brian Price, and Xudong Jiang. Phraseclick: toward achieving flexible interactive segmenta- tion by phrase and click. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16 , pages 417–435. Springer,

    2020

  2. [10]

    Vision-language transformer and query generation for refer- ring segmentation

    Henghui Ding, Chang Liu, Suchen Wang, and Xudong Jiang. Vision-language transformer and query generation for refer- ring segmentation. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 16321–16330,

  3. [11]

    Open- vocabulary universal image segmentation with maskclip

    Zheng Ding, Jieke Wang, and Zhuowen Tu. Open- vocabulary universal image segmentation with maskclip

  4. [12]

    The pascal visual object classes (voc) challenge

    Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88:303–338, 2010. 6, 7

    2010

  5. [13]

    Scott, and Weilin Huang

    Chengjian Feng, Yujie Zhong, Yu Gao, Matthew R. Scott, and Weilin Huang. TOOD: Task-aligned One-stage Object Detection . In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 3490–3499, Los Alamitos, CA, USA, 2021. IEEE Computer Society. 2

    2021

  6. [14]

    Prompt- det: Towards open-vocabulary detection using uncurated im- ages

    Chengjian Feng, Yujie Zhong, Zequn Jie, Xiangxiang Chu, Haibing Ren, Xiaolin Wei, Weidi Xie, and Lin Ma. Prompt- det: Towards open-vocabulary detection using uncurated im- ages. In Computer Vision – ECCV 2022 , pages 701–717, Cham, 2022. Springer Nature Switzerland. 1

    2022

  7. [15]

    Instagen: Enhancing object detection by training on syn- thetic dataset

    Chengjian Feng, Yujie Zhong, Zequn Jie, Weidi Xie, and Lin Ma. Instagen: Enhancing object detection by training on syn- thetic dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14121–14130, 2024. 2

    2024

  8. [16]

    Video-r1: Reinforcing video reasoning in mllms, 2025

    Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng, Junfei Wu, Xiaoying Zhang, Benyou Wang, and Xiangyu Yue. Video-r1: Reinforcing video reasoning in mllms, 2025. 3

    2025

  9. [17]

    Frozen-detr: Enhancing detr with image understanding from frozen foundation models

    Shenghao Fu, Junkai Yan, Qize Yang, Xihan Wei, Xiao- hua Xie, and Wei-Shi Zheng. Frozen-detr: Enhancing detr with image understanding from frozen foundation models. In NeurIPS, 2024. 2

    2024

  10. [18]

    Open-vocabulary object detection via vision and language knowledge distillation

    Xiuye Gu, Tsung-Yi Lin, Weicheng Kuo, and Yin Cui. Open-vocabulary object detection via vision and language knowledge distillation. In ICLR, 2021. 2

    2021

  11. [19]

    Dataseg: Taming a universal multi-dataset multi-task segmentation model.Ad- vances in Neural Information Processing Systems, 36, 2024

    Xiuye Gu, Yin Cui, Jonathan Huang, Abdullah Rashwan, Xuan Yang, Xingyi Zhou, Golnaz Ghiasi, Weicheng Kuo, Huizhong Chen, Liang-Chieh Chen, et al. Dataseg: Taming a universal multi-dataset multi-task segmentation model.Ad- vances in Neural Information Processing Systems, 36, 2024. 6

    2024

  12. [20]

    Textbooks are all you need

    Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio C´esar Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, et al. Textbooks are all you need. arXiv preprint arXiv:2306.11644, 2023. 3, 6

    2023 arXiv

  13. [21]

    Llava-uhd: An lmm perceiving any aspect ratio and high- resolution images

    Zonghao Guo, Ruyi Xu, Yuan Yao, Junbo Cui, Zanlin Ni, Chunjiang Ge, Tat-Seng Chua, Zhiyuan Liu, and Gao Huang. Llava-uhd: An lmm perceiving any aspect ratio and high- resolution images. In Computer Vision – ECCV 2024, pages 390–406, Cham, 2025. Springer Nature Switzerland. 3

    2024

  14. [22]

    Open-vocabulary semantic segmentation with decou- pled one-pass network

    Cong Han, Yujie Zhong, Dengjie Li, Kai Han, and Lin Ma. Open-vocabulary semantic segmentation with decou- pled one-pass network. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision (ICCV) , pages 1086–1096, 2023. 2, 6

    2023

  15. [23]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017. 1

    2017

  16. [24]

    Bi-directional relationship inferring net- work for referring image segmentation

    Zhiwei Hu, Guang Feng, Jiayu Sun, Lihe Zhang, and Huchuan Lu. Bi-directional relationship inferring net- work for referring image segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4424–4433, 2020. 3

    2020

  17. [25]

    Densely connected parameter- efficient tuning for referring image segmentation, 2025

    Jiaqi Huang, Zunnan Xu, Ting Liu, Yong Liu, Haonan Han, Kehong Yuan, and Xiu Li. Densely connected parameter- efficient tuning for referring image segmentation, 2025

    2025

  18. [26]

    Referring im- age segmentation via cross-modal progressive comprehen- sion

    Shaofei Huang, Tianrui Hui, Si Liu, Guanbin Li, Yunchao Wei, Jizhong Han, Luoqi Liu, and Bo Li. Referring im- age segmentation via cross-modal progressive comprehen- sion. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 10488–10497, 2020

    2020

  19. [27]

    Linguistic structure guided context modeling for referring image segmentation

    Tianrui Hui, Si Liu, Shaofei Huang, Guanbin Li, Sansi Yu, Faxi Zhang, and Jizhong Han. Linguistic structure guided context modeling for referring image segmentation. In European Conference on Computer Vision, pages 59–75. Springer, 2020. 3

    2020

  20. [28]

    Oneformer: One transformer to rule universal image segmentation

    Jitesh Jain, Jiachen Li, Mang Tik Chiu, Ali Hassani, Nikita Orlov, and Humphrey Shi. Oneformer: One transformer to rule universal image segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2989–2998, 2023. 6

    2023

  21. [29]

    Mdetr- modulated detection for end-to-end multi-modal understand- ing

    Aishwarya Kamath, Mannat Singh, Yann LeCun, Gabriel Synnaeve, Ishan Misra, and Nicolas Carion. Mdetr- modulated detection for end-to-end multi-modal understand- ing. In Proceedings of the IEEE/CVF international confer- ence on computer vision, pages 1780–1790, 2021. 1, 2

    2021

  22. [30]

    A spoken language dataset of descrip- tions for speech-based grounded language learning

    Gaoussou Youssouf Kebe, Padraig Higgins, Patrick Jenk- ins, Kasra Darvish, Rishabh Sachdeva, Ryan Barron, John Winder, Donald Engel, Edward Raff, Francis Ferraro, and Cynthia Matuszek. A spoken language dataset of descrip- tions for speech-based grounded language learning. In ...

    2021

  23. [31]

    F-vlm: Open-vocabulary object detection upon frozen vision and language models

    Weicheng Kuo, Yin Cui, Xiuye Gu, AJ Piergiovanni, and Anelia Angelova. F-vlm: Open-vocabulary object detection upon frozen vision and language models. In ICLR, 2023. 2

    2023

  24. [32]

    Lisa: Reasoning segmentation via large language model

    Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisa: Reasoning segmentation via large language model. ArXiv, abs/2308.00692, 2023. 1, 6

    2023 arXiv

  25. [33]

    Lisa: Reasoning segmentation via large language model

    Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisa: Reasoning segmentation via large language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9579–9589, 2024. 2, 3

    2024

  26. [34]

    Discobox: Weakly supervised instance segmentation and semantic correspondence from box super- vision

    Shiyi Lan, Zhiding Yu, Christopher Choy, Subhashree Rad- hakrishnan, Guilin Liu, Yuke Zhu, Larry S Davis, and An- ima Anandkumar. Discobox: Weakly supervised instance segmentation and semantic correspondence from box super- vision. In Proceedings of the IEEE/CVF International ...

    2021

  27. [35]

    Vision transformers are good mask auto-labelers

    Shiyi Lan, Xitong Yang, Zhiding Yu, Zuxuan Wu, Jose M Alvarez, and Anima Anandkumar. Vision transformers are good mask auto-labelers. arXiv preprint arXiv:2301.03992,

    arXiv

  28. [36]

    Mask dino: Towards a unified transformer-based framework for object detection and segmentation

    Feng Li, Hao Zhang, Huaizhe Xu, Shilong Liu, Lei Zhang, Lionel M Ni, and Heung-Yeung Shum. Mask dino: Towards a unified transformer-based framework for object detection and segmentation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pa...

    2023

  29. [37]

    Distilling detr with visual-linguistic knowledge for open-vocabulary object detection

    Liangqi Li, Jiaxu Miao, Dahu Shi, Wenming Tan, Ye Ren, Yi Yang, and Shiliang Pu. Distilling detr with visual-linguistic knowledge for open-vocabulary object detection. In ICCV,

  30. [38]

    Grounded language-image pre-training

    Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jian- wei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, et al. Grounded language-image pre-training. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, page...

    2022

  31. [39]

    Box-supervised instance seg- mentation with level set evolution

    Wentong Li, Wenyu Liu, Jianke Zhu, Miaomiao Cui, Xian- Sheng Hua, and Lei Zhang. Box-supervised instance seg- mentation with level set evolution. In Computer Vision– ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXIX, pages 1–18....

    2022

  32. [40]

    Fully convolutional networks for panoptic segmentation with point-based supervision

    Yanwei Li, Hengshuang Zhao, Xiaojuan Qi, Yukang Chen, Lu Qi, Liwei Wang, Zeming Li, Jian Sun, and Jiaya Jia. Fully convolutional networks for panoptic segmentation with point-based supervision. IEEE transactions on pattern anal- ysis and machine intelligence, 45(4):4552–4568, 2022. 2

    2022

  33. [41]

    A real-time cross-modality correlation fil- tering method for referring expression comprehension

    Yue Liao, Si Liu, Guanbin Li, Fei Wang, Yanjie Chen, Chen Qian, and Bo Li. A real-time cross-modality correlation fil- tering method for referring expression comprehension. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 10880–1088...

    2020

  34. [42]

    Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll ´ar, and C

    Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll ´ar, and C. Lawrence Zitnick. Microsoft coco: Common objects in context. In European Conference on Computer Vision, 2014. 6

    2014

  35. [43]

    Gres: Gener- alized referring expression segmentation

    Chang Liu, Henghui Ding, and Xudong Jiang. Gres: Gener- alized referring expression segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23592–23601, 2023. 3, 6, 7

    2023

  36. [44]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024. 6

    2024

  37. [45]

    Poly- former: Referring image segmentation as sequential poly- gon generation

    Jiang Liu, Hui Ding, Zhaowei Cai, Yuting Zhang, Ravi Ku- mar Satzoda, Vijay Mahadevan, and R Manmatha. Poly- former: Referring image segmentation as sequential poly- gon generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 186...

    2023

  38. [46]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision , pages 38–55. Springe...

    2024

  39. [47]

    Knowledge-guided pairwise recon- struction network for weakly supervised referring expression grounding

    Xuejing Liu, Liang Li, Shuhui Wang, Zheng-Jun Zha, Li Su, and Qingming Huang. Knowledge-guided pairwise recon- struction network for weakly supervised referring expression grounding. In Proceedings of the 27th ACM International Conference on Multimedia, pages 539–547, 2019. 3

    2019

  40. [48]

    Learn- ing cross-modal context graph for visual grounding

    Yongfei Liu, Bo Wan, Xiaodan Zhu, and Xuming He. Learn- ing cross-modal context graph for visual grounding. In Pro- ceedings of the AAAI conference on artificial intelligence , pages 11645–11652, 2020. 3

    2020

  41. [49]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 3, 6

    2021

  42. [50]

    Visual- rft: Visual reinforcement fine-tuning, 2025

    Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual- rft: Visual reinforcement fine-tuning, 2025. 3

    2025

  43. [51]

    Deepseek-vl: towards real-world vision- language understanding

    Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, et al. Deepseek-vl: towards real-world vision- language understanding. arXiv preprint arXiv:2403.05525,

    arXiv

  44. [52]

    Scaling open-vocabulary object detection

    Matthias Minderer, Alexey Gritsenko, and Neil Houlsby. Scaling open-vocabulary object detection. Advances in Neu- ral Information Processing Systems, 36:72983–73007, 2023. 1

    2023

  45. [53]

    The role of context for object detection and semantic segmentation in the wild

    Roozbeh Mottaghi, Xianjie Chen, Xiaobai Liu, Nam-Gyu Cho, Seong-Whan Lee, Sanja Fidler, Raquel Urtasun, and Alan Yuille. The role of context for object detection and semantic segmentation in the wild. In Proceedings of the IEEE conference on computer vision and pattern recogni...

    2014

  46. [55]

    Mod- eling context between objects for referring expression under- standing

    Varun K Nagaraja, Vlad I Morariu, and Larry S Davis. Mod- eling context between objects for referring expression under- standing. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14 , pages 792–807. Springer,

    2016

  47. [56]

    Vision-aware text features in referring image segmentation: From object understanding to context understanding, 2024

    Hai Nguyen-Truong, E-Ro Nguyen, Tuan-Anh Vu, Minh- Triet Tran, Binh-Son Hua, and Sai-Kit Yeung. Vision-aware text features in referring image segmentation: From object understanding to context understanding, 2024. 3

    2024

  48. [57]

    Glamm: Pixel grounding large multimodal model

    Hanoona Rasheed, Muhammad Maaz, Sahal Shaji, Abdel- rahman Shaker, Salman Khan, Hisham Cholakkal, Rao M Anwer, Erix Xing, Ming-Hsuan Yang, and Fahad S Khan. Glamm: Pixel grounding large multimodal model. arXiv preprint arXiv:2311.03356, 2023. 1, 2, 3

    2023 arXiv

  49. [58]

    Pixellm: Pixel reasoning with large multimodal model

    Zhongwei Ren, Zhicheng Huang, Yunchao Wei, Yao Zhao, Dongmei Fu, Jiashi Feng, and Xiaojie Jin. Pixellm: Pixel reasoning with large multimodal model. ArXiv, abs/2312.02228, 2023. 1, 6

    2023 arXiv

  50. [59]

    Pixellm: Pixel reasoning with large multimodal model

    Zhongwei Ren, Zhicheng Huang, Yunchao Wei, Yao Zhao, Dongmei Fu, Jiashi Feng, and Xiaojie Jin. Pixellm: Pixel reasoning with large multimodal model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26374–26383, 2024. 2, 3

    2024

  51. [60]

    Grounding of textual phrases in images by reconstruction

    Anna Rohrbach, Marcus Rohrbach, Ronghang Hu, Trevor Darrell, and Bernt Schiele. Grounding of textual phrases in images by reconstruction. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14 , pages 817–

    2016

  52. [61]

    Objects365: A large-scale, high-quality dataset for object detection

    Shuai Shao, Zeming Li, Tianyuan Zhang, Chao Peng, Gang Yu, Xiangyu Zhang, Jing Li, and Jian Sun. Objects365: A large-scale, high-quality dataset for object detection. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 8430–8439, 2019. 6

    2019

  53. [62]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. 3

    2024

  54. [63]

    Conceptual captions: A cleaned, hypernymed, im- age alt-text dataset for automatic image captioning

    Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypernymed, im- age alt-text dataset for automatic image captioning. In Pro- ceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper...

    2018

  55. [64]

    Boxinst: High-performance instance segmentation with box annotations

    Zhi Tian, Chunhua Shen, Xinlong Wang, and Hao Chen. Boxinst: High-performance instance segmentation with box annotations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5443– 5452, 2021. 1

    2021

  56. [65]

    Max-deeplab: End-to-end panoptic segmentation with mask transformers

    Huiyu Wang, Yukun Zhu, Hartwig Adam, Alan Yuille, and Liang-Chieh Chen. Max-deeplab: End-to-end panoptic segmentation with mask transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5463–5474, 2021. 2

    2021

  57. [66]

    Ov-dino: Unified open-vocabulary de- tection with language-aware selective fusion

    Hao Wang, Pengzhen Ren, Zequn Jie, Xiao Dong, Chengjian Feng, Yinlong Qian, Lin Ma, Dongmei Jiang, Yaowei Wang, Xiangyuan Lan, et al. Ov-dino: Unified open-vocabulary de- tection with language-aware selective fusion. arXiv preprint arXiv:2407.07844, 2024. 1, 2

    2024 arXiv

  58. [67]

    Neighbourhood watch: Refer- ring expression comprehension via language-guided graph attention networks

    Peng Wang, Qi Wu, Jiewei Cao, Chunhua Shen, Lianli Gao, and Anton van den Hengel. Neighbourhood watch: Refer- ring expression comprehension via language-guided graph attention networks. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pag...

    1960

  59. [68]

    Time-R1: Post- Training Large Vision Language Model for Temporal Video Grounding

    Ye Wang, Ziheng Wang, Boshen Xu, Yang Du, Kejun Lin, Zihan Xiao, Zihao Yue, Jianzhong Ju, Liang Zhang, Dingyi Yang, Xiangnan Fang, Zewen He, Zhenbo Luo, Wenxuan Wang, Junqi Lin, Jian Luan, and Qin Jin. Time-R1: Post- Training Large Vision Language Model for Temporal Video Grou...

    2025 arXiv

  60. [69]

    Cris: Clip- driven referring image segmentation

    Zhaoqing Wang, Yu Lu, Qiang Li, Xunqiang Tao, Yandong Guo, Mingming Gong, and Tongliang Liu. Cris: Clip- driven referring image segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11686–11695, 2022. 6

    2022

  61. [70]

    Lasagna: Language-based segmentation assistant for complex queries

    Cong Wei, Haoxian Tan, Yujie Zhong, Yujiu Yang, and Lin Ma. Lasagna: Language-based segmentation assistant for complex queries. arXiv preprint arXiv:2404.08506, 2024. 6

    2024 arXiv

  62. [71]

    Hyperseg: Hybrid segmentation assistant with fine-grained visual perceiver

    Cong Wei, Yujie Zhong, Haoxian Tan, Yong Liu, Jie Hu, Dengjie Li, Zheng Zhao, and Yujiu Yang. Hyperseg: Hybrid segmentation assistant with fine-grained visual perceiver. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 8931–8941, 2025. 1

    2025

  63. [72]

    Chain-of-thought prompting elicits reasoning in large lan- guage models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large lan- guage models. Advances in neural information processing systems, 35:24824–24837, 2022. 2, 3

    2022

  64. [73]

    Aligning bag of regions for open- vocabulary object detection

    Size Wu, Wenwei Zhang, Sheng Jin, Wentao Liu, and Chen Change Loy. Aligning bag of regions for open- vocabulary object detection. In CVPR, 2023. 2

    2023

  65. [74]

    Gsva: Generalized segmen- tation via multimodal large language models

    Zhuofan Xia, Dongchen Han, Yizeng Han, Xuran Pan, Shiji Song, and Gao Huang. Gsva: Generalized segmen- tation via multimodal large language models. arXiv preprint arXiv:2312.10103, 2023. 6

    2023 arXiv

  66. [75]

    Upsnet: A unified panoptic segmentation network

    Yuwen Xiong, Renjie Liao, Hengshuang Zhao, Rui Hu, Min Bai, Ersin Yumer, and Raquel Urtasun. Upsnet: A unified panoptic segmentation network. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8818–8826, 2019. 2

    2019

  67. [76]

    A simple baseline for open- vocabulary semantic segmentation with pre-trained vision- language model

    Mengde Xu, Zheng Zhang, Fangyun Wei, Yutong Lin, Yue Cao, Han Hu, and Xiang Bai. A simple baseline for open- vocabulary semantic segmentation with pre-trained vision- language model. In European Conference on Computer Vi- sion, pages 736–753. Springer, 2022. 6

    2022

  68. [77]

    Dynamic graph at- tention for referring expression comprehension

    Sibei Yang, Guanbin Li, and Yizhou Yu. Dynamic graph at- tention for referring expression comprehension. In Proceed- ings of the IEEE/CVF international conference on computer vision, pages 4644–4653, 2019. 3

    2019

  69. [78]

    R1-onevision: Advancing generalized multimodal reasoning through cross- modal formalization, 2025

    Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, Bo Zhang, and Wei Chen. R1-onevision: Advancing generalized multimodal reasoning through cross- modal formalization, 2025. 3

    2025

  70. [79]

    Lavt: Language-aware vision transformer for referring image segmentation

    Zhao Yang, Jiaqi Wang, Yansong Tang, Kai Chen, Heng- shuang Zhao, and Philip HS Torr. Lavt: Language-aware vision transformer for referring image segmentation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18155–18165, 2022. 1, 6

    2022

  71. [80]

    Modeling context in referring expres- sions

    Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. Modeling context in referring expres- sions. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, pages 69–85. Springer, 2016. 6

    2016

  72. [81]

    Modeling context in referring expres- sions

    Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. Modeling context in referring expres- sions. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, pages 69–85. Springer, 2016. 7

    2016

  73. [82]

    v-clr: View-consistent learning for open-world instance seg- mentation

    Chang-Bin Zhang, Jinhong Ni, Yujie Zhong, and Kai Han. v-clr: View-consistent learning for open-world instance seg- mentation. In Proceedings of the Computer Vision and Pat- tern Recognition Conference (CVPR) , pages 20307–20317,

  74. [83]

    Chang-Bin Zhang, Yujie Zhong, and Kai Han. Mr. detr: In- structive multi-route training for detection transformers. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pages 9933–9943, 2025. 2

    2025

  75. [84]

    Grounding referring expressions in images by variational context

    Hanwang Zhang, Yulei Niu, and Shih-Fu Chang. Grounding referring expressions in images by variational context. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4158–4166, 2018. 3

    2018

  76. [85]

    A simple framework for open-vocabulary segmentation and detection

    Hao Zhang, Feng Li, Xueyan Zou, Shilong Liu, Chunyuan Li, Jianwei Yang, and Lei Zhang. A simple framework for open-vocabulary segmentation and detection. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 1020–1031, 2023. 1, 2, 4, 6

    2023

  77. [86]

    R1-vl: Learning to reason with multimodal large language models via step- wise group relative policy optimization, 2025

    Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. R1-vl: Learning to reason with multimodal large language models via step- wise group relative policy optimization, 2025. 3

    2025

  78. [87]

    Omg-llava: Bridging image-level, object-level, pixel-level reasoning and understanding

    Tao Zhang, Xiangtai Li, Hao Fei, Haobo Yuan, Shengqiong Wu, Shunping Ji, Chen Change Loy, and Shuicheng Yan. Omg-llava: Bridging image-level, object-level, pixel-level reasoning and understanding. arXiv preprint arXiv:2406.19389, 2024. 1, 2, 6

    2024 arXiv

  79. [88]

    Psalm: Pixelwise segmentation with large multi-modal model

    Zheng Zhang, Yeyao Ma, Enming Zhang, and Xiang Bai. Psalm: Pixelwise segmentation with large multi-modal model. In European Conference on Computer Vision, pages 74–91. Springer, 2025. 1, 2, 3, 6, 7

    2025

  80. [89]

    Semantic under- standing of scenes through the ade20k dataset

    Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fi- dler, Adela Barriuso, and Antonio Torralba. Semantic under- standing of scenes through the ade20k dataset. International Journal of Computer Vision, 127(3):302–321, 2019. 6

    2019

  81. [90]

    Generalized decoding for pixel, image, and lan- guage

    Xueyan Zou, Zi-Yi Dou, Jianwei Yang, Zhe Gan, Linjie Li, Chunyuan Li, Xiyang Dai, Harkirat Behl, Jianfeng Wang, Lu Yuan, et al. Generalized decoding for pixel, image, and lan- guage. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages...

  82. [91]

    Segment everything everywhere all at once

    Xueyan Zou, Jianwei Yang, Hao Zhang, Feng Li, Linjie Li, Jianfeng Wang, Lijuan Wang, Jianfeng Gao, and Yong Jae Lee. Segment everything everywhere all at once. Advances in Neural Information Processing Systems, 36, 2024. 6

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.