Pith. sign in

REVIEW 2 major objections 6 minor 95 references

Text4Seg++: Advancing Image Segmentation via Generative Language Modeling

T0 review · 2 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that image segmentation can be cast as pure text generation, replacing mask decoders with bounded boxes and brick codes that a multimodal large language model writes autoregressively, and that this approach beats…

desk verdict A genuinely new text-as-mask encoding with strong results on several benchmarks, but the 'consistently surpasses SOTA' claim overreaches and the representation ceiling on small remote-sensing objects is untested. read the letter →

arxiv 2509.06321 v1 pith:LBX2PA4G submitted 2025-09-08 cs.CV

Mengcheng Lan , Chaofeng Chen , Jiaxing Xu , Zongrui Li , Yiping Ke , Xudong Jiang , Yingchen Yu , Yunqing Zhao
show 1 more author
Song Bai
This is my paper · ORCID
classification cs.CV
keywords imagesegmentationmultimodallargelanguagemodelstext-as-masksemanticdescriptorsbricksnext-brickpredictionreferringexpressionreasoning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a multimodal large language model (MLLM) can perform image segmentation as pure text generation, replacing the usual mask decoder with a short structured string: a quantized bounding box, a label or region tag, and brick codes that draw the mask on a 64x64 grid. The claim is that this representation is not only simpler but also more compact, more precise, and more scalable than decoder-based alternatives. A single model trained once on mixed segmentation and visual-question data is said to transfer across referring, reasoning, open-vocabulary, and remote-sensing segmentation without task-specific fine-tuning. If true, segmentation becomes a capability that an instruction-tuned multimodal language model can acquire by changing its training data rather than its architecture.

What carries the argument

The load-bearing object is the box-wise semantic descriptor (B-SD), a text tuple of the form <ref>label or region tag</ref><box>[[x1 y1 x2 y2]]</box><seg>[[brick codes]]</seg>. Inside the box, the mask is drawn with semantic bricks: 63 foreground and 63 background special tokens, each standing for a run of filled or empty cells on a 64x64 grid, so segmentation becomes next-brick prediction in raster-scan order. This compresses a mask to about 150 tokens on the referring segmentation benchmark, versus roughly 768 tokens for the earlier full-image patch labels, while providing finer spatial detail. The mechanism works because the autoregressive language model jointly decides where to look, what to call the object, and how to paint it, and the box constraint removes the background-token dominance that makes whole-image text masks verbose and less scalable.

What would settle it

Take a dataset of masks with thin structures such as power lines, hair, or small boats in satellite imagery, render each ground-truth mask through the 64x64 brick encoding, and measure the IoU ceiling between the encoded mask and the original. If that ceiling is already below the reported gains, then the representation, not the model, bounds performance and the pure-text claim fails at the encoding stage.

Watch

Extended reading notes

Core claim

The central claim is that segmentation masks can be encoded entirely as text and predicted by a generative language model, with no auxiliary mask decoder and no architectural modification. The paper introduces box-wise semantic descriptors: for each object, the model outputs a referring label or abstract region tag, a bounding box quantized into 64 discrete bins, and a sequence of semantic bricks that reconstruct the binary mask on a 64x64 canvas in raster-scan order. The authors report that this next-brick prediction formulation outperforms prior decoder-based and polygon-based generalist segmenters on referring expression segmentation, generalized referring segmentation, multi-object reasoning segmentation, referring expression comprehension, and remote sensing benchmarks, while remaining compatible with several existing MLLM backbones. The earlier image-wise variant, which labels every image patch with text and applies row-wise run-length encoding, is presented as already competitive on open-vocabulary and visual-question tasks.

Load-bearing premise

The load-bearing premise is that every target mask is faithfully representable as a 64x64 grid of foreground and background brick runs inside a quantized bounding box; if masks need finer detail than that grid can express, the pure-text approach cannot represent them no matter how well the model learns.

Editorial extensions

If this is right

  • A single model trained once on the mixed corpus sets state-of-the-art numbers on referring expression segmentation at both 7B and 13B scales, with reported RefCOCO val cIoU of 81.5 without any mask refiner.
  • The same model reaches a reported 91.1 average Acc@0.5 on referring expression comprehension, suggesting that dense mask supervision sharpens grounding even when only the bounding box is evaluated.
  • On multi-object reasoning segmentation, the paper reports an average score of 63.8, about 11 points above the prior best, without task-specific fine-tuning.
  • At 64x64 resolution, the brick-based mask matches or exceeds the quality of the same model with an external mask refiner, making the refiner optional rather than necessary.
  • Semantic bricks cut token length from 767.6 to 150.4 at 64x64 resolution, so higher resolution no longer implies proportionally longer output sequences.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the mask is expressed with a fixed vocabulary of 126 bricks and a 64x64 grid, the method's ceiling is representational: finer masks or unusual shapes would require more bricks or a learned codebook, not just more model capacity.
  • The box-first output stream makes grounding and segmentation share one set of token probabilities, so a model that misplaces a box could in principle be corrected by decoding the brick sequence back into a mask and checking consistency.
  • The remote-sensing gains suggest a testable extension to other sparse-mask domains, such as medical or aerial imagery, where boxes are natural priors and small structures remain within the 64x64 grid.
  • Treating the brick sequence as a latent code opens an avenue for verification or chain-of-thought: if the model can regenerate the box from its own bricks, the representation is self-consistent enough for agentic loops.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes Text4Seg++ (and its predecessor Text4Seg), a decoder-free paradigm that casts image segmentation as pure text generation by representing masks as semantic descriptors. The key innovations are image-wise semantic descriptors with row-wise run-length encoding (R-RLE) and, for Text4Seg++, box-wise semantic descriptors (B-SD) that couple a quantized bounding box with a compact 64x64 binary mask encoded through a vocabulary of 126 'semantic bricks'. A single MLLM (Qwen2-VL-7B or Qwen2.5-14B) is fine-tuned with LoRA on a mixed corpus of segmentation, grounding, and VQA data, then evaluated on ten benchmarks covering referring, generalized referring, reasoning, multi-object reasoning, open-vocabulary, remote-sensing, comprehension, and VQA tasks, all without task-specific fine-tuning or architectural modification. The paper reports strong results on RefCOCO-family referring segmentation, MUSE, EarthReason, and referring expression comprehension, alongside ablations on resolution, R-RLE, and semantic bricks.

Significance. If the central claim holds, this is a significant advance: it demonstrates that a decoder-scale MLLM can perform dense segmentation purely through autoregressive text generation, with no mask decoder and no architecture changes, while remaining competitive with or superior to specialized and generalist methods on several benchmarks. The paper includes extensive multi-benchmark evaluation, token-length analysis showing a 74% reduction from R-RLE and further compression from semantic bricks, and ablations that support the design choices. The method is also shown to be backbone-agnostic, working with Qwen2-VL, Deepseek-VL2, and InternVL3. However, the paper's own tables contradict the headline claim of 'consistently surpassing state-of-the-art': Seg-Zero beats Text4Seg++ on ReasonSeg, and both SegEarth-R1 and RMSIN beat it on RRSIS-D. The representation ceiling of the 64x64 B-SD grid for small remote-sensing objects is also not tested, leaving a load-bearing question unresolved.

major comments (2)
  1. [Abstract; Section V-A (Conclusion)] The claim that Text4Seg++ 'consistently surpasses existing state-of-the-art methods' and 'achieves this superior performance without any task-specific fine-tuning' is contradicted by the paper's own results. In Table IV (ReasonSeg), Seg-Zero (56.2) outperforms Text4Seg++ (54.5) on average gIoU/cIoU. In Table VII (RRSIS-D), SegEarth-R1 (74.4) and RMSIN (72.4) both outperform Text4Seg++ (70.8). The conclusion should be qualified to name the benchmarks where Text4Seg++ actually sets the state of the art, and the authors should discuss the observed gaps on ReasonSeg and RRSIS-D. This is load-bearing because the abstract and conclusion frame the contribution as universal superiority, and the current wording is not supported by the evidence in the tables.
  2. [Section III-C.2; Section V-E.1; Table VII; Figure 13] The B-SD representation fixes the mask grid at 64x64 and the bounding boxes at 64 discrete coordinate bins, so the representable mask fidelity is bounded by roughly one sixty-fourth of the image width. The resolution ablation in Figure 13 is conducted only on the RefCOCO-family benchmarks, where objects are typically large relative to the grid. On RRSIS-D, whose images are 800x800 pixels, many referents are only a few pixels wide and can fall below one 64-bin grid cell. The paper reports no boundary-sensitive metric anywhere, and it does not evaluate higher B-SD resolutions on RRSIS-D. Consequently, the hypothesis that the 64x64 representation ceiling, rather than the MLLM's learning or reasoning ability, explains the RRSIS-D gap (70.8 vs. 74.4 for SegEarth-R1) is untested. I recommend adding an oracle experiment that quantizes the ground-truth masks into the B-SD format and computes the resulting IoU ceiling, or running B-SD at higher resolution on RRSIS-D to separate representation error from model error.
minor comments (6)
  1. [Table III] The row 'Text4Seg++ Qwen2-7B ✓' appears in the 'Generalist Segmentation Models (≥13B)' block, but it is a 7B model and also appears in the 7B block; this appears to be a table formatting error.
  2. [Abstract; Section V-A] The phrase 'consistently outperforms state-of-the-art models' is too strong given the exceptions in Tables IV and VII; consider rewording to 'outperforms state-of-the-art methods on most benchmarks, with notable exceptions on ReasonSeg and RRSIS-D'.
  3. [Section II-B; Reference [11]; Table II] Reference [11] is cited as 'Towards open vocabulary learning: A survey' in the reference list, but Table II attributes the SETOKIM method to [11]; the citation numbering appears to be misaligned.
  4. [Figure 7] The caption lists 'I-SD, B-SD, and B-SD without semantic bricks' but the legend labels in the figure are not immediately mappable to these configurations; please make the legend explicit.
  5. [Section IV-B; Table I] The RRSIS-D dataset description states 17,402 image-mask-expression triplets, while Table I reports 12k images and 61k samples; clarify the relationship between these counts (e.g., number of expressions versus number of unique images).
  6. [Section IV-C.7; Table X] Table X evaluates only Text4Seg on VQA and RES benchmarks, not Text4Seg++; if the compatibility-with-visual-understanding claim is intended for Text4Seg++ as well, please include its VQA results or state clearly that this evaluation covers only Text4Seg.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; training targets are derived from ground-truth masks and evaluation is against held-out labels.

full rationale

The paper's derivation chain is a supervised learning pipeline, not a mathematical derivation: ground-truth masks are converted into semantic descriptors (I-SD or B-SD with semantic bricks), the MLLM is fine-tuned with LoRA to generate those descriptors, and final performance is measured by IoU, cIoU, gIoU, and Acc@0.5 against held-out test labels. There is no fitted parameter that is later renamed as a prediction, and no equation in Section III reduces an output to an input by construction. The 64x64 mask grid, 126-brick vocabulary, and 64-bin box quantization are design choices; the resolution ablation in Section V-E.1 and Figure 13 tunes them on validation splits and reports the trade-off honestly. Table XI even shows that removing semantic bricks leaves performance almost unchanged (79.7/74.9/78.1 vs. 79.7/75.1/78.3), which rules out the possibility that the representation itself manufactures the headline accuracy. Self-citations to the authors' prior Text4Seg paper and to GeoGround, ClearCLIP, and ProxyCLIP are used as baselines or as prior work, not as load-bearing evidence for the new model's capabilities; no uniqueness theorem is imported, and no ansatz is smuggled in via citation. The abstract's claim that Text4Seg++ 'consistently outperforms state-of-the-art models' is overstated relative to Table VII on RRSIS-D, where Text4Seg++ (70.8) trails SegEarth-R1 (74.4), but that is a correctness or wording concern, not circularity. No circular step can be exhibited, so the appropriate finding is no significant circularity with score 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 3 invented entities

The central claim is a method and empirical claim, so the ledger is dominated by representation choices and domain assumptions rather than physical constants. The free parameters are downstream of the method: grid resolution, brick vocabulary, box quantization, and input resolution are all tuned on validation data and materially affect the reported numbers. No invented physical entities appear; the invented entities are textual encodings and token vocabularies.

free parameters (4)
  • Box-wise descriptor grid resolution = 64x64 (default)
    Selected after ablations in Figure 13; 80x80 is similar but longer, 32x32 is worse. This encoding capacity anchors all segmentation outputs.
  • Semantic brick vocabulary size = 126 (63 fg + 63 bg)
    Hand-chosen to represent run lengths 1 to 63 for foreground and background. Table XI shows only minor accuracy effect, but it drives token length and next-brick decoding.
  • Bounding box coordinate bins = 64
    Coordinates are quantized into 64 bins to align with the 64x64 grid (Section III-C.2); not optimized separately from grid resolution.
  • Minimum input image pixels = 784^2
    Chosen from the input-resolution study (Figure 14) as the best trade-off; segmentation performance is sensitive to this choice.
assumptions (5)
  • domain assumption Pretrained MLLMs (Qwen2-VL, InternVL3, etc.) already encode enough visual detail for fine segmentation via LoRA fine-tuning alone.
    The whole method assumes the frozen vision encoder and connector can supply 64x64 spatial detail at 784x784 input without any new visual module (Section III-C.4).
  • domain assumption A 64x64 binary mask with 126 run bricks faithfully represents every mask in the evaluated benchmarks.
    Encoding premise for box-wise semantic descriptors; no proof that all targets are representable at this resolution, only ablations on RefCOCO (Section V-E.1).
  • domain assumption Autoregressive next-token prediction can learn row-major spatial structure from text sequences.
    The text-as-mask paradigm depends on the LLM learning to emit spatially ordered tokens across rows (Sections III-B and III-C.3).
  • domain assumption Unified SFT on the mixed dataset transfers to each benchmark without task-specific fine-tuning.
    Core empirical premise; Tables II through X test it, but results vary by dataset.
  • domain assumption Row-wise run-length encoding preserves the spatial information needed for segmentation.
    The 4 cIoU drop of image-wise RLE is presented as empirical evidence; R-RLE is assumed lossless for learned spatial structure even though it compresses the text (Section III-B.3).
invented entities (3)
  • Image-wise semantic descriptors (I-SD)
    purpose: Textual sequence mapping each image patch to its semantic label, casting masks as text for autoregressive generation.
    Introduced in Text4Seg; utility is shown only by the paper's own benchmarks, no external falsifiable prediction beyond held-out labels.
  • Box-wise semantic descriptors (B-SD)
    purpose: Compact region-level representation combining a bounding box and a textual mask for each instance.
    New representation for Text4Seg++; its effectiveness is demonstrated through in-paper experiments, not by an external prediction.
  • Semantic bricks (fg1..fg63, bg1..bg63)
    purpose: Special tokens representing binary segments of varying lengths, enabling segmentation as next-brick prediction.
    Invented vocabulary. Table XI shows minor direct accuracy gain; the main benefit is sequence length reduction, and there is no independent evidence outside the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Text4Seg++: Advancing Image Segmentation via Generative Language Modeling." pith.science (2026). https://pith.science/paper/LBX2PA4G

@misc{pith2026250906321,
  author       = {Pith},
  title        = {Pith review of: Text4Seg++: Advancing Image Segmentation via Generative Language Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LBX2PA4G}},
  note         = {Machine review of arXiv:2509.06321}
}
abstract

Multimodal Large Language Models (MLLMs) have shown exceptional capabilities in vision-language tasks. However, effectively integrating image segmentation into these models remains a significant challenge. In this work, we propose a novel text-as-mask paradigm that casts image segmentation as a text generation problem, eliminating the need for additional decoders and significantly simplifying the segmentation process. Our key innovation is semantic descriptors, a new textual representation of segmentation masks where each image patch is mapped to its corresponding text label. We first introduce image-wise semantic descriptors, a patch-aligned textual representation of segmentation masks that integrates naturally into the language modeling pipeline. To enhance efficiency, we introduce the Row-wise Run-Length Encoding (R-RLE), which compresses redundant text sequences, reducing the length of semantic descriptors by 74% and accelerating inference by $3\times$, without compromising performance. Building upon this, our initial framework Text4Seg achieves strong segmentation performance across a wide range of vision tasks. To further improve granularity and compactness, we propose box-wise semantic descriptors, which localizes regions of interest using bounding boxes and represents region masks via structured mask tokens called semantic bricks. This leads to our refined model, Text4Seg++, which formulates segmentation as a next-brick prediction task, combining precision, scalability, and generative efficiency. Comprehensive experiments on natural and remote sensing datasets show that Text4Seg++ consistently outperforms state-of-the-art models across diverse benchmarks without any task-specific fine-tuning, while remaining compatible with existing MLLM backbones. Our work highlights the effectiveness, scalability, and generalizability of text-driven image segmentation within the MLLM framework.

Figures

Figures reproduced from arXiv: 2509.06321 by the authors.

Figure 1
Figure 1. Different paradigms of MLLMs based image segmentation: (a) [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. An illustration of a popular MLLM architecture. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. An illustration of image patches, image-wise semantic descriptors and two token compression techniques. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (8 more)
Figure 5
Figure 5. Figure 5: Text4Seg architecture. 5) Text4Seg Architecture: With such text-only response design, Text4Seg can be seamlessly integrated with existing MLLMs without any architectural modifications, as shown in [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: An illustration of (a) box-wise semantic descriptors for images, (b) semantic bricks and (c) Text4Seg++ framework. [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Token counts with varying resolution of semantic descriptors for three [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 9
Figure 9. Figure 9: Qualitative results of Text4Seg and GSVA [ [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Qualitative results of Text4Seg and GSVA [ [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 12
Figure 12. Figure 12: Visualization of RES results across different resolutions, and with SAM as mask refiner. [PITH_FULL_IMAGE:figures/full_fig_p013_12.png]
Figure 15
Figure 15. Figure 15: Comparison of different encoding strategies for semantic descriptors [PITH_FULL_IMAGE:figures/full_fig_p013_15.png]
Figure 14
Figure 14. Figure 14: Quantitative comparison of referring expression segmentation results [PITH_FULL_IMAGE:figures/full_fig_p013_14.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

95 extracted references · 33 canonical work pages

  1. [1]

    A survey on multimodal large language models,

    S. Yin, C. Fu, S. Zhao, K. Li, X. Sun, T. Xu, and E. Chen, “A survey on multimodal large language models,”National Science Review, vol. 11, no. 12, 2024

    2024

  2. [2]

    Visual instruction tuning,

    H. Liu, C. Li, Q. Wu, and Y . J. Lee, “Visual instruction tuning,” Advances in neural information processing systems, vol. 36, 2024

    2024

  3. [3]

    Deepseek- vl: Towards real-world vision-language understanding,

    H. Lu, W. Liu, B. Zhang, B. Wang, K. Dong, B. Liu, J. Sun, T. Ren, Z. Li, H. Yang, Y . Sun, C. Deng, H. Xu, Z. Xie, and C. Ruan, “Deepseek- vl: Towards real-world vision-language understanding,” 2024

    2024

  4. [4]

    Improved baselines with visual instruction tuning,

    H. Liu, C. Li, Y . Li, and Y . J. Lee, “Improved baselines with visual instruction tuning,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 26 296–26 306

    2024

  5. [5]

    Qwen-vl: A versatile vision-language model for un- derstanding, localization, text reading, and beyond,

    J. Bai, S. Bai, S. Yang, S. Wang, S. Tan, P. Wang, J. Lin, C. Zhou, and J. Zhou, “Qwen-vl: A versatile vision-language model for un- derstanding, localization, text reading, and beyond,”arXiv preprint arXiv:2308.12966, 2023

    arXiv 2023

  6. [6]

    How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites,

    Z. Chen, W. Wang, H. Tian, S. Ye, Z. Gao, E. Cui, W. Tong, K. Hu, J. Luo, Z. Maet al., “How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites,”Science China Information Sciences, vol. 67, no. 12, p. 220101, 2024

    2024

  7. [7]

    Moma: Multimodal llm adapter for fast personalized image generation,

    K. Song, Y . Zhu, B. Liu, Q. Yan, A. Elgammal, and X. Yang, “Moma: Multimodal llm adapter for fast personalized image generation,” in European Conference on Computer Vision. Springer, 2024, pp. 117– 132

    2024

  8. [8]

    Genartist: Multimodal llm as an agent for unified image generation and editing,

    Z. Wang, A. Li, Z. Li, and X. Liu, “Genartist: Multimodal llm as an agent for unified image generation and editing,”Advances in Neural Information Processing Systems, vol. 37, pp. 128 374–128 395, 2024

    2024

Show all 95 references
  1. [9]

    Visionllm: Large language model is also an open-ended decoder for vision-centric tasks,

    W. Wang, Z. Chen, X. Chen, J. Wu, X. Zhu, G. Zeng, P. Luo, T. Lu, J. Zhou, Y . Qiaoet al., “Visionllm: Large language model is also an open-ended decoder for vision-centric tasks,”Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  2. [10]

    Groma: Localized visual tokenization for grounding multimodal large language models,

    C. Ma, Y . Jiang, J. Wu, Z. Yuan, and X. Qi, “Groma: Localized visual tokenization for grounding multimodal large language models,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 417– 435

    2024

  3. [11]

    Towards open vocabulary learning: A survey,

    J. Wu, X. Li, S. Xu, H. Yuan, H. Ding, Y . Yang, X. Li, J. Zhang, Y . Tong, X. Jianget al., “Towards open vocabulary learning: A survey,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 7, pp. 5092–5113, 2024

    2024

  4. [12]

    Next- chat: An lmm for chat, detection and segmentation,

    A. Zhang, L. Zhao, C.-W. Xie, Y . Zheng, W. Ji, and T.-S. Chua, “Next- chat: An lmm for chat, detection and segmentation,”arXiv preprint arXiv:2311.04498, 2023

    2023 arXiv

  5. [13]

    Transformer-based visual segmentation: A survey,

    X. Li, H. Ding, H. Yuan, W. Zhang, J. Pang, G. Cheng, K. Chen, Z. Liu, and C. C. Loy, “Transformer-based visual segmentation: A survey,”IEEE transactions on pattern analysis and machine intelligence, 2024

    2024

  6. [14]

    Smooseg: smoothness prior for unsupervised semantic segmentation,

    M. Lan, X. Wang, Y . Ke, J. Xu, L. Feng, and W. Zhang, “Smooseg: smoothness prior for unsupervised semantic segmentation,”Advances in Neural Information Processing Systems, vol. 36, pp. 11 353–11 373, 2023

    2023

  7. [15]

    Lisa: Reasoning segmentation via large language model,

    X. Lai, Z. Tian, Y . Chen, Y . Li, Y . Yuan, S. Liu, and J. Jia, “Lisa: Reasoning segmentation via large language model,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 9579–9589

    2024

  8. [16]

    Gsva: Generalized segmentation via multimodal large language models,

    Z. Xia, D. Han, Y . Han, X. Pan, S. Song, and G. Huang, “Gsva: Generalized segmentation via multimodal large language models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 3858–3869

    2024

  9. [17]

    Groundhog: Grounding large language models to holistic segmentation,

    Y . Zhang, Z. Ma, X. Gao, S. Shakiah, Q. Gao, and J. Chai, “Groundhog: Grounding large language models to holistic segmentation,” inPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 14 227–14 238

    2024

  10. [18]

    Multi-modal instruction tuned llms with fine-grained visual perception,

    J. He, Y . Wang, L. Wang, H. Lu, J.-Y . He, J.-P. Lan, B. Luo, and X. Xie, “Multi-modal instruction tuned llms with fine-grained visual perception,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 13 980–13 990

    2024

  11. [19]

    Pixellm: Pixel reasoning with large multimodal model,

    Z. Ren, Z. Huang, Y . Wei, Y . Zhao, D. Fu, J. Feng, and X. Jin, “Pixellm: Pixel reasoning with large multimodal model,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 26 374–26 383

    2024

  12. [20]

    Glamm: Pixel grounding large multimodal model,

    H. Rasheed, M. Maaz, S. Shaji, A. Shaker, S. Khan, H. Cholakkal, R. M. Anwer, E. Xing, M.-H. Yang, and F. S. Khan, “Glamm: Pixel grounding large multimodal model,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 13 009–13 018

    2024

  13. [21]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Loet al., “Segment anything,” IEEE TRANSACTIONS, 2025 15 inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 4015–4026

    2025

  14. [22]

    Jack of all tasks master of many: Designing general-purpose coarse-to-fine vision-language model,

    S. Pramanick, G. Han, R. Hou, S. Nag, S.-N. Lim, N. Ballas, Q. Wang, R. Chellappa, and A. Almahairi, “Jack of all tasks master of many: Designing general-purpose coarse-to-fine vision-language model,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2024

  15. [23]

    Florence-2: Advancing a unified representation for a variety of vision tasks,

    B. Xiao, H. Wu, W. Xu, X. Dai, H. Hu, Y . Lu, M. Zeng, C. Liu, and L. Yuan, “Florence-2: Advancing a unified representation for a variety of vision tasks,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 4818–4829

    2024

  16. [24]

    Visionllm v2: An end-to-end generalist multimodal large language model for hundreds of vision-language tasks,

    J. Wu, M. Zhong, S. Xing, Z. Lai, Z. Liu, Z. Chen, W. Wang, X. Zhu, L. Lu, T. Luet al., “Visionllm v2: An end-to-end generalist multimodal large language model for hundreds of vision-language tasks,”Advances in Neural Information Processing Systems, vol. 37, pp. 69 925–69 975, 2024

    2024

  17. [25]

    Text4seg: Reimagining image segmentation as text generation,

    M. Lan, C. Chen, Y . Zhou, J. Xu, Y . Ke, X. Wang, L. Feng, and W. Zhang, “Text4seg: Reimagining image segmentation as text generation,” inThe Thirteenth International Conference on Learning Representations, 2025. [Online]. Available: https://openreview.net/ forum?id=vkakKdznFS

    2025

  18. [26]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” inInternational Conference on Learning ...

    2021

  19. [27]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution,

    P. Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Geet al., “Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution,”arXiv preprint arXiv:2409.12191, 2024

    2024 arXiv

  20. [28]

    Deepseek-vl2: Mixture-of-experts vision-language models for advanced multimodal understanding,

    Z. Wu, X. Chen, Z. Pan, X. Liu, W. Liu, D. Dai, H. Gao, Y . Ma, C. Wu, B. Wanget al., “Deepseek-vl2: Mixture-of-experts vision-language models for advanced multimodal understanding,”arXiv preprint arXiv:2412.10302, 2024

    2024 arXiv

  21. [29]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models,

    J. Zhu, W. Wang, Z. Chen, Z. Liu, S. Ye, L. Gu, Y . Duan, H. Tian, W. Su, J. Shaoet al., “Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models,”arXiv preprint arXiv:2504.10479, 2025

    2025 arXiv

  22. [30]

    Flamingo: a visual language model for few-shot learning,

    J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y . Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynoldset al., “Flamingo: a visual language model for few-shot learning,”Advances in neural information processing systems, vol. 35, pp. 23 716–23 736, 2022

    2022

  23. [31]

    Openflamingo: An open-source framework for training large autoregressive vision-language models,

    A. Awadalla, I. Gao, J. Gardner, J. Hessel, Y . Hanafy, W. Zhu, K. Marathe, Y . Bitton, S. Gadre, S. Sagawaet al., “Openflamingo: An open-source framework for training large autoregressive vision-language models,”arXiv preprint arXiv:2308.01390, 2023

    2023 arXiv

  24. [32]

    Otter: A multi-modal model with in-context instruction tun- ing,

    B. Li, Y . Zhang, L. Chen, J. Wang, F. Pu, J. A. Cahyono, J. Yang, C. Li, and Z. Liu, “Otter: A multi-modal model with in-context instruction tun- ing,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    2025

  25. [33]

    Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models,

    J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models,” inInternational conference on machine learning. PMLR, 2023, pp. 19 730–19 742

    2023

  26. [34]

    InstructBLIP: Towards general-purpose vision-language models with instruction tuning,

    W. Dai, J. Li, D. Li, A. Tiong, J. Zhao, W. Wang, B. Li, P. Fung, and S. Hoi, “InstructBLIP: Towards general-purpose vision-language models with instruction tuning,” inThirty-seventh Conference on Neural Information Processing Systems, 2023. [Online]. Available: https://openre...

    2023

  27. [35]

    mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration,

    Q. Ye, H. Xu, J. Ye, M. Yan, A. Hu, H. Liu, Q. Qian, J. Zhang, and F. Huang, “mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 13 040–13 051

    2024

  28. [36]

    Llava-next: Improved reasoning, ocr, and world knowledge,

    H. Liu, C. Li, Y . Li, B. Li, Y . Zhang, S. Shen, and Y . J. Lee, “Llava-next: Improved reasoning, ocr, and world knowledge,” 2024

    2024

  29. [37]

    Llava-uhd: an lmm perceiving any aspect ratio and high-resolution images,

    Z. Guo, R. Xu, Y . Yao, J. Cui, Z. Ni, C. Ge, T.-S. Chua, Z. Liu, and G. Huang, “Llava-uhd: an lmm perceiving any aspect ratio and high-resolution images,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 390–406

    2024

  30. [38]

    LLaV A-onevision: Easy visual task transfer,

    B. Li, Y . Zhang, D. Guo, R. Zhang, F. Li, H. Zhang, K. Zhang, P. Zhang, Y . Li, Z. Liu, and C. Li, “LLaV A-onevision: Easy visual task transfer,”Transactions on Machine Learning Research, 2025. [Online]. Available: https://openreview.net/forum?id=zKv8qULV6n

    2025

  31. [39]

    Mini-gemini: Mining the potential of multi-modality vision language models,

    Y . Li, Y . Zhang, C. Wang, Z. Zhong, Y . Chen, R. Chu, S. Liu, and J. Jia, “Mini-gemini: Mining the potential of multi-modality vision language models,”arXiv preprint arXiv:2403.18814, 2024

    2024 arXiv

  32. [40]

    Monkey: Image resolution and text label are important things for large multi-modal models,

    Z. Li, B. Yang, Q. Liu, Z. Ma, S. Zhang, J. Yang, Y . Sun, Y . Liu, and X. Bai, “Monkey: Image resolution and text label are important things for large multi-modal models,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 26 763–26 773

    2024

  33. [41]

    Sphinx: A mixer of weights, visual embeddings and image scales for multi-modal large language models,

    Z. Lin, D. Liu, R. Zhang, P. Gao, L. Qiu, H. Xiao, H. Qiu, W. Shao, K. Chen, J. Hanet al., “Sphinx: A mixer of weights, visual embeddings and image scales for multi-modal large language models,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 36–55

    2024

  34. [42]

    Instructseg: Unifying instructed visual segmentation with multi-modal large language models,

    C. Wei, Y . Zhong, H. Tan, Y . Zeng, Y . Liu, Z. Zhao, and Y . Yang, “Instructseg: Unifying instructed visual segmentation with multi-modal large language models,”arXiv preprint arXiv:2412.14006, 2024

    2024 arXiv

  35. [43]

    Expanding performance boundaries of open- source multimodal models with model, data, and test-time scaling,

    Z. Chen, W. Wang, Y . Cao, Y . Liu, Z. Gao, E. Cui, J. Zhu, S. Ye, H. Tian, Z. Liuet al., “Expanding performance boundaries of open- source multimodal models with model, data, and test-time scaling,” arXiv preprint arXiv:2412.05271, 2024

    2024 arXiv

  36. [44]

    Qwen2. 5-vl technical report,

    S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tanget al., “Qwen2. 5-vl technical report,”arXiv preprint arXiv:2502.13923, 2025

    2025 arXiv

  37. [45]

    Omg-llava: Bridging image-level, object-level, pixel-level reasoning and understanding,

    T. Zhang, X. Li, H. Fei, H. Yuan, S. Wu, S. Ji, C. C. Loy, and S. Yan, “Omg-llava: Bridging image-level, object-level, pixel-level reasoning and understanding,”Advances in Neural Information Processing Sys- tems, vol. 37, pp. 71 737–71 767, 2024

    2024

  38. [46]

    MMR: A large-scale benchmark dataset for multi-target and multi-granularity reasoning segmentation,

    D. Jang, Y . Cho, S. Lee, T. Kim, and D. Kim, “MMR: A large-scale benchmark dataset for multi-target and multi-granularity reasoning segmentation,” inThe Thirteenth International Conference on Learning Representations, 2025. [Online]. Available: https: //openreview.net/forum?i...

    2025

  39. [47]

    SegLLM: Multi-round reasoning segmentation with large language models,

    X. Wang, S. Zhang, S. Li, K. Li, K. Kallidromitis, Y . Kato, K. Kozuka, and T. Darrell, “SegLLM: Multi-round reasoning segmentation with large language models,” inThe Thirteenth International Conference on Learning Representations, 2025. [Online]. Available: https://openreview...

    2025

  40. [48]

    Segearth-r1: Geospatial pixel reasoning via large language model,

    K. Li, Z. Xin, L. Pang, C. Pang, Y . Deng, J. Yao, G. Xia, D. Meng, Z. Wang, and X. Cao, “Segearth-r1: Geospatial pixel reasoning via large language model,”arXiv preprint arXiv:2504.09644, 2025

    2025 arXiv

  41. [49]

    Geopix: A multimodal large language model for pixel-level image understanding in remote sensing,

    R. Ou, Y . Hu, F. Zhang, J. Chen, and Y . Liu, “Geopix: A multimodal large language model for pixel-level image understanding in remote sensing,”IEEE Geoscience and Remote Sensing Magazine, 2025

    2025

  42. [50]

    Ufo: A unified approach to fine-grained visual perception via open-ended language interface,

    H. Tang, C. Xie, H. Wang, X. Bao, T. Weng, P. Li, Y . Zheng, and L. Wang, “Ufo: A unified approach to fine-grained visual perception via open-ended language interface,”arXiv preprint arXiv:2503.01342, 2025

    2025

  43. [51]

    Pixel-sail: Single transformer for pixel-grounded understand- ing,

    T. Zhang, X. Li, Z. Huang, Y . Li, W. Lei, X. Deng, S. Chen, S. Ji, and J. Feng, “Pixel-sail: Single transformer for pixel-grounded understand- ing,”arXiv preprint arXiv:2504.10465, 2025

    2025 arXiv

  44. [52]

    Himtok: Learning hierarchical mask tokens for image segmentation with large multimodal model,

    T. Wang, C. Cheng, L. Wang, S. Chen, and W. Zhao, “Himtok: Learning hierarchical mask tokens for image segmentation with large multimodal model,”arXiv preprint arXiv:2503.13026, 2025

    2025 arXiv

  45. [53]

    Alto: Adaptive-length tokenizer for autoregressive mask generation,

    L. Wang, H. Lin, S. Chen, T. Wang, C. Cheng, Y . Zhong, D. Zheng, and W. Zhao, “Alto: Adaptive-length tokenizer for autoregressive mask generation,”arXiv preprint arXiv:2505.16495, 2025

    2025

  46. [54]

    Pix2seq: A language modeling framework for object detection,

    T. Chen, S. Saxena, L. Li, D. J. Fleet, and G. Hinton, “Pix2seq: A language modeling framework for object detection,” inInternational Conference on Learning Representations, 2022. [Online]. Available: https://openreview.net/forum?id=e42KbIw6Wb

    2022

  47. [55]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection,

    S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, Q. Jiang, C. Li, J. Yang, H. Suet al., “Grounding dino: Marrying dino with grounded pre-training for open-set object detection,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 38–55

    2024

  48. [56]

    Universal instance perception as object discovery and retrieval,

    B. Yan, Y . Jiang, J. Wu, D. Wang, P. Luo, Z. Yuan, and H. Lu, “Universal instance perception as object discovery and retrieval,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 15 325–15 336

    2023

  49. [57]

    Grounding multimodal large language models to the world,

    Z. Peng, W. Wang, L. Dong, Y . Hao, S. Huang, S. Ma, Q. Ye, and F. Wei, “Grounding multimodal large language models to the world,” inThe Twelfth International Conference on Learning Representations,

  50. [58]

    Shikra: Unleashing multimodal llm’s referential dialogue magic,

    K. Chen, Z. Zhang, W. Zeng, R. Zhang, F. Zhu, and R. Zhao, “Shikra: Unleashing multimodal llm’s referential dialogue magic,”arXiv preprint arXiv:2306.15195, 2023

    2023 arXiv

  51. [59]

    Vitron: A unified pixel-level vision llm for understanding, generating, segmenting, edit- ing,

    H. Fei, S. Wu, H. Zhang, T.-S. Chua, and S. Yan, “Vitron: A unified pixel-level vision llm for understanding, generating, segmenting, edit- ing,” 2024. IEEE TRANSACTIONS, 2025 16

    2024

  52. [60]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clarket al., “Learning transferable visual models from natural language supervision,” inInternational conference on machine learning. PmLR, 2021, pp. 8748–8763

    2021

  53. [61]

    Sigmoid loss for language image pre-training,

    X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer, “Sigmoid loss for language image pre-training,” inProceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 11 975–11 986

    2023

  54. [62]

    Qwen3 technical report,

    A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lvet al., “Qwen3 technical report,”arXiv preprint arXiv:2505.09388, 2025

    2025 arXiv

  55. [63]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azharet al., “Llama: Open and efficient foundation language models,”arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  56. [64]

    Referitgame: Referring to objects in photographs of natural scenes,

    S. Kazemzadeh, V . Ordonez, M. Matten, and T. Berg, “Referitgame: Referring to objects in photographs of natural scenes,” inProceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, pp. 787–798

    2014

  57. [65]

    Run-length encodings (corresp.),

    S. Golomb, “Run-length encodings (corresp.),”IEEE transactions on information theory, vol. 12, no. 3, pp. 399–401, 1966

    1966

  58. [66]

    LoRA: Low-rank adaptation of large language models,

    E. J. Hu, yelong shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” inInternational Conference on Learning Representations, 2022. [Online]. Available: https://openreview.net/forum?id=nZeVKeeFYf9

    2022

  59. [67]

    SAMRefiner: Taming segment anything model for universal mask refinement,

    Y . Lin, H. Li, W. Shao, Z. Yang, J. Zhao, X. He, P. Luo, and K. Zhang, “SAMRefiner: Taming segment anything model for universal mask refinement,” inThe Thirteenth International Conference on Learning Representations, 2025. [Online]. Available: https://openreview.net/forum?id=...

    2025

  60. [68]

    Swift: A scalable lightweight infrastructure for fine-tuning,

    Y . Zhao, J. Huang, J. Hu, X. Wang, Y . Mao, D. Zhang, Z. Jiang, Z. Wu, B. Ai, A. Wang, W. Zhou, and Y . Chen, “Swift: A scalable lightweight infrastructure for fine-tuning,” inAAAI, 2025, pp. 29 733–29 735. [Online]. Available: https://doi.org/10.1609/aaai.v39i28.35383

    2025 doi

  61. [69]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in International Conference on Learning Representations, 2019. [Online]. Available: https://openreview.net/forum?id=Bkg6RiCqY7

    2019

  62. [70]

    Zero: Memory optimizations toward training trillion parameter models,

    S. Rajbhandari, J. Rasley, O. Ruwase, and Y . He, “Zero: Memory optimizations toward training trillion parameter models,” inSC20: International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, 2020, pp. 1–16

    2020

  63. [71]

    Phrasecut: Language- based image segmentation in the wild,

    C. Wu, Z. Lin, S. Cohen, T. Bui, and S. Maji, “Phrasecut: Language- based image segmentation in the wild,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 10 216–10 225

    2020

  64. [72]

    Gres: Generalized referring expression segmentation,

    C. Liu, H. Ding, and X. Jiang, “Gres: Generalized referring expression segmentation,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 23 592–23 601

    2023

  65. [73]

    Generation and comprehension of unambiguous object descriptions,

    J. Mao, J. Huang, A. Toshev, O. Camburu, A. L. Yuille, and K. Murphy, “Generation and comprehension of unambiguous object descriptions,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 11–20

    2016

  66. [74]

    Polyformer: Referring image segmentation as sequential polygon generation,

    J. Liu, H. Ding, Z. Cai, Y . Zhang, R. K. Satzoda, V . Mahadevan, and R. Manmatha, “Polyformer: Referring image segmentation as sequential polygon generation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 18 653–18 663

    2023

  67. [75]

    Language-aware vision transformer for referring segmentation,

    Z. Yang, J. Wang, X. Ye, Y . Tang, K. Chen, H. Zhao, and P. H. Torr, “Language-aware vision transformer for referring segmentation,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  68. [76]

    Sam4mllm: Enhance multi-modal large language model for referring expression segmentation,

    Y .-C. Chen, W.-H. Li, C. Sun, Y .-C. F. Wang, and C.-S. Chen, “Sam4mllm: Enhance multi-modal large language model for referring expression segmentation,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 323–340

    2024

  69. [77]

    Segagent: Exploring pixel understanding capabilities in mllms by imitating human annotator trajectories,

    M. Zhu, Y . Tian, H. Chen, C. Zhou, Q. Guo, Y . Liu, M. Yang, and C. Shen, “Segagent: Exploring pixel understanding capabilities in mllms by imitating human annotator trajectories,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 3686– 3696

    2025

  70. [78]

    Popen: Preference-based optimization and ensemble for lvlm-based reasoning segmentation,

    L. Zhu, T. Chen, Q. Xu, X. Liu, D. Ji, H. Wu, D. W. Soh, and J. Liu, “Popen: Preference-based optimization and ensemble for lvlm-based reasoning segmentation,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 30 231–30 240

    2025

  71. [79]

    Microsoft coco: Common objects in context,

    T.-Y . Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll ´ar, and C. L. Zitnick, “Microsoft coco: Common objects in context,” inComputer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part v 13. Springer, 2...

    2014

  72. [80]

    Pix2cap-coco: Ad- vancing visual comprehension via pixel-level captioning,

    Z. You, J. Wang, L. Kong, B. He, and Z. Wu, “Pix2cap-coco: Ad- vancing visual comprehension via pixel-level captioning,”arXiv preprint arXiv:2501.13893, 2025

    2025 arXiv

  73. [81]

    Rotated multi-scale interaction network for referring remote sensing image seg- mentation,

    S. Liu, Y . Ma, X. Zhang, H. Wang, J. Ji, X. Sun, and R. Ji, “Rotated multi-scale interaction network for referring remote sensing image seg- mentation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 26 658–26 668

    2024

  74. [82]

    Seg- zero: Reasoning-chain guided segmentation via cognitive reinforce- ment,

    Y . Liu, B. Peng, Z. Zhong, Z. Yue, F. Lu, B. Yu, and J. Jia, “Seg- zero: Reasoning-chain guided segmentation via cognitive reinforce- ment,”arXiv preprint arXiv:2503.06520, 2025

    2025 arXiv

  75. [83]

    Clearclip: Decomposing clip representations for dense vision-language inference,

    M. Lan, C. Chen, Y . Ke, X. Wang, L. Feng, and W. Zhang, “Clearclip: Decomposing clip representations for dense vision-language inference,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 143– 160

    2024

  76. [84]

    Proxyclip: Proxy attention improves clip for open-vocabulary segmentation,

    ——, “Proxyclip: Proxy attention improves clip for open-vocabulary segmentation,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 70–88

    2024

  77. [85]

    Open-vocabulary universal image seg- mentation with maskclip,

    Z. Ding, J. Wang, and Z. Tu, “Open-vocabulary universal image seg- mentation with maskclip,” inProceedings of the 40th International Conference on Machine Learning, ser. ICML’23. JMLR.org, 2023

    2023

  78. [86]

    Groupvit: Semantic segmentation emerges from text supervision,

    J. Xu, S. De Mello, S. Liu, W. Byeon, T. Breuel, J. Kautz, and X. Wang, “Groupvit: Semantic segmentation emerges from text supervision,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 18 134–18 144

    2022

  79. [87]

    Open-vocabulary semantic segmentation with mask-adapted clip,

    F. Liang, B. Wu, X. Dai, K. Li, Y . Zhao, H. Zhang, P. Zhang, P. Vajda, and D. Marculescu, “Open-vocabulary semantic segmentation with mask-adapted clip,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 7061–7070

    2023

  80. [88]

    San: side adapter network for open-vocabulary semantic segmentation,

    M. Xu, Z. Zhang, F. Wei, H. Hu, and X. Bai, “San: side adapter network for open-vocabulary semantic segmentation,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 12, pp. 15 546– 15 561, 2023

    2023

  81. [89]

    Lasagna: Language- based segmentation assistant for complex queries,

    C. Wei, H. Tan, Y . Zhong, Y . Yang, and L. Ma, “Lasagna: Language- based segmentation assistant for complex queries,”arXiv preprint arXiv:2404.08506, 2024

    2024 arXiv

  82. [90]

    Geoground: A unified large vision-language model. for remote sensing visual grounding,

    Y . Zhou, M. Lan, X. Li, Y . Ke, X. Jiang, L. Feng, and W. Zhang, “Geoground: A unified large vision-language model. for remote sensing visual grounding,”arXiv preprint arXiv:2411.11904, 2024

    2024 arXiv

  83. [91]

    Semantic understanding of scenes through the ade20k dataset,

    B. Zhou, H. Zhao, X. Puig, T. Xiao, S. Fidler, A. Barriuso, and A. Torralba, “Semantic understanding of scenes through the ade20k dataset,”International Journal of Computer Vision, vol. 127, pp. 302– 321, 2019

    2019

  84. [92]

    The role of context for object detection and semantic segmentation in the wild,

    R. Mottaghi, X. Chen, X. Liu, N.-G. Cho, S.-W. Lee, S. Fidler, R. Urtasun, and A. Yuille, “The role of context for object detection and semantic segmentation in the wild,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 891– 898

    2014

  85. [93]

    The pascal visual object classes challenge 2007,

    M. Everingham, “The pascal visual object classes challenge 2007,” inhttp://www. pascal-network. org/challenges/VOC/voc2007/workshop/index. html, 2009

    2007

  86. [94]

    Side adapter network for open-vocabulary semantic segmentation,

    M. Xu, Z. Zhang, F. Wei, H. Hu, and X. Bai, “Side adapter network for open-vocabulary semantic segmentation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 2945–2954

    2023

  87. [2024]

    Available: https://openreview.net/forum?id=lLmqxkfSIw

    [Online]. Available: https://openreview.net/forum?id=lLmqxkfSIw

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.