Pith. sign in

REVIEW 3 major objections 6 minor 50 references

Personalized OVSS: Understanding Personal Concept in Open-Vocabulary Semantic Segmentation

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that adding a negative mask and visual-embedding injection to text prompt tuning lets open-vocabulary segmentation models learn a user's personal object from a few image-mask pairs without losing original accuracy.

desk verdict A genuinely new task and a simple plug-in that works, but the 'without compromising' claim is only half-measured; worth a serious referee with revisions required. read the letter →

arxiv 2507.11030 v1 pith:LXHZ6GQZ submitted 2025-07-15 cs.CV

Sunghyun Park , Jungsoo Lee , Shubhankar Borse , Munawar Hayat , Sungha Choi , Kyuwoong Hwang , Fatih Porikli This is my paper · ORCID
classification cs.CV
keywords personalizedopen-vocabularysemanticsegmentationtextprompttuningnegativemaskproposalvisualembeddinginjectionfew-shotCLIPinstance-levelpersonalization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Open-vocabulary semantic segmentation (OVSS) models can label arbitrary classes described in text, but they cannot learn which particular object is 'mine'—the mug, bird, or dog that belongs to a specific user. This paper defines the task of personalized OVSS: given a few pairs of images and masks of one personal concept, the model must segment that concept when it appears among other objects of the same class, while keeping its original open-vocabulary accuracy. The proposed plug-in method combines text prompt tuning with a negative mask proposal and visual-embedding injection, and it is evaluated on three new benchmarks built from FSS-1000, CUB-200, and ADE-20K. The paper reports higher personalized IoU on both SAN and ODISE backbones with roughly unchanged mIoU, which is what the task's dual goal requires.

What carries the argument

The load-bearing mechanism is the negative mask proposal. The paper adds a learnable linear layer $W_Z$ that combines the existing mask embeddings into one negative embedding $Z_{\text{neg}}$ and a learnable convolution $W_M$ that combines the existing mask proposals into one negative mask $M_{\text{neg}}$. The negative embedding is trained with a loss that spreads probability over all classes except the personal concept, and the negative mask is trained against the complement $1 - M_{\text{gt}}$ of the personal ground-truth mask. A second mechanism, visual embedding injection, replaces the learnable textual prompt $T_{\text{per}}$ with the interpolation $\alpha \cdot F_{\text{per}} + (1-\alpha) \cdot T_{\text{per}}$, where $F_{\text{per}}$ is the CLIP feature of the personal-concept pixels averaged under the ground-truth mask. Together they enforce the paper's core observation: reducing false positives is what makes prompt tuning safe for personal concepts.

What would settle it

Recompute mIoU on FSS$^\text{per}$ and CUB$^\text{per}$ using human-annotated ground-truth labels for every open-vocabulary class instead of treating SAN or ODISE predictions as ground truth; if the gap between personalized and unpersonalized mIoU widens beyond the paper's reported near-flat differences, the 'maintains original performance' claim fails. A second check: run the method on pairs of near-identical instances of the same class and count how often the wrong instance is labelled as the personal concept.

Watch

Extended reading notes

Core claim

The central claim is that a personal visual concept can be grafted onto a pretrained OVSS model by tuning a single learnable textual embedding for the concept, adding a negative mask proposal that learns everything except the concept, and interpolating that textual embedding with a mask-averaged visual embedding of the concept. The negative mask proposal is the key correction: it consists of one extra mask embedding and one extra mask, supervised respectively to spread probability uniformly over all non-personal classes and to predict the complement of the personal ground-truth mask. This counteracts the false positives that plain text prompt tuning produces when another instance of the same class appears. Across the new FSS$^\text{per}$, CUB$^\text{per}$, and ADE$^\text{per}$ benchmarks, the method raises SAN's average IoU$^\text{per}$ from 41.08 to 53.56 on FSS$^\text{per}$ and from 6.88 to 22.67 on ADE$^\text{per}$, with mIoU nearly flat, and it also improves ODISE. The paper frames this as the first task definition, method, and benchmark suite for personalized OVSS.

Load-bearing premise

The evaluation of 'maintaining original OVSS performance' on FSS$^\text{per}$ and CUB$^\text{per}$ assumes that the open-vocabulary model's own predictions are the correct labels for all non-personal classes, so if those predictions are biased or wrong, the mIoU claim is not actually measuring the model's true accuracy.

Editorial extensions

If this is right

  • The plug-in works on two structurally different OVSS backbones, SAN and ODISE, and improves personalized IoU even with a single image-mask pair, so the mechanism does not depend on a particular architecture.
  • Because the negative mask is trained to fire on everything outside the personal concept, the method avoids the labor-intensive collection of hard-negative images of the same class that earlier personalization approaches required.
  • With only about 0.4M trainable parameters and 200 tuning iterations, personalization is cheap enough to be run per user or per device rather than as a global retraining step.
  • A personalized prompt can be composed with ordinary text prompts, so the same learned concept can be used to describe things like 'hat on my dog', extending reference from the object to its parts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's mIoU figures on FSS$^\text{per}$ and CUB$^\text{per}$ treat the pretrained OVSS model's own predictions as ground truth, so a stricter human-annotated evaluation could either confirm or overturn the claim that original performance is maintained.
  • The negative-mask idea is a general cure for the overconfidence that comes with learning an individual instance; it could transfer to referring expression segmentation or personalized visual question answering wherever same-class distractors dominate the errors.
  • The benchmarks mostly contain visually distinguishable classes, so a natural stress test is near-duplicate instances of the same object, such as two identical mugs; the paper's concat-dataset experiment hints at this but does not cover true near-duplicates.
  • The method requires masks at personalization time; replacing them with clicks or boxes from an interactive segmenter and measuring the IoU$^\text{per}$ drop would show how far the plug-in can go with weaker supervision.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces a new task, personalized open-vocabulary semantic segmentation (OVSS), in which a model must segment a user-provided personal concept (e.g., "my mug cup") in addition to performing standard open-vocabulary segmentation. To address this task, the authors propose a plug-in method built on text prompt tuning, augmented with a negative mask proposal that captures non-personal concepts and with visual embeddings injected into the textual prompt. The method is applied to two existing OVSS models (SAN and ODISE) and evaluated on three newly constructed benchmarks: FSS^per, CUB^per, and ADE^per, built from FSS-1000, CUB-200, and ADE-20K respectively. The paper reports consistent improvements in IoU^per across backbones and datasets, and claims that the original OVSS mIoU is maintained.

Significance. The task is well motivated and timely: existing OVSS models cannot distinguish a user-specific instance from other objects of the same category, and few-shot segmentation methods lack open-vocabulary flexibility. The proposed method is simple, parameter-efficient (about 0.4M trainable parameters), and architecture-agnostic, and the ablation study in Table 2 clearly isolates the contribution of each component. The paper also provides a useful set of benchmarks for a new task. However, the evaluation of the "maintaining OVSS performance" half of the central claim is weakened by the use of pseudo-labels from the pretrained model itself on two of the three benchmarks, and the comparison with few-shot methods in Table 4 contains a mislabeled experimental setting. These issues are fixable and do not undermine the core method's potential, but they do affect the strength of the stated claims.

major comments (3)
  1. [Sec. 4.1 (Evaluation Metrics); Supp. A.2] The mIoU metric on FSS^per and CUB^per is computed against the pretrained OVSS model's own predictions as pseudo ground truth (stated in Sec. 4.1 and again in Supp. A.2). Since the baseline in Table 1 is the same pretrained model, these mIoU scores primarily measure agreement with the unpersonalized model, not open-vocabulary segmentation accuracy. A method that leaves all non-personal predictions unchanged would score highly by construction, so the claim in Sec. 4.2 that "our method maintains mIoU" is not established for two of the three benchmarks. On ADE^per, which has real labels, Table 1 shows a small drop for ODISE (12.22 to 12.19); without error bars this does not support an unconditional "without compromising" statement. Please either evaluate mIoU on a benchmark with human labels, or explicitly restrict the preservation claim to datasets with real labels and report variance.
  2. [Table 4] The comparison with few-shot methods is mislabeled. Table 4's caption says "CUBper (K = 1)" but the reported "Ours" value (76.80) matches the K=5 row in Table 1; the corresponding K=1 value is 76.70. This makes the claimed advantage over SEEM and SegGPT at K=1 inaccurate. Please correct the table or clarify the experimental setup, and if the comparison is at K=5, also report the baseline methods' performance at K=5.
  3. [Table 1 and Supp. A.1] The quantitative results lack error bars or multiple runs, and hyperparameters (alpha, lambda_neg_M, lambda_neg_Z, learning rates) are tuned separately per dataset and per backbone, as listed in Supp. A.1. For small differences such as the ODISE ADE^per mIoU change (12.22 to 12.19), it is impossible to tell whether the method preserves performance or slightly degrades it. Please report standard deviations over multiple random support-set selections and state whether the chosen hyperparameters are constant across classes or selected per dataset; a sensitivity analysis would help support the claim of consistent improvement.
minor comments (6)
  1. [Title] The title contains an erroneous space in "Open-V ocabulary"; it should be "Open-Vocabulary".
  2. [Supp. B.2, Fig. 11 caption] The caption reads "SAN wihtout personalization"; "wihtout" should be "without".
  3. [Supp. B.1] The text says "we select α = 0.1 since it achives the best IoUper"; "achives" should be "achieves".
  4. [Supp. B.4, Table 7 caption] The caption uses "Quantative results"; it should be "Quantitative results".
  5. [Eq. (6)] The normalization in Eq. (6) is hard to read: the denominator is written with a sum over 1(M'_gt = 1) but with a leading "1" that appears to be an indicator; please clarify the notation explicitly.
  6. [Sec. 4.2] In the sentence "by average, we improve IoUper of SAN by ...", the phrase "by average" should be "on average".

Circularity Check

1 steps flagged · score 6.0 of 10

mIoU on FSSper and CUBper is scored against the evaluated backbone's own predictions, making the 'maintains original OVSS performance' claim self-referential on two of three benchmarks.

  1. self definitional [Section 4.1 (Evaluation Metrics); used in Section 4.2, Table 1]
    "However, FSS-1000 and CUB-200 do not include the ground truth labels for open-vocabulary classes, so we assume the predictions of pretrained open-vocabulary segmentation models as the ground truth labels for calculating mIoU for FSSper and CUBper."

    The mIoU numbers on FSSper and CUBper are computed by comparing the personalized model's open-vocabulary class predictions with the unpersonalized pretrained model's own predictions, which is the same model used as the Table 1 baseline. 'Maintaining the original OVSS performance' therefore reduces, on these two benchmarks, to 'agreeing with the pretrained model's outputs'; it does not measure open-vocabulary segmentation accuracy against real labels. A plug-in that leaves non-personal predictions unchanged trivially preserves this score. On ADEper, the only benchmark with genuine open-vocabulary labels, Table 1 shows ODISE mIoU drops (23.86 to 22.48, 47.48 to 44.99, 12.22 to 12.19), so the abstract's unconditional 'without compromising' claim is not independently established.

full rationale

The central personalized-segmentation improvement (IoUper) is evaluated against real foreground masks and is independent of the method's own outputs; the training losses are standard supervised objectives and no fitted parameter is renamed as a prediction. There is no load-bearing self-citation chain and no imported uniqueness theorem. The circular element is confined to the mIoU half of the claim on FSSper and CUBper, where the pretrained OVSS model's own predictions serve as ground truth while the same model is the baseline in Table 1. Because a method that leaves the original model's non-personal outputs unchanged would automatically score at the baseline mIoU, 'maintaining original OVSS performance' on those two benchmarks is partly definitional rather than an external measure of accuracy. The ADEper results with real labels further show that the unconditional 'without compromising' statement is not fully supported, especially for ODISE. Overall, the paper's primary contribution—recognizing personal concepts—is empirically credible, but the preservation claim is partially circular and should be scored as partial circularity rather than as a fully independent validation.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method depends on several hyperparameters chosen per dataset and on evaluation assumptions (pseudo-ground-truth, curated class selection). No new physical or conceptual entities are introduced; the negative mask is a learned network component. The learnable parameters T_per, W_M, W_Z are trained, not free in the evaluation sense.

free parameters (5)
  • alpha = 0.1 (FSS^per, CUB^per), 0.01 (ADE^per)
    Interpolation weight between masked visual embedding and learnable textual embedding (Eq. 7). Tuned per dataset via hyperparameter sweep (Fig. 10).
  • lambda_neg_M = 10 (FSS, SAN/ODISE), 500 (CUB, both models), 1 (ADE, ODISE), 10 (ADE, SAN)
    Weight for the negative mask BCE loss (Eq. 5). Selected via sweep, chosen when performance saturates (Fig. 10).
  • lambda_neg_Z = 0.1
    Weight for the negative mask embedding loss (Eq. 3). Chosen as the best IoU^per value from the sweep.
  • learning_rate (SAN) = 5e-4
    Learning rate for T_per, W_M, W_Z when using SAN backbone.
  • learning_rate (ODISE) = 2e-3 (FSS^per), 1e-4 (CUB^per, ADE^per)
    Learning rate for ODISE experiments, set per dataset.
assumptions (4)
  • domain assumption Masked average visual features represent the personal concept well enough for prompt enrichment (Eq. 6).
    The method injects F_per, the average of CLIP/Stable Diffusion features under the mask, into the text prompt. If the mask is imperfect or the feature is dominated by background, the injection degrades performance.
  • domain assumption Pseudo-ground-truth labels from the pretrained OVSS model are valid for measuring mIoU on FSS^per and CUB^per.
    Section 4.1 states that FSS-1000 and CUB-200 lack open-vocabulary labels, so mIoU is computed against the base model's own predictions. These pseudo-labels may be inaccurate and can bias the forgetting measurement.
  • domain assumption The selected 30 classes per dataset and the equal positive/negative test split are representative of the personalized OVSS task.
    Class selection is manual (Supplementary Table 5) and chosen as 'challenging to recognize based solely on text descriptions', which may favor the method. Results may not generalize to all concepts or to natural positive/negative ratios.
  • domain assumption The text embedding space of CLIP/Stable Diffusion is shared, and prompt tuning in this space preserves the original open-vocabulary capabilities.
    The plug-in assumes that adding a learnable prompt does not distort the original vocabulary embeddings. The paper checks this only through mIoU on the same curated data distribution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Personalized OVSS: Understanding Personal Concept in Open-Vocabulary Semantic Segmentation." pith.science (2026). https://pith.science/paper/LXHZ6GQZ

@misc{pith2026250711030,
  author       = {Pith},
  title        = {Pith review of: Personalized OVSS: Understanding Personal Concept in Open-Vocabulary Semantic Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LXHZ6GQZ}},
  note         = {Machine review of arXiv:2507.11030}
}
abstract

While open-vocabulary semantic segmentation (OVSS) can segment an image into semantic regions based on arbitrarily given text descriptions even for classes unseen during training, it fails to understand personal texts (e.g., `my mug cup') for segmenting regions of specific interest to users. This paper addresses challenges like recognizing `my mug cup' among `multiple mug cups'. To overcome this challenge, we introduce a novel task termed \textit{personalized open-vocabulary semantic segmentation} and propose a text prompt tuning-based plug-in method designed to recognize personal visual concepts using a few pairs of images and masks, while maintaining the performance of the original OVSS. Based on the observation that reducing false predictions is essential when applying text prompt tuning to this task, our proposed method employs `negative mask proposal' that captures visual concepts other than the personalized concept. We further improve the performance by enriching the representation of text prompts by injecting visual embeddings of the personal concept into them. This approach enhances personalized OVSS without compromising the original OVSS performance. We demonstrate the superiority of our method on our newly established benchmarks for this task, including FSS$^\text{per}$, CUB$^\text{per}$, and ADE$^\text{per}$.

Figures

Figures reproduced from arXiv: 2507.11030 by the authors.

Figure 1
Figure 1. Description of personalized open-vocabulary semantic segmentation (personalized OVSS). (a) While existing OVSS models [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. How personalized open-vocabulary semantic segmentation (personalized OVSS) differentiates with existing tasks. Few-shot [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Increased false positives with text prompt tuning. Light [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Overview of our proposed method. (a) We additionally use a learnable textual embedding [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Qualitative results of personalized OVSS on test samples from COCO dataset. While existing OVSS models simply recognize [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Segmentation results on CUB-200. We show the effect of each module in our method. Visual embedding injection alone identifies [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Visualization of negative mask proposal. The white and [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 9
Figure 9. Figure 9: Examples of segmentation maps we used in ADE [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Hyperparameter ablation studies on CUB-200 dataset. [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Additional segmentation results on CUBper. While SAN wihtout personalization fails to capture the personal visual concept (i.e. my bird), our method applied to SAN recognizes it. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_11.png]
Figure 12
Figure 12. Figure 12: Test images of concat dataset (FSSper). These datasets are used for evaluating the performance on distinguishing between the target visual concept (e.g., “my boat”, “my bird”) and its corresponding similar classes (e.g., “boat”, “bird”) within the same image. signific…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 47 canonical work pages

  1. [1]

    Ex- ploiting a joint embedding space for generalized zero-shot semantic segmentation

    Donghyeon Baek, Youngmin Oh, and Bumsub Ham. Ex- ploiting a joint embedding space for generalized zero-shot semantic segmentation. In ICCV, 2021. 3

    work page 2021

  2. [2]

    Towards in-context scene understanding

    Ivana Balazevic, David Steiner, Nikhil Parthasarathy, Relja Arandjelovic, and Olivier J Henaff. Towards in-context scene understanding. In NeurIPS, 2023. 3

    work page 2023

  3. [3]

    Zero-shot semantic segmentation

    Maxime Bucher, Tuan-Hung VU, Matthieu Cord, and Patrick P´erez. Zero-shot semantic segmentation. InNeurIPS,

    work page

  4. [4]

    Coco- stuff: Thing and stuff classes in context

    Holger Caesar, Jasper Uijlings, and Vittorio Ferrari. Coco- stuff: Thing and stuff classes in context. In CVPR, 2018. 5, 6

    work page 2018

  5. [5]

    Encoder-decoder with atrous separable convolution for semantic image segmentation

    Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV), pages 801–818, 2018. 1

    2018

  6. [6]

    Distribution-aware prompt tuning for vision-language mod- els

    Eulrang Cho, Jooyeon Kim, and Hyunwoo J Kim. Distribution-aware prompt tuning for vision-language mod- els. In ICCV, 2023. 5

    work page 2023

  7. [7]

    this is my unicorn, fluffy

    Niv Cohen, Rinon Gal, Eli A Meirom, Gal Chechik, and Yuval Atzmon. “this is my unicorn, fluffy”: Personalizing frozen vision-language representations. In European confer- ence on computer vision, pages 558–577. Springer, 2022. 3

    work page 2022

  8. [8]

    De- coupling zero-shot semantic segmentation

    Jian Ding, Nan Xue, Gui-Song Xia, and Dengxin Dai. De- coupling zero-shot semantic segmentation. In CVPR, 2022. 2, 3

    work page 2022

Show all 50 references
  1. [9]

    Context-aware feature generation for zero- shot semantic segmentation

    Zhangxuan Gu, Siyuan Zhou, Li Niu, Zihan Zhao, and Liqing Zhang. Context-aware feature generation for zero- shot semantic segmentation. In ACM International Confer- ence on Multimedia, 2020. 3

    2020

  2. [10]

    knn-clip: Retrieval enables training-free segmentation on continually expanding large vocabularies

    Zhongrui Gui, Shuyang Sun, Runjia Li, Jianhao Yuan, Zhao- chong An, Karsten Roth, Ameya Prabhu, and Philip Torr. knn-clip: Retrieval enables training-free segmentation on continually expanding large vocabularies. CoRR, 2024. 3, 7

    2024

  3. [11]

    A strong baseline for generalized few-shot semantic segmentation

    Sina Hajimiri, Malik Boudiaf, Ismail Ben Ayed, and Jose Dolz. A strong baseline for generalized few-shot semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11269– 11278, 2023. 2, 8

    2023

  4. [12]

    Tcp: Textual-based class-aware prompt tuning for visual-language model

    Changsheng Xu Hantao Yao, Rui Zhang. Tcp: Textual-based class-aware prompt tuning for visual-language model. In CVPR, 2024. 3

    2024

  5. [13]

    V oP: Text-video co- operative prompt tuning for cross-modal retrieval

    Siteng Huang, Biao Gong, Yulin Pan, Jianwen Jiang, Yil- iang Lv, Yuyuan Li, and Donglin Wang. V oP: Text-video co- operative prompt tuning for cross-modal retrieval. In CVPR,

  6. [14]

    Huynh, J

    D. Huynh, J. Kuen, Z. Lin, J. Gu, and E. Elhamifar. Open- vocabulary instance segmentation via robust cross-modal pseudo-labeling. In CVPR, 2022. 3

    2022

  7. [15]

    tsf: Transformer-based semantic filter for few-shot learning

    Lai Jinxiang, Yang Siqian, Liu Wenlong, Zeng Yi, Huang Zhongyi, Wu Wenlong, Liu Jun, Gao Bin-Bin, and Wang Chengjie. tsf: Transformer-based semantic filter for few-shot learning. In ECCV, 2022. 2, 8

    2022

  8. [16]

    SLime: Segment like me

    Aliasghar Khani, Saeid Asgari, Aditya Sanghi, Ali Mahdavi Amiri, and Ghassan Hamarneh. SLime: Segment like me. In ICLR, 2024. 3

    2024

  9. [17]

    Maple: Multi-modal prompt learning

    Muhammad Uzair khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. Maple: Multi-modal prompt learning. In CVPR, 2023. 3, 5

    2023

  10. [18]

    Self-regulating prompts: Foundational model adaptation without forgetting

    Muhammad Uzair Khattak, Syed Talal Wasim, Muzammal Naseer, Salman Khan, Ming-Hsuan Yang, and Fahad Shah- baz Khan. Self-regulating prompts: Foundational model adaptation without forgetting. In ICCV, 2023. 5

    2023

  11. [19]

    Berg, Wan-Yen Lo, Piotr Dollar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Dollar, and Ross Girshick. Segment anything. In ICCV, 2023. 3

    2023

  12. [20]

    Language-driven semantic seg- mentation

    Boyi Li, Kilian Q Weinberger, Serge Belongie, Vladlen Koltun, and Rene Ranftl. Language-driven semantic seg- mentation. In ICLR, 2022. 3

    2022

  13. [21]

    Fss-1000: A 1000-class dataset for few- shot segmentation

    Xiang Li, Tianhan Wei, Yau Pun Chen, Yu-Wing Tai, and Chi-Keung Tang. Fss-1000: A 1000-class dataset for few- shot segmentation. In CVPR, 2020. 3, 5

    2020

  14. [22]

    Visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. 4

    2023

  15. [23]

    Matcher: Segment anything with one shot using all-purpose feature matching

    Yang Liu, Muzhi Zhu, Hengtao Li, Hao Chen, Xinlong Wang, and Chunhua Shen. Matcher: Segment anything with one shot using all-purpose feature matching. In ICLR, 2024. 3

    2024

  16. [24]

    Simpler is better: Few-shot semantic segmenta- tion with classifier weight transformer

    Zhihe Lu, Sen He, Xiatian Zhu, Li Zhang, Yi-Zhe Song, and Tao Xiang. Simpler is better: Few-shot semantic segmenta- tion with classifier weight transformer. In ICCV, 2021. 2, 8

    2021

  17. [25]

    SegCLIP: Patch aggregation with learn- able centers for open-vocabulary semantic segmentation

    Huaishao Luo, Junwei Bao, Youzheng Wu, Xiaodong He, and Tianrui Li. SegCLIP: Patch aggregation with learn- able centers for open-vocabulary semantic segmentation. In ICML, 2023. 2

    2023

  18. [26]

    Bagdanov, and Joost van de Weijer

    Marc Masana, Xialei Liu, Bartłomiej Twardowski, Mikel Menta, Andrew D. Bagdanov, and Joost van de Weijer. Class-incremental learning: Survey and performance evalu- ation on image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020. 8

    2020

  19. [27]

    Alvarez, Zuxuan Wu, and Yu-Gang Jiang

    Lingchen Meng, Shiyi Lan, Hengduo Li, Jose M. Alvarez, Zuxuan Wu, and Yu-Gang Jiang. Segic: Unleashing the emergent correspondence for in-context segmentation, 2024. 3

    2024

  20. [28]

    Es- sentials for class incremental learning

    Sudhanshu Mittal, Silvio Galesso, and Thomas Brox. Es- sentials for class incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2021. 8

    2021

  21. [29]

    Yo’llava: Your personalized lan- guage and vision assistant, 2024

    Thao Nguyen, Haotian Liu, Yuheng Li, Mu Cai, Utkarsh Ojha, and Yong Jae Lee. Yo’llava: Your personalized lan- guage and vision assistant, 2024. 2, 4

    2024

  22. [30]

    Maxime Oquab, Timoth ´ee Darcet, Theo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Rus- sell Howes, Po-Yao Huang, Hu Xu, Vasu Sharma, Shang- Wen Li, Wojciech Galuba, Mike Rabbat, Mido Assran, Ni...

    2023

  23. [31]

    Freeseg: Unified, universal and open-vocabulary im- age segmentation

    Jie Qin, Jie Wu, Pengxiang Yan, Ming Li, Ren Yuxi, Xue- feng Xiao, Yitong Wang, Rui Wang, Shilei Wen, Xin Pan, et al. Freeseg: Unified, universal and open-vocabulary im- age segmentation. In CVPR, 2023. 3

    2023

  24. [32]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In ICML, 2021. 1, 2, 3, 11

    2021

  25. [33]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, 2022. 3, 5, 11

    2022

  26. [34]

    C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. Caltech-ucsd birds 200. Technical report, California Institute of Technology, 2011. 5

    2011

  27. [35]

    Learning to learn better visual prompts

    Fengxiang Wang, Wanrong Huang, Shaowu Yang, Qi Fan, and Long Lan. Learning to learn better visual prompts. 2024. 3

    2024

  28. [36]

    Seggpt: Segmenting ev- erything in context

    Xinlong Wang, Xiaosong Zhang, Yue Cao, Wen Wang, Chunhua Shen, and Tiejun Huang. Seggpt: Segmenting ev- erything in context. In ICCV, 2023. 3, 7, 8

    2023

  29. [37]

    Semantic projection network for zero- and few-label semantic segmentation

    Yongqin Xian, Subhabrata Choudhury, Yang He, Bernt Schiele, and Zeynep Akata. Semantic projection network for zero- and few-label semantic segmentation. In CVPR, 2019. 1, 2, 3

    2019

  30. [38]

    Open-vocabulary panop- tic segmentation with text-to-image diffusion models

    Jiarui Xu, Sifei Liu, Arash Vahdat, Wonmin Byeon, Xiao- long Wang, and Shalini De Mello. Open-vocabulary panop- tic segmentation with text-to-image diffusion models. In CVPR, 2023. 1, 3, 4, 5, 6, 11, 14

    2023

  31. [39]

    A simple baseline for open vocabulary semantic segmentation with pre-trained vision- language model

    Mengde Xu, Zheng Zhang, Fangyun Wei, Yutong Lin, Yue Cao, Han Hu, and Xiang Bai. A simple baseline for open vocabulary semantic segmentation with pre-trained vision- language model. In ECCV, 2022

    2022

  32. [40]

    Side adapter network for open-vocabulary semantic segmentation

    Mengde Xu, Zheng Zhang, Fangyun Wei, Han Hu, and Xi- ang Bai. Side adapter network for open-vocabulary semantic segmentation. In CVPR, 2023. 2, 3, 4, 5, 6, 11, 14

    2023

  33. [41]

    Few-shot segmentation via cycle-consistent trans- former

    Gengwei Zhang, Guoliang Kang, Yi Yang, and Yunchao Wei. Few-shot segmentation via cycle-consistent trans- former. In NeurIPS, 2021. 8

    2021

  34. [42]

    Personalize segment anything model with one shot

    Renrui Zhang, Zhengkai Jiang, Ziyu Guo, Shilin Yan, Junt- ing Pan, Hao Dong, Yu Qiao, Peng Gao, and Hongsheng Li. Personalize segment anything model with one shot. InICLR,

  35. [43]

    Open-vocabulary uni- versal image segmentation with maskclip

    Zhuowen Tu Zheng Ding, Jieke Wang. Open-vocabulary uni- versal image segmentation with maskclip. InICML, 2023. 2, 3

    2023

  36. [44]

    Scene parsing through ade20k dataset

    Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. In CVPR, 2017. 5

    2017

  37. [45]

    Class-incremental learning: A survey

    Da-Wei Zhou, Qi-Wei Wang, Zhi-Hong Qi, Han-Jia Ye, De- Chuan Zhan, and Ziwei Liu. Class-incremental learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. 8

    2024

  38. [46]

    Conditional prompt learning for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Zi- wei Liu. Conditional prompt learning for vision-language models. In CVPR, 2022. 3

    2022

  39. [47]

    Learning to prompt for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. IJCV,

  40. [48]

    Class-incremental learning via dual augmentation

    Fei Zhu, Zhen Cheng, Xu-yao Zhang, and Cheng-lin Liu. Class-incremental learning via dual augmentation. In NeurIPS, 2021. 8

    2021

  41. [49]

    Segment everything everywhere all at once

    Xueyan Zou, Jianwei Yang, Hao Zhang, Feng Li, Linjie Li, Jianfeng Wang, Lijuan Wang, Jianfeng Gao, and Yong Jae Lee. Segment everything everywhere all at once. InNeurIPS,

  42. [2023]

    my boat”, “my bird

    3, 7 10 Supplementary Material Foreground segmentation map Combined segmentation mapImage Open-vocab. segmentation map ChestHot tub <my concept> <my concept> <my concept><my concept> Figure 9. Examples of segmentation maps we used in ADE per dataset. We combine the open-vocabu...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.