Pith. sign in

REVIEW 4 major objections 5 minor 71 references

Test-Time Optimization for Domain Adaptive Open Vocabulary Segmentation

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Seg-TTO is a plug-and-play test-time optimization module that improves open-vocabulary segmentation on specialized domains by aligning visual and textual features with each test image, raising mean mIoU from 37.63 to 40.27 on the…

desk verdict A solid first test-time optimization framework for open-vocabulary segmentation; the plug-and-play claim holds up, though the headline mIoU gain trades on a few very small datasets. read the letter →

arxiv 2501.04696 v2 pith:LOCWJAUW submitted 2025-01-08 cs.CV

Ulindu De Silva , Didula Samaraweera , Sasini Wanigathunga , Kavindu Kariyawasam , Kanchana Ranasinghe , Muzammal Naseer , Ranga Rodrigo This is my paper · ORCID
classification cs.CV
keywords test-timeoptimizationopen-vocabularysemanticsegmentationzero-shotdomainadaptationprompttuningvisualfeatureaggregationpseudo-labelself-trainingLLM-generatedattributes
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Seg-TTO is a plug-and-play module that adapts an existing open-vocabulary semantic segmentation model to a specialized domain at test time, using only the test image and its category names. The paper claims this is the first test-time optimization framework for OVSS, and that it improves zero-shot performance across all 22 datasets in the MESS benchmark without any additional training data. On the strongest baseline, CAT-Seg-Large, Seg-TTO raises mean mIoU from 37.63 to 40.27, with relative gains up to 27% on individual datasets. The method works by tuning per-category text embeddings with a segmentation-specific self-supervised loss while aggregating low-entropy augmented visual features back into the original feature map, preserving spatial structure.

What carries the argument

The load-bearing mechanism is the self-supervised selector loss $L_{SSL}$, a pixel-level objective that scores how well current visual and textual features explain the test image. It is a PCGrad-combined sum of an entropy term $L_{ent}$ and a pseudo-label cross-entropy term $L_{ce}$, where pseudo-labels $\hat{y}$ come from normalized cross-modal similarity; minimizing it is the signal that drives both textual and visual adaptation. On the text side, each category carries multiple learnable prompts that are tuned per test sample and then blended, via weight $\beta$, with LLM-generated category attributes that have been pre-aggregated by cosine similarity to the category embedding. On the vision side, visual feature aggregation interpolates the lowest-entropy augmented views back into the original feature map locations, preserving spatial structure while enhancing the objects present.

What would settle it

Run Seg-TTO on a specialized-domain dataset while corrupting the pseudo-labels in Eq. (2) by cyclically permuting the category labels; if mean mIoU barely changes, the cross-entropy term is not doing the claimed work, and if it drops sharply, the method depends on correct pseudo-labels.

Watch

Extended reading notes

Core claim

Seg-TTO's central claim is that open-vocabulary segmentation models can be improved on specialized domains by aligning their representations to each test image at inference time, with no labels and no retraining. The framework treats segmentation as a multi-concept, spatially structured task, so it does not just borrow classification-style test-time prompt tuning. Instead, it learns several prompt embeddings per category and updates them with a self-supervised loss that combines per-pixel entropy minimization with a pseudo-label cross-entropy term; simultaneously, it selects low-entropy views among 64 augmented copies of the image and aggregates them into the original visual feature map so locality is retained. The paper shows the module works when attached to three existing OVSS approaches, including a mask-free unsupervised method, and reports consistent gains over the 22-dataset MESS benchmark, with the best configuration reaching 40.27 mean mIoU versus 37.63 for the unmodified baseline.

Load-bearing premise

The load-bearing premise is that the model's own per-pixel similarity scores are reliable enough to serve as pseudo-labels, so systematically wrong early guesses on unfamiliar categories would make the tuning lock in errors instead of fixing them.

Editorial extensions

If this is right

  • Adding Seg-TTO to an existing OVSS model improves out-of-domain segmentation without retraining, fine-tuning, or pixel-level labels on the target domain.
  • The gains are consistent across a broad spread of specialized domains: medical imaging, earth monitoring, engineering, agriculture, and biology, as well as general categories such as food and body parts.
  • The method transfers to mask-free, unsupervised OVSS models, not only pixel-supervised ones, which widens its applicability.
  • Classification-style test-time prompt tuning (TPT) is not a drop-in replacement; a segmentation-specific objective with spatial aggregation is what produces the improvement.
  • The main cost is inference speed: about 1.5 seconds per image versus 0.5 seconds for the baseline, and the paper names this as the limitation motivating future distillation work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: Nothing in the design is segmentation-exclusive; the same selector loss and feature-aggregation loop could be applied to any dense prediction task with language-aligned pixel features, such as open-vocabulary detection or panoptic segmentation.
  • Inference: The paper's attribute ablations show LLM quality directly moves results, which leaves open a cheaper route: if high-quality human-written attributes are available, they may substitute for a large LLM while keeping most of the gain.
  • Inference: Because the module does not need training data, it could be used as a bootstrap: run Seg-TTO on unlabeled target images and distill the adapted prompts or features into a lightweight model, addressing the stated inference-speed limitation.
  • Inference: A natural stress test is whether the pseudo-label term helps in a domain where the baseline's top prediction is wrong almost everywhere; if not, the entropy term alone may carry most of the benefit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents Seg-TTO, a plug-and-play test-time optimization module for open-vocabulary semantic segmentation (OVSS). At test time, the method (i) optimizes learnable text prompts with a segmentation-specific self-supervised loss combining entropy minimization and pseudo-label cross-entropy, together with PCGrad gradient surgery; (ii) selects low-entropy augmented visual views and aggregates them into the original image embedding to preserve spatial structure; and (iii) augments text embeddings with LLM-generated, sample-filtered category attributes. The framework is evaluated on the 22-dataset MESS benchmark when attached to CAT-Seg-B, CAT-Seg-L, and CLIP-DINOiser, reporting consistent mIoU gains, e.g., CAT-Seg-L-TTO at 40.27 mean mIoU versus 37.63 for the baseline, with the largest relative gains on specialized datasets such as CHASE DB1, CryoNuSeg, and CWFID.

Significance. If the reported gains are robust, Seg-TTO would be a practically useful, training-free way to improve existing OVSS models on specialized domains, and the first test-time optimization framework aimed at dense open-vocabulary prediction. The paper's strengths are its broad evaluation across 22 datasets and three baselines, a clean ablative decomposition into textual and visual components, and a consistent story about why classification-style TTO is insufficient for segmentation. The authors also publicly commit to code and model release. However, the evaluation's independence from hyperparameter tuning, the absence of variance estimates, and a typo in a central aggregation equation currently prevent me from treating the central quantitative claim as fully established.

major comments (4)
  1. [§4, Implementation Details] The text says: "We tune hyperparameters using two held-out datasets and evaluate across all datasets and model variants using the same, fixed hyperparameters," but the two datasets are never named. If the two tuning datasets are members of the 22-dataset MESS benchmark, then a portion of the reported mean gain comes from settings that were explicitly selected on those datasets, and the claim of plug-and-play generality is weaker than presented. Please name the two held-out datasets and report the mean mIoU both including and excluding them.
  2. [Tables 1–6] No error bars, confidence intervals, or number of random seeds are reported anywhere, and several of the largest per-dataset gains occur on very small test sets (CHASE DB1 has 20 images, CWFID 21, CryoNuSeg 30). For example, the CAT-Seg-L-TTO gain on CWFID is +8.47 mIoU and on CHASE DB1 is +4.97 mIoU; with such few images, the unweighted 22-dataset mean can be moved by one or two datasets. Please provide per-dataset standard deviations over at least three runs, or a bootstrap confidence interval for the mean gains.
  3. [§3.3, Eq. (7)] Equation (7) is printed with an identical numerator and denominator, so γattr(Aj) would be identically 1 for every category, which cannot be the intended operation. The denominator should almost certainly be the sum of the cosine similarity weights, not the sum of the weighted embeddings. As written, the equation is a central part of the attribute aggregation and cannot be reproduced. Please correct the equation and, in the revision, state the exact normalization used in the experiments.
  4. [§3.2, Eq. (2) and Table 13] The cross-entropy term in Eq. (2) treats the model's own normalized cross-modal similarity ŷ as pseudo-labels, yet the paper does not analyze the accuracy of these pseudo-labels or potential confirmation bias. The concern is not hypothetical: on datasets with highly similar categories (e.g., Corrosion CS and ZeroWaste-f), a systematically wrong early prediction could be reinforced by the CE term. Table 13 indicates the CE term contributes only +0.2 mIoU on Dark Zurich, but that dataset may not be representative of the high-similarity failure mode. Please provide pseudo-label accuracy or a failure-case analysis on high-similarity datasets, and state what happens when the CE term is weighted differently.
minor comments (5)
  1. [§3.2, Eq. (3)] The denominator in Eq. (3) sums over "K" but the index is j over the n categories; the symbol K is not defined anywhere. Please replace K with n (or define K = n).
  2. [Table 1] Several entries in the Random row contain formatting errors, such as "31.539.3", "89.749.92", and "83.7749.92". This makes the table hard to read and should be fixed.
  3. [§4.1, paragraph 2] The sentence "Our framework demonstrates gains across 22 datasets" is stronger than what Table 1 shows: CAT-Seg-B-TTO has a small drop on BDD100K (44.58 to 44.03) and CAT-Seg-L-TTO drops on several datasets (e.g., FloodNet 39.84 to 39.79, Corrosion CS 16.87 to 16.40). Please soften or qualify this sentence.
  4. [§3.1 and Figure 2] The selector module is introduced but its internal loss aggregation is described only verbally; in Figure 2, the text "Low Entropy" and "High Entropy" is not explained in the caption. A concise formal statement of γsel (e.g., retention of the lowest 20% entropy views) would help reproducibility.
  5. [Supplementary Material, §A.3] The abbreviation "TTFO" appears for the first time in the supplementary; please use "TTO" consistently, or define the abbreviation on first use.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Seg-TTO's gains are evaluated on an external benchmark, and the self-referential pseudo-label loss is a standard self-training signal rather than a derivation.

full rationale

Seg-TTO is an empirical test-time adaptation method; its central claim is that adding its optimization module to existing OVSS models improves zero-shot mIoU on the external MESS benchmark. The components — entropy minimization (Eq. 1), pseudo-label cross-entropy (Eq. 2), visual feature aggregation (Eq. 9), and LLM attribute aggregation (Eqs. 7–8) — are defined directly in terms of the model's own features and text embeddings, not in terms of the reported ground-truth metric. The pseudo-label term in Eq. (2) uses the model's own normalized cross-modal similarity as a target, which is a standard self-training or self-distillation signal; it makes the optimization self-referential at the level of the loss, but it does not make the evaluation circular, because the reported mIoU is computed against external annotations and the optimization can move predictions away from or toward ground truth. No fitted parameter is renamed as a prediction: hyperparameters (p, m, m', step counts, learning rate) are fixed globally after tuning on two unnamed held-out datasets, and the per-dataset numbers in Table 1 are not fitted values. The paper's prior-work citations, including the co-authored CLIPpy baseline [44] and COPT [38], are used as baselines or related work, not as justification for the central claim. The main evaluation is against an external benchmark (MESS) with three independent baselines, so the central claim does not reduce to its inputs by construction. The only concerns are evaluation-transparency issues — the two tuning datasets are not named and no error bars are given — which affect the robustness of the empirical claim, not the circularity of the derivation.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on several unproven modeling assumptions: that the self-supervised loss measures suitability, that pseudo-labels are reliable, and that LLM attributes help. The free parameters are hyperparameters tuned on benchmark subsets. No invented physical or conceptual entities are introduced; the selector and augmentors are software modules within the method.

free parameters (6)
  • p (number of learnable prompts) = 5
    Controls capacity of per-category text embeddings; set by the authors without a reported sweep.
  • m (number of visual augmentations) = 64
    Number of augmented views generated per image for feature selection and aggregation.
  • entropy retention ratio = 20% (m')
    Fraction of lowest-entropy augmentations retained for visual feature aggregation.
  • gradient steps for entropy and CE losses = 2/3
    Number of test-time optimization iterations per loss using PCGrad.
  • beta in Eq. (8) = not reported
    Blending weight between tuned prompts and LLM attribute embeddings; 'fixed experimentally' but value not given.
  • temperature tau in Eq. (3) = not reported
    Softmax temperature for cross-modal similarity; value not specified.
assumptions (5)
  • domain assumption Pre-trained CLIP/CAT-Seg features are informative enough for specialized-domain tasks after test-time adaptation.
    Invoked throughout Section 3; if base features are useless, TTO cannot recover them.
  • ad hoc to paper The model's own per-pixel predictions \hat{y} are reliable pseudo-labels for the cross-entropy term in Eq. (2).
    The optimization updates prompts to match \hat{y}; no analysis of pseudo-label noise or collapse is provided.
  • ad hoc to paper Features minimizing the self-supervised loss L_SSL are the most informative for segmentation.
    Stated in Section 3.2 as a hypothesis; supported only indirectly by ablations.
  • domain assumption LLM-generated category attributes are visually relevant and discriminative for the target domain.
    Section 3.3 relies on attributes generated by Llama-3-70B; quality varies by LLM and prompt style.
  • standard math PCGrad correctly combines entropy and cross-entropy gradients.
    Adopted from Yu et al. [62]; treated as background algorithm without re-derivation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Test-Time Optimization for Domain Adaptive Open Vocabulary Segmentation." pith.science (2026). https://pith.science/paper/LOCWJAUW

@misc{pith2026250104696,
  author       = {Pith},
  title        = {Pith review of: Test-Time Optimization for Domain Adaptive Open Vocabulary Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LOCWJAUW}},
  note         = {Machine review of arXiv:2501.04696}
}
read the original abstract

We present Seg-TTO, a novel framework for zero-shot, open-vocabulary semantic segmentation (OVSS), designed to excel in specialized domain tasks. While current open-vocabulary approaches show impressive performance on standard segmentation benchmarks under zero-shot settings, they fall short of supervised counterparts on highly domain-specific datasets. We focus on segmentation-specific test-time optimization to address this gap. Segmentation requires an understanding of multiple concepts within a single image while retaining the locality and spatial structure of representations. We propose a novel self-supervised objective adhering to these requirements and use it to align the model parameters with input images at test time. In the textual modality, we learn multiple embeddings for each category to capture diverse concepts within an image, while in the visual modality, we calculate pixel-level losses followed by embedding aggregation operations specific to preserving spatial structure. Our resulting framework termed Seg-TTO is a plug-and-play module. We integrate Seg-TTO with three state-of-the-art OVSS approaches and evaluate across 22 challenging OVSS tasks covering a range of specialized domains. Our Seg-TTO demonstrates clear performance improvements (up to 27% mIoU increase on some datasets) establishing new state-of-the-art. Our code and models will be released publicly.

Figures

Figures reproduced from arXiv: 2501.04696 by the authors.

Figure 1
Figure 1. Our Seg-TTO (row 4) improves state-of-the-art base [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of Seg-TTO (a) Our image embedding updating framework consists of filtering out confident image patches followed by updating the original image embedding. (b) Our test time optimization framework consists of updating prompts based on the most confident crops using backpropagation followed by the addition of attributes for generalization. proved zero-shot performance of open vocabulary classifi￾cation [9, 26… view at source ↗
Figure 3
Figure 3. Qualitative Evaluation: Our proposed Seg-TTO outperforms state-of-the-art CAT-Seg [10] across diverse specialized-domain OVSS tasks as illustrated. We highlight the highly technical nature of some specialized domain category names (e.g., mediastinum under X-Ray). Our category attributes allow models to better understand such objects. essary for these augmented prompts to be effective. Re￾sults presented in [PITH_FU… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Illustration of improved attribute generation for FoodSeg103[55] dataset images (a) The original image. (b) Ground truth segmentation map. (c) Baseline [40] attribute gener￾ation method, which included general and irrelevant features such as “feathered body” and “wings…
Figure 5
Figure 5. Figure 5: Qualitative compar￾ison between Vis. Feat. Aggr. and Test Time Opt.: Our ap￾proach (d) successfully identi￾fies more fish and (e) identifies sea-floor, whereas baseline (c) fails. Image Ground Truth Baseline Baseline + TTO CHASE DB CUB-200 FoodSeg103 [PITH_FULL_IMAGE:…
Figure 6
Figure 6. Figure 6: Qualitative Evaluation: We illustrate both success and failure cases of our proposed Seg-TTO. We highlight how Seg-TTO is still better than the baseline even in failure cases. 5 [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Qualitative Evaluation: We illustrate both success and failure cases of our proposed Seg-TTO. We highlight how Seg-TTO is still better than the baseline even in failure cases. 6 [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Qualitative Evaluation: We illustrate both success and failure cases of our proposed Seg-TTO. We highlight how Seg-TTO is still better than the baseline even in failure cases. 7 [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Qualitative Evaluation: We illustrate both success and failure cases of our proposed Seg-TTO. We highlight how Seg-TTO is still better than the baseline even in failure cases. 8 [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

71 extracted references · 57 canonical work pages

  1. [1]

    Align your prompts: Test-time prompting with distribution align- ment for zero-shot generalization

    Jameel Abdul Samadh, Mohammad Hanan Gani, Noor Hus- sein, Muhammad Uzair Khattak, Muhammad Muzammal Naseer, Fahad Shahbaz Khan, and Salman H Khan. Align your prompts: Test-time prompting with distribution align- ment for zero-shot generalization. NeurIPS, 36, 2024. 2, 3

    work page 2024

  2. [2]

    Llama 3 model card

    AI@Meta. Llama 3 model card. 2024. 1

    work page 2024

  3. [3]

    Landcovernet: A global benchmark land cover classification training dataset

    Hamed Alemohammad and Kevin Booth. Landcovernet: A global benchmark land cover classification training dataset. arXiv preprint arXiv:2012.03111, 2020. 6, 2, 3

    arXiv 2012

  4. [4]

    Zerowaste dataset: To- wards deformable object segmentation in cluttered scenes

    Dina Bashkirova, Mohamed Abdelfattah, Ziliang Zhu, James Akl, Fadi Alladkani, Ping Hu, Vitaly Ablavsky, Berk Calli, Sarah Adel Bargal, and Kate Saenko. Zerowaste dataset: To- wards deformable object segmentation in cluttered scenes. In CVPR, pages 21147–21157, 2022. 2, 6, 3

    work page 2022

  5. [5]

    Corrosion condition state semantic segmentation dataset

    Eric Bianchi and Matthew Hebdon. Corrosion condition state semantic segmentation dataset. University Libraries, Virginia Tech: Blacksburg, VA, USA, 3, 2021. 2, 6, 3

    work page 2021

  6. [6]

    What a MESS: Multi-Domain Evaluation of Zero-Shot Semantic Segmentation

    Benedikt Blumenstiel, Johannes Jakubik, Hilde Kuhne, and Michael V ossing. What a mess: Multi-domain evaluation of zero-shot semantic segmentation. ArXiv, abs/2306.15521,

    work page Pith review arXiv

  7. [7]

    Zero-shot semantic segmentation

    Maxime Bucher, Tuan-Hung Vu, Matthieu Cord, and Patrick P´erez. Zero-shot semantic segmentation. NeurIPS, 32, 2019. 7, 3

    work page 2019

  8. [8]

    Smoothing mat- ters: Momentum transformer for domain adaptive semantic segmentation, 2022

    Runfa Chen, Yu Rong, Shangmin Guo, Jiaqi Han, Fuchun Sun, Tingyang Xu, and Wenbing Huang. Smoothing mat- ters: Momentum transformer for domain adaptive semantic segmentation, 2022. 2

    work page 2022

Show all 71 references
  1. [9]

    Evolving interpretable visual classifiers with large language models

    Mia Chiquier, Utkarsh Mall, and Carl V ondrick. Evolving interpretable visual classifiers with large language models. arXiv preprint arXiv:2404.09941, 2024. 2, 3

    2024 arXiv

  2. [10]

    Cat-seg: Cost aggregation for open- vocabulary semantic segmentation

    Seokju Cho, Heeseong Shin, Sung-Jin Hong, Seungjun An, Seungjun Lee, Anurag Arnab, Paul Hongsuck Seo, and Seung Wook Kim. Cat-seg: Cost aggregation for open- vocabulary semantic segmentation. In CVPR, 2024. 1, 2, 4, 6, 7, 8, 3

    2024

  3. [11]

    Semantic segmentation in art paintings

    Nadav Cohen, Yael Newman, and Ariel Shamir. Semantic segmentation in art paintings. In Computer graphics forum, pages 261–275. Wiley Online Library, 2022. 6, 7, 2, 3

    2022

  4. [12]

    De- coupling zero-shot semantic segmentation

    Jian Ding, Nan Xue, Gui-Song Xia, and Dengxin Dai. De- coupling zero-shot semantic segmentation. In CVPR, pages 11583–11592, 2022. 2, 6

    2022

  5. [13]

    Atlantis: A benchmark for semantic segmentation of waterbody images

    Seyed Mohammad Hassan Erfani, Zhenyao Wu, Xinyi Wu, Song Wang, and Erfan Goharian. Atlantis: A benchmark for semantic segmentation of waterbody images. Environmental Modelling & Software, 149:105333, 2022. 6, 2, 3

    2022

  6. [14]

    An ensemble classification-based approach applied to retinal blood vessel segmentation

    Muhammad Moazam Fraz, Paolo Remagnino, Andreas Hoppe, Bunyarit Uyyanonvara, Alicja R Rudnicka, Christo- pher G Owen, and Sarah A Barman. An ensemble classification-based approach applied to retinal blood vessel segmentation. IEEE Transactions on Biomedical Engineer- ing, 59(9)...

    2012

  7. [15]

    Thomas Mesnard Gemma Team, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Laurent Sifre, Morgane Rivi`ere, Mihir Sanjay Kale, Juliette Love, Pouya Tafti, L ´eonard Hussenot, and et al. Gemma. 2024. 1

    2024

  8. [16]

    A crop/weed field im- age dataset for the evaluation of computer vision based pre- cision agriculture tasks

    Sebastian Haug and J ¨orn Ostermann. A crop/weed field im- age dataset for the evaluation of computer vision based pre- cision agriculture tasks. In Computer Vision-ECCV 2014 Workshops: Zurich, Switzerland, September 6-7 and 12, 2014, Proceedings, Part IV 13 , pages 105–116. Springer,

    2014

  9. [17]

    Pointseg: A training-free paradigm for 3d scene segmentation via foundation models

    Qingdong He, Jinlong Peng, Zhengkai Jiang, Xiaobin Hu, Jiangning Zhang, Qiang Nie, Yabiao Wang, and Chengjie Wang. Pointseg: A training-free paradigm for 3d scene segmentation via foundation models. arXiv preprint arXiv:2403.06403, 2024. 3

    2024 arXiv

  10. [18]

    Daformer: Improving network architectures and training strategies for domain-adaptive semantic segmentation

    Lukas Hoyer, Dengxin Dai, and Luc Van Gool. Daformer: Improving network architectures and training strategies for domain-adaptive semantic segmentation. In CVPR, 2022. 2

    2022

  11. [19]

    Hrda: Context-aware high-resolution domain-adaptive semantic segmentation

    Lukas Hoyer, Dengxin Dai, and Luc Van Gool. Hrda: Context-aware high-resolution domain-adaptive semantic segmentation. In ECCV, 2022

    2022

  12. [20]

    Mic: Masked image consistency for context-enhanced domain adaptation

    Lukas Hoyer, Dengxin Dai, Haoran Wang, and Luc Van Gool. Mic: Masked image consistency for context-enhanced domain adaptation. In CVPR, 2023. 2

    2023

  13. [21]

    Semantic segmentation of underwater im- agery: Dataset and benchmark

    Md Jahidul Islam, Chelsey Edge, Yuyang Xiao, Peigen Luo, Muntaqim Mehtaz, Christopher Morse, Sadman Sakib Enan, and Junaed Sattar. Semantic segmentation of underwater im- agery: Dataset and benchmark. In 2020 IEEE/RSJ Interna- tional Conference on Intelligent Robots and System...

    2020

  14. [22]

    Kvasir-instrument: Diagnostic and therapeu- tic tool segmentation dataset in gastrointestinal endoscopy

    Debesh Jha, Sharib Ali, Krister Emanuelsen, Steven A Hicks, Vajira Thambawita, Enrique Garcia-Ceja, Michael A Riegler, Thomas de Lange, Peter T Schmidt, H ˚avard D Jo- hansen, et al. Kvasir-instrument: Diagnostic and therapeu- tic tool segmentation dataset in gastrointestinal ...

    2021

  15. [23]

    Scaling up visual and vision-language representa- tion learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representa- tion learning with noisy text supervision. pages 4904–4916. PMLR, 2021. 1, 2, 3

    2021

  16. [24]

    Belongie, Bharath Hariharan, and Ser Nam Lim

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge J. Belongie, Bharath Hariharan, and Ser Nam Lim. Vi- sual prompt tuning. ArXiv, abs/2203.12119, 2022. 3

    2022 arXiv

  17. [25]

    Mistral 7b

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las 9 Casas, Florian Bressand, Gianna Lengyel, Guillaume Lam- ple, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023. 1

    arXiv

  18. [26]

    Llms meet vlms: Boost open vocabulary ob- ject detection with fine-grained descriptors

    Sheng Jin, Xueying Jiang, Jiaxing Huang, Lewei Lu, and Shijian Lu. Llms meet vlms: Boost open vocabulary ob- ject detection with fine-grained descriptors. arXiv preprint arXiv:2402.04630, 2024. 3

    2024 arXiv

  19. [27]

    Venkatesh Babu

    Jogendra Nath Kundu, Akshay Kulkarni, Amit Singh, Varun Jampani, and R. Venkatesh Babu. Generalize then adapt: Source-free domain adaptive semantic segmentation. In ICCV, 2021. 2

    2021

  20. [28]

    Proxyclip: Proxy at- tention improves clip for open-vocabulary segmentation

    Mengcheng Lan, Chaofeng Chen, Yiping Ke, Xinjiang Wang, Litong Feng, and Wayne Zhang. Proxyclip: Proxy at- tention improves clip for open-vocabulary segmentation. In ECCV, 2024. 1, 2

    2024

  21. [29]

    Content-consistent matching for domain adaptive semantic segmentation

    Guangrui Li, Guoliang Kang, Wu Liu, Yunchao Wei, and Yi Yang. Content-consistent matching for domain adaptive semantic segmentation. In ECCV, 2020. 2

    2020

  22. [30]

    Multiple- human parsing in the wild.arXiv preprint arXiv:1705.07206,

    Jianshu Li, Jian Zhao, Yunchao Wei, Congyan Lang, Yidong Li, Terence Sim, Shuicheng Yan, and Jiashi Feng. Multiple- human parsing in the wild.arXiv preprint arXiv:1705.07206,

    arXiv

  23. [31]

    Junnan Li, Silvio Savarese, and Steven C. H. Hoi. Masked unsupervised self-training for label-free image classification. In ICLR, 2023. 2

    2023

  24. [32]

    Open-vocabulary semantic segmentation with mask-adapted clip

    Feng Liang, Bichen Wu, Xiaoliang Dai, Kunpeng Li, Yinan Zhao, Hang Zhang, Peizhao Zhang, Peter Vajda, and Diana Marculescu. Open-vocabulary semantic segmentation with mask-adapted clip. In CVPR, pages 7061–7070, 2023. 1, 2, 6, 7

    2023

  25. [33]

    Deepcrack: A deep hierarchical feature learning architec- ture for crack segmentation

    Yahui Liu, Jian Yao, Xiaohu Lu, Renping Xie, and Li Li. Deepcrack: A deep hierarchical feature learning architec- ture for crack segmentation. Neurocomputing, 338:139–153,

  26. [34]

    Bidirectional self-training with multiple anisotropic prototypes for domain adaptive semantic seg- mentation

    Yulei Lu, Yawei Luo, Li Zhang, Zheyang Li, Yi Yang, and Jun Xiao. Bidirectional self-training with multiple anisotropic prototypes for domain adaptive semantic seg- mentation. In ACM MM, 2022. 2

    2022

  27. [35]

    Uavid: A semantic segmentation dataset for uav imagery

    Ye Lyu, George V osselman, Gui-Song Xia, Alper Yilmaz, and Michael Ying Yang. Uavid: A semantic segmentation dataset for uav imagery. ISPRS journal of photogrammetry and remote sensing, 165:108–119, 2020. 6, 2, 3

    2020

  28. [36]

    Swapprompt: Test-time prompt adaptation for vision- language models

    Xiaosong Ma, Jie Zhang, Song Guo, and Wenchao Xu. Swapprompt: Test-time prompt adaptation for vision- language models. NeurIPS, 36, 2024. 3

    2024

  29. [37]

    Cryonuseg: A dataset for nuclei instance segmenta- tion of cryosectioned h&e-stained histological images.Com- puters in biology and medicine, 132:104349, 2021

    Amirreza Mahbod, Gerald Schaefer, Benjamin Bancher, Christine L ¨ow, Georg Dorffner, Rupert Ecker, and Isabella Ellinger. Cryonuseg: A dataset for nuclei instance segmenta- tion of cryosectioned h&e-stained histological images.Com- puters in biology and medicine, 132:104349, 2...

    2021

  30. [38]

    Copt: Unsupervised domain adaptive segmentation using domain-agnostic text embeddings

    Cristina Mata, Kanchana Ranasinghe, and Michael Ryoo. Copt: Unsupervised domain adaptive segmentation using domain-agnostic text embeddings. In ECCV, 2024. 2

    2024

  31. [39]

    Towards global flood mapping onboard low cost satellites with machine learning

    Gonzalo Mateo-Garcia, Joshua Veitch-Michaelis, Lewis Smith, Silviu Vlad Oprea, Guy Schumann, Yarin Gal, Atılım G¨unes ¸ Baydin, and Dietmar Backes. Towards global flood mapping onboard low cost satellites with machine learning. Scientific reports, 11(1):7249, 2021. 6, 2, 3

    2021

  32. [40]

    Visual classification via description from large language models

    Sachit Menon and Carl V ondrick. Visual classification via description from large language models. ICLR, 2023. 2, 3, 1

    2023

  33. [41]

    In- telligent multi-view test time augmentation

    Efe Ozturk, Mohit Prabhushankar, and Ghassan AlRegib. In- telligent multi-view test time augmentation. arXiv preprint arXiv:2406.08593, 2024. 2, 3

    2024 arXiv

  34. [42]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. pages 8748–8763. PMLR, 2021. 1, 2, 3, 4, 6

    2021

  35. [43]

    Floodnet: A high resolution aerial imagery dataset for post flood scene understanding

    Maryam Rahnemoonfar, Tashnim Chowdhury, Argho Sarkar, Debvrat Varshney, Masoud Yari, and Robin Rober- son Murphy. Floodnet: A high resolution aerial imagery dataset for post flood scene understanding. IEEE Access , 9:89644–89654, 2021. 6, 2, 3

    2021

  36. [44]

    Per- ceptual grouping in contrastive vision-language models

    Kanchana Ranasinghe, Brandon McKinzie, Sachin Ravi, Yinfei Yang, Alexander Toshev, and Jonathon Shlens. Per- ceptual grouping in contrastive vision-language models. In CVPR, pages 5571–5584, 2023. 1, 2, 4, 6, 7

    2023

  37. [45]

    Grounded sam: Assembling open-world models for diverse visual tasks,

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kun- chang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. Grounded sam: Assembling open-world models for diverse visual tasks,

  38. [46]

    Guided curriculum model adaptation and uncertainty-aware evalua- tion for semantic nighttime image segmentation

    Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Guided curriculum model adaptation and uncertainty-aware evalua- tion for semantic nighttime image segmentation. In ICCV, pages 7374–7383, 2019. 6, 7, 1, 2, 3

    2019

  39. [47]

    De- tailed annotations of chest x-rays via ct projection for report understanding

    Constantin Seibold, Simon Reiß, Saquib Sarfraz, Matthias A Fink, Victoria Mayer, Jan Sellner, Moon Sung Kim, Klaus H Maier-Hein, Jens Kleesiek, and Rainer Stiefelhagen. De- tailed annotations of chest x-rays via ct projection for report understanding. arXiv preprint arXiv:2210...

    arXiv

  40. [48]

    Jinghuan Shang and Michael S. Ryoo. Active vision re- inforcement learning under limited visual observability. In NeurIPS, 2023. 3

    2023

  41. [49]

    Pst900: Rgb- thermal calibration, dataset and segmentation network

    Shreyas S Shivakumar, Neil Rodrigues, Alex Zhou, Ian D Miller, Vijay Kumar, and Camillo J Taylor. Pst900: Rgb- thermal calibration, dataset and segmentation network. In 2020 IEEE international conference on robotics and au- tomation (ICRA), pages 9441–9447. IEEE, 2020. 2, 6, 3

    2020

  42. [50]

    Test- time prompt tuning for zero-shot generalization in vision- language models

    Manli Shu, Weili Nie, De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, and Chaowei Xiao. Test- time prompt tuning for zero-shot generalization in vision- language models. NeurIPS, 35:14274–14289, 2022. 2, 3, 5, 7, 8

    2022

  43. [51]

    C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. Caltech-ucsd birds 200. Technical Report CNS-TR-2011- 001, California Institute of Technology, 2011. 2, 6

    2011

  44. [52]

    Caltech-UCSD Birds 200

    Catherine Wah, Steve Branson, Peter Welinder, Pietro Per- ona, and Serge Belongie. Caltech-UCSD Birds 200. Cali- fornia Institute of Technology, 2011. 2, 3

    2011

  45. [53]

    Cross-region domain adaptation for class-level alignment, 2022

    Zhijie Wang, Xing Liu, Masanori Suganuma, and Takayuki Okatani. Cross-region domain adaptation for class-level alignment, 2022. 2 10

    2022

  46. [54]

    isaid: A large- scale dataset for instance segmentation in aerial images

    Syed Waqas Zamir, Aditya Arora, Akshita Gupta, Salman Khan, Guolei Sun, Fahad Shahbaz Khan, Fan Zhu, Ling Shao, Gui-Song Xia, and Xiang Bai. isaid: A large- scale dataset for instance segmentation in aerial images. In CVPRW, pages 28–37, 2019. 6, 2, 3

    2019

  47. [55]

    A large-scale benchmark for food im- age segmentation

    Xiongwei Wu, Xin Fu, Ying Liu, Ee-Peng Lim, Steven CH Hoi, and Qianru Sun. A large-scale benchmark for food im- age segmentation. In Proceedings of the 29th ACM interna- tional conference on multimedia, pages 506–515, 2021. 6, 1, 2, 3

    2021

  48. [56]

    Clip-dinoiser: Teaching clip a few dino tricks for open- vocabulary semantic segmentation

    Monika Wysocza ´nska, Oriane Sim´eoni, Micha¨el Ramamon- jisoa, Andrei Bursuc, Tomasz Trzci ´nski, and Patrick P ´erez. Clip-dinoiser: Teaching clip a few dino tricks for open- vocabulary semantic segmentation. arXiv, 2023. 1, 2, 6, 7, 3, 4

    2023

  49. [57]

    A simple baseline for open- vocabulary semantic segmentation with pre-trained vision- language model

    Mengde Xu, Zheng Zhang, Fangyun Wei, Yutong Lin, Yue Cao, Han Hu, and Xiang Bai. A simple baseline for open- vocabulary semantic segmentation with pre-trained vision- language model. In ECCV, pages 736–753. Springer, 2022. 2, 6

    2022

  50. [58]

    Side adapter network for open-vocabulary semantic segmentation

    Mengde Xu, Zheng Zhang, Fangyun Wei, Han Hu, and Xi- ang Bai. Side adapter network for open-vocabulary semantic segmentation. In CVPR, pages 2945–2954, 2023. 1, 2, 6, 7

    2023

  51. [59]

    Opendas: Open-vocabulary domain adaptation for 2d and 3d segmentation

    Gonca Yilmaz, Songyou Peng, Marc Pollefeys, Francis En- gelmann, and Hermann Blum. Opendas: Open-vocabulary domain adaptation for 2d and 3d segmentation. arXiv preprint arXiv:2405.20141, 2024. 3

    2024 arXiv

  52. [60]

    Bdd100k: A diverse driving dataset for heterogeneous multitask learning

    Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Dar- rell. Bdd100k: A diverse driving dataset for heterogeneous multitask learning. In CVPR, pages 2636–2645, 2020. 6, 2, 3

    2020

  53. [61]

    Kola: Carefully benchmarking world knowledge of large language models

    Jifan Yu, Xiaozhi Wang, Shangqing Tu, Shulin Cao, Daniel Zhang-li, Xin Lv, Hao Peng, Zijun Yao, Xiaohan Zhang, Hanming Li, Chun yan Li, Zheyuan Zhang, Yushi Bai, Yan- tao Liu, Amy Xin, Nianyi Lin, Kaifeng Yun, Linlu Gong, Jianhui Chen, Zhili Wu, Yun Peng Qi, Weikai Li, Yong Gu...

    2024

  54. [62]

    Gradient surgery for multi-task learning

    Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. NeurIPS, 33:5824–5836, 2020. 4, 6

    2020

  55. [63]

    A simple framework for open-vocabulary segmentation and detection

    Hao Zhang, Feng Li, Xueyan Zou, Shilong Liu, Chunyuan Li, Jianwei Yang, and Lei Zhang. A simple framework for open-vocabulary segmentation and detection. In ICCV, pages 1020–1031, 2023. 2, 6

    2023

  56. [64]

    Prototypical pseudo label denoising and tar- get structure learning for domain adaptive semantic segmen- tation

    Pan Zhang, Bo Zhang, Ting Zhang, Dong Chen, Yong Wang, and Fang Wen. Prototypical pseudo label denoising and tar- get structure learning for domain adaptive semantic segmen- tation. In CVPR, 2021. 2

    2021

  57. [65]

    Test-time adaptation with CLIP reward for zero-shot gener- alization in vision-language models

    Shuai Zhao, Xiaohan Wang, Linchao Zhu, and Yi Yang. Test-time adaptation with CLIP reward for zero-shot gener- alization in vision-language models. In ICLR, 2024. 3

    2024

  58. [66]

    Unsupervised scene adaptation with memory regularization in vivo

    Zhedong Zheng and Yi Yang. Unsupervised scene adaptation with memory regularization in vivo. In IJCAI, 2020. 2

    2020

  59. [67]

    Conditional prompt learning for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Zi- wei Liu. Conditional prompt learning for vision-language models. In CVPR, 2022. 3

    2022

  60. [68]

    Learning to prompt for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. IJCV,

  61. [69]

    Generalized decoding for pixel, image, and language

    Xueyan Zou, Zi-Yi Dou, Jianwei Yang, Zhe Gan, Linjie Li, Chunyuan Li, Xiyang Dai, Harkirat Behl, Jianfeng Wang, Lu Yuan, et al. Generalized decoding for pixel, image, and language. In CVPR, pages 15116–15127, 2023. 2, 6 11 Test-Time Optimization for Domain Adaptive Open V ocab...

    2023

  62. [70]

    feathered body

    dataset for attributes generated by Gemma-2B-Instruct (Gemma-2B) [15], Mistral-7B-Instruct-v0.2 (Mistral-7B) [25] and Meta-Llama-3-70B- Instruct (Llama3-70B) [2] LLMs. (a) Original Image (b) Ground Truth (d) Ours(c) Baseline Chicken Duck "feathered body" "beak", "webbed feet",...

  63. [71]

    They represent SOTA in their respective supervised and self- supervised approaches

    as baselines for evaluating our framework. They represent SOTA in their respective supervised and self- supervised approaches. Implementation of VFA in CA T-Seg: CAT-Seg [10] pro- cesses an image by diving it into overlapping patches. For each patch and the original image, two...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.