Pith. sign in

REVIEW 3 major objections 5 minor 84 references

DINOde: Continuous Vision-Text Alignment for Open-Vocabulary Semantic Segmentation

T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that replacing the single-step MLP projection between CLIP text and DINOv3 visual features with a continuous ODE trajectory — constrained to the tangent space of the unit sphere — preserves semantic topology and yields sta

desk verdict Useful OVSS method; the depth confound between the 10-step ODE and the single-step MLP baseline undercuts the 'continuous geometry' interpretation, but the broad empirical work deserves a serious referee. read the letter →

arxiv 2607.21371 v1 pith:U7IKS3QO submitted 2026-07-23 cs.CV cs.AI

Sung-Hoon Yoon , Hoyong Kwon , Changgyoon Oh , Kuk-Jin Yoon This is my paper
classification cs.CVcs.AI
keywords open-vocabularysemanticsegmentationneuralordinarydifferentialequationscross-modalalignmentDINOv3CLIPtext-to-visualflowmanifoldgeometrycontrastivelearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the gap between CLIP text embeddings and DINOv3 visual features can be bridged by learning a smooth, continuous transformation rather than a one-shot projection. It proposes DINOde, which evolves text embeddings along an ODE toward the DINO visual manifold, while a second flow refines the global CLS token. Velocity Tangent Projection keeps the trajectory on the hypersphere and preserves neighborhood structure that naive MLP mappings distort. The authors report 49.5 average mIoU across eight open-vocabulary segmentation benchmarks without mask refinement (50.1 with PAMR), outperforming the strongest comparable DINOv3-based baseline by 1.9 points.

What carries the argument

Semantic Text Flow (STF): a neural ODE that evolves a linearly projected CLIP text embedding from t=0 to t=1 under a learned velocity network, integrated with explicit Euler steps of size 1/10; after each step the state is re-normalized to the unit sphere. Velocity Tangent Projection (VTP): at each state, the raw velocity is projected onto the tangent space by subtracting its component along the current state, keeping the flow on the hypersphere. Global Context Flow (GCF): a second ODE that similarly evolves the DINOv3 CLS token. The two flows are trained jointly with a CLIP-style symmetric contrastive loss using pooled patch features plus the refined CLS token.

What would settle it

Train a depth-matched iterative residual MLP that uses the same 10 Euler-style steps and the same tangent projection as DINOde but without the ODE formulation (e.g., a residual MLP with shared weights and time conditioning removed). If it reaches the same 49.5 average mIoU across the eight benchmarks, the claimed advantage of continuous geometry is falsified; if it falls short, the ODE/tangent-flow framing is the cause.

Watch

Extended reading notes

Core claim

The central claim: continuous flow-based alignment from CLIP text space to DINOv3 visual space outperforms discrete MLP projection because the ODE trajectory, with velocities projected onto the tangent space of the unit sphere, preserves the relative geometry of class embeddings. The trained Semantic Text Flow maps text anchors into the DINO manifold over ten Euler steps; Global Context Flow refines the pooled visual representation; and the two are trained with a symmetric contrastive loss on image-text pairs. Geometric diagnostics (neighborhood overlap, geodesic correlation, class-structure preservation, alignment compactness) all favor the ODE mapping over the MLP baseline, and the method

Load-bearing premise

The central comparison is against a single-step MLP with a comparable parameter count; the ODE runs its velocity network ten times in sequence, so it is ten times deeper, and no depth-matched iterative MLP with the same tangent normalization is tested.

Editorial extensions

If this is right

  • If the claim holds, any single-step MLP projector between frozen modalities could be replaced by a short ODE with tangent projection, improving dense prediction without retraining the backbones.
  • The 10-step flow achieves results that previously required larger training data (118k captions vs millions), suggesting continuous alignment is data-efficient.
  • The progressive improvement with steps 1→10 indicates the trajectory itself encodes semantic structure; intermediate states could be used for confidence or interpretation.
  • Generalization across DINOv3 sizes and non-DINO backbones (SAM, SigLIP, iBOT, V-JEPA 2.1) suggests the benefit is not tied to DINO-specific features.
  • Higher-order solvers give no gain over Euler, so the learned velocity field is smooth; this is evidence that the ODE is not overfitting to the solver.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's ablations never test an iterative MLP with the same number of steps and the same tangent re-normalization; if such a depth-matched baseline matched DINOde's 49.5 mIoU, the reported gain would come from iterative refinement rather than from continuous ODE geometry. This is the key open experiment.
  • A testable extension: replace the single linear projector in a multimodal LLM's vision connector with a 10-step STF-style flow and measure fine-grained grounding (e.g., referring expression segmentation) against the MLP connector.
  • The ODE framing suggests a principled way to interpolate between classes: evolve intermediate text embeddings along the learned trajectory and inspect what the visual manifold says, possibly enabling few-shot class synthesis or interpretable class-relation analyses.
  • Because the flow is invertible and smooth, the learned velocity field could be used to probe the geometry of the DINO manifold — e.g., compute geodesic distances between classes and compare with human semantic judgments.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes DINOde, an ODE-based framework for open-vocabulary semantic segmentation built on a frozen DINOv3 visual encoder and a frozen CLIP text encoder. The method consists of Semantic Text Flow (STF), which evolves CLIP text embeddings along an Euler-integrated neural ODE toward the DINO visual manifold, and Global Context Flow (GCF), which similarly refines DINO's [CLS] token. Velocity Tangent Projection (VTP) constrains the learned velocity field to the tangent plane of the hypersphere. The model is trained with a symmetric contrastive loss on COCO caption data and evaluated on eight OVSS benchmarks. The paper reports an average mIoU of 49.5 without mask refinement and 50.1 with PAMR, outperforming a re-implemented DINOv3-based Talk2DINO baseline by 1.9 and 1.6 points, respectively. The central claim is that continuous ODE-based alignment preserves manifold geometry better than single-step discrete MLP projections.

Significance. If the central claim holds, the paper makes a useful conceptual and empirical contribution: it shows that modeling cross-modal alignment as a continuous flow can improve text-to-vision feature alignment for OVSS. The empirical scope is broad—eight benchmarks, multiple backbones, ablations, solver comparisons, PEFT baselines, and a reported code release—and the practical results are competitive. The paper also includes geometric diagnostics (Table 4), which are a welcome attempt to directly test the manifold-preservation hypothesis. However, the distinctive conceptual claim is currently not isolated empirically: the main comparison is confounded by the difference in iterative depth between the ODE and the MLP baseline, and the claimed geometric benefits rest on small differences without statistical uncertainty. The paper's strengths are its breadth of experiments and the clarity of the ODE formulation, but the central evidence needs a controlled depth-matched baseline.

major comments (3)
  1. [Section 3.2, Eq. (9); Table 7; S1.2] The central ODE-vs-MLP comparison is confounded by functional depth. STF evaluates the velocity network Nstep=10 times (17.70 ms per Table 7), while the MLP baseline is a single forward pass (1.05 ms). S1.2 matches parameter counts (7.5M vs 8.4M) but not inference depth. Since Eq. (3) itself identifies the Euler update as the continuous-depth limit of a residual network, the missing control is an iterative residual MLP with the same 10 shared-weight updates and the same per-step normalization/tangent projection. Without this control, the 1.8-point gap in Tables 1/2 (47.7→49.5) may be due to iterative refinement or increased depth, not to the continuous ODE geometry emphasized in the abstract and Section 3.2.
  2. [Table 1 and Table 4] The VTP ablation yields only about 0.3 mIoU in both paired comparisons ((b)→(c) and (d)→(e)), and the GCF contributions are also on the order of 0.7–1.0 mIoU. No error bars or repeated-seed variances are reported anywhere. In Table 4, the geometric diagnostics show differences of 0.01–0.03 without any uncertainty or significance assessment. Given the small increments, the component-level claims that VTP 'preserves geometry' and that STF 'avoids manifold entanglement' are not statistically supported. At minimum, report standard deviations over multiple seeds for the primary ablation and backbone tables.
  3. [Section 4.2 and Table 9] The PEFT baselines in Table 9 do not control for iterative depth either: LoRA, OFT, and COFT are single-pass adaptations of the text encoder, and the MLP baseline is single-pass. The statement in S2 that DINOde's improvement 'originates from the ODE-based cross-modal alignment itself' is therefore too strong. The paper should either add an iterative residual MLP control (with matching per-step normalization and tangent projection) or temper the conceptual claim that the gains come specifically from continuous geometry rather than from multi-step refinement.
minor comments (5)
  1. [Section 1, p. 2] Typo: 'the recent workdino.txt[24]' should be 'the recent work dino.txt [24]'.
  2. [Equation (6) and Fig. 2] The Pool operation is only defined as 'top-K pooling' in Eq. (6); the Min-Max Top-K definition appears only in the supplementary figure caption. Please move the formal definition to the main text or give a precise reference to [71].
  3. [Section 4.2, Table 2] The step-ablation in Table 2 shows performance at Nstep=5, 10, 50, but no corresponding iterative MLP at 5 or 10 steps. A simple multi-step MLP (same update rule, no ODE interpretation) would clarify the source of the gains.
  4. [Supplementary, RefCOCO+ discussion] The RefCOCO+ qualitative examples are only illustrative; no quantitative referring-expression segmentation results are provided. The claim that the aligned manifold 'generalizes to richer language' should be framed as anecdotal unless quantitative results are added.
  5. [Table 5] For the dino.txt row, several benchmarks are left blank. If those numbers are unavailable, a note explaining why would help; otherwise the comparison is incomplete.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the method is empirically trained on COCO captions and evaluated on external benchmarks; the central claim does not reduce to its inputs.

full rationale

DINOde is an empirical benchmark paper rather than a derivation from first principles. The velocity networks (v_theta, u_phi) are trained with a CLIP-style symmetric contrastive loss (Eq. 16-17) on COCO 2017 Caption image-caption pairs, and no benchmark mIoU is used as a fitting target. The reported predictions are dense cosine similarities between DINOv3 patch tokens and STF-aligned text anchors, evaluated on eight external OVSS benchmarks (Table 5). The ODE formulation (Eq. 1-3, 7-9) is a genuinely continuous-time construction: the Euler update z_{t+dt} = Norm(z_t + dt * tilde v_theta(z_t, t)) is not a restatement of the MLP baseline, and the VTP projection (Eq. 7) is an added geometric constraint. The MLP baseline (47.7 avg mIoU) is a reimplemented, parameter-matched single-step projector, and the STF/GCF/VTP ablations (Table 1) provide controlled internal comparisons. The paper does self-cite for top-K pooling (ref [71]) and weakly-supervised segmentation context (refs [16,28-30,72,73]), but these citations are not load-bearing for the central claim: pooling is a plug-in choice that is itself ablated against GAP in Fig. 6, and the WSSS references are contextual. The skeptic concern that the ODE vs MLP comparison is confounded by depth/iterations (10 Euler steps vs one MLP pass) is a legitimate experimental-design or correctness-risk issue, not circularity: the ODE baseline is not statistically forced by the target metric, and no reduction of a predicted quantity to a fitted input or to a self-citation chain is exhibited. The geometric diagnostics in Table 4 are post-hoc measurements, not circular definitions. Therefore the paper receives a score of 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or conceptual entities: Velocity Tangent Projection is an operation on the velocity field, not a new object. The main free parameters are standard hyperparameters, and the domain assumptions are inherited from the choice of frozen backbones and datasets.

free parameters (4)
  • Number of ODE steps Nstep = 10
    Ablation in Table 2 shows Nstep=5 already beats MLP; Nstep=10 chosen as trade-off between mIoU and compute. No theoretical justification for the specific number.
  • Top-K pooling parameter K = 20
    Ablation in Fig. 6 shows peak at K=10 but K=20 selected for 'training stability'; changes the pooled visual target in Eq. 6 and affects results.
  • Contrastive temperature tau = 0.07
    Used in Eq. 16 similarity matrix; taken from CLIP convention, not tuned in this paper.
  • Training schedule (lr, wd, epochs, batch) = 1e-4, 0.01, 20, 256
    AdamW settings chosen without reported sensitivity analysis; could affect the magnitude of the reported gains.
assumptions (5)
  • domain assumption Frozen DINOv3 patch tokens contain sufficient semantic structure for dense open-vocabulary classification once aligned with text.
    The entire method depends on DINOv3's self-supervised features being a good visual manifold for zero-shot segmentation; if they lack class-level semantics, aligning CLIP text to them cannot recover it.
  • standard math CLIP text embeddings lie on a hypersphere and cosine similarity is a valid semantic measure for classification.
    All features are normalized and both loss and inference use cosine similarity; this is standard CLIP-based practice.
  • standard math Euler integration with Nstep=10 is an adequate numerical approximation of the continuous ODE.
    Eq. 9 and Eq. 14 use Euler steps; Table 8 shows RK2/RK4 give nearly identical results, supporting this assumption empirically.
  • domain assumption COCO 2017 Captions (118k images) provides enough supervision to align CLIP text embeddings to DINOv3 visual features.
    The SOTA claim depends on this dataset being sufficient; no learning-curve analysis with other dataset sizes is shown.
  • domain assumption Pixel-Adaptive Mask Refinement does not bias the comparison toward the proposed method.
    PAMR is applied to all methods following prior work, but if it interacts differently with DINOde outputs the refined mIoU gains could be inflated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DINOde: Continuous Vision-Text Alignment for Open-Vocabulary Semantic Segmentation." pith.science (2026). https://pith.science/paper/U7IKS3QO

@misc{pith2026260721371,
  author       = {Pith},
  title        = {Pith review of: DINOde: Continuous Vision-Text Alignment for Open-Vocabulary Semantic Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U7IKS3QO}},
  note         = {Machine review of arXiv:2607.21371}
}
read the original abstract

Open-vocabulary semantic segmentation (OVSS) leverages textual semantics to segment objects beyond predefined categories. While the self-supervised model DINOv3 provides strong structured visual representations, its lack of native textual alignment hinders its direct application to OVSS. To bridge this gap, we propose DINOde, an ODE-based framework that continuously aligns CLIP text embeddings with the DINO visual manifold. Our approach employs two complementary components: (i) Semantic Text Flow (STF), which evolves text embeddings toward the DINO manifold through a continuous ODE trajectory, and (ii) Global Context Flow (GCF), which progressively refines the holistic image representation carried by DINO's CLS token. To preserve the hyperspherical geometry of the feature space during this evolution, we further introduce Velocity Tangent Projection, which constrains the learned velocity field to the tangent space. By modeling alignment as a continuous trajectory, DINOde avoids the manifold entanglement inherent in discrete MLP projections and yields more robust cross-modal alignment. Extensive experiments demonstrate that DINOde consistently outperforms existing methods and achieves state-of-the-art performance across multiple OVSS benchmarks. The code is available at https://github.com/yoon307/DINOde.

Figures

Figures reproduced from arXiv: 2607.21371 by the authors.

Figure 1
Figure 1. Overview of the DINOde framework for OVSS [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed DINOde framework. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Analysis of Semantic Text Flow (STF) with varying numbers of ODE [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison with the state-of-the-art methods. The result of Talk2DINO* is obtained using the DINOv3 backbone for a fair comparison. erogeneous representation manifolds through a single-step mapping, which often fails to preserve the intrinsic geometry of th…
Figure 5
Figure 5. Figure 5: Illustration of network architectures. (a) MLP baseline, (b) Velocity network of DINOde. dataset. Here, we compare a discrete single-step projection (MLP) with the pro￾posed Semantic Text Flow (STF) evaluated using different numbers of Euler integration steps (Nstep). …
Figure 6
Figure 6. Figure 6: Ablation study on the pooling strategy and hyperparameter K. Ablation Study on Pooling. Regarding the pooling function “Pool” intro￾duced in Eq. 6 of the main paper, we conducted an ablation study to evaluate the impact of different pooling strategies and the hyperpara…
Figure 7
Figure 7. Figure 7: Additional Qualitative comparison with the state-of-the-art methods. The re￾sult of Talk2DINO* is obtained using the DINOv3 backbone for a fair comparison. Language: the black and white teddy bear the wooden bench the man in a blue coat walking through the park the lea…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

84 extracted references · 10 linked inside Pith

  1. [1]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition

    Araslanov, N., Roth, S.: Single-stage semantic segmentation from image labels. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition. pp. 4253–4262 (2020)

    2020

  2. [2]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Barsellotti, L., Amoroso, R., Cornia, M., Baraldi, L., Cucchiara, R.: Training-free open-vocabulary segmentation with offline diffusion-augmented prototype gener- ation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3689–3698 (2024)

    2024

  3. [3]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Barsellotti,L.,Bianchi,L.,Messina,N.,Carrara,F.,Cornia,M.,Baraldi,L.,Falchi, F., Cucchiara, R.: Talking to dino: Bridging self-supervised vision backbones with language for open-vocabulary segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 22025–22035 (2025)

    2025

  4. [4]

    In: International conference on machine learning

    Behrmann, J., Grathwohl, W., Chen, R.T., Duvenaud, D., Jacobsen, J.H.: In- vertible residual networks. In: International conference on machine learning. pp. 573–582. PMLR (2019)

    2019

  5. [5]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Bousselham, W., Petersen, F., Ferrari, V., Kuehne, H.: Grounding everything: Emerging localization properties in vision-language transformers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3828–3837 (2024)

    2024

  6. [6]

    In:ProceedingsoftheIEEEconferenceoncomputervisionandpatternrecognition

    Caesar, H., Uijlings, J., Ferrari, V.: Coco-stuff: Thing and stuff classes in context. In:ProceedingsoftheIEEEconferenceoncomputervisionandpatternrecognition. pp. 1209–1218 (2018)

    2018

  7. [7]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Cai, K., Ren, P., Zhu, Y., Xu, H., Liu, J., Li, C., Wang, G., Liang, X.: Mixreorg: Cross-modal mixed patch reorganization is a good mask learner for open-world se- mantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 1196–1205 (2023)

    2023

  8. [8]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging properties in self-supervised vision transformers. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 9650–9660 (2021)

    2021

Show all 84 references
  1. [9]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Cha, J., Mun, J., Roh, B.: Learning to generate text-grounded mask for open- world semantic segmentation from only image-text pairs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11165– 11174 (2023)

    2023

  2. [10]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Changpinyo, S., Sharma, P., Ding, N., Soricut, R.: Conceptual 12m: Pushing web- scale image-text pre-training to recognize long-tail visual concepts. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 3558–3568 (2021)

    2021

  3. [11]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Chen, J., Zhu, D., Qian, G., Ghanem, B., Yan, Z., Zhu, C., Xiao, F., Culatana, S.C., Elhoseiny, M.: Exploring open-vocabulary semantic segmentation from clip vision encoder distillation only. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 699–...

    2023

  4. [12]

    In: Proceedings of the European conference on computer vision (ECCV)

    Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European conference on computer vision (ECCV). pp. 801–818 (2018)

    2018

  5. [13]

    Advances in neural information processing systems31(2018)

    Chen, R.T., Rubanova, Y., Bettencourt, J., Duvenaud, D.K.: Neural ordinary dif- ferential equations. Advances in neural information processing systems31(2018)

    2018

  6. [14]

    arXiv preprint arXiv:1504.00325 (2015) DINOde: Continuous Vision-Text Alignment for OVSS 17

    Chen, X., Fang, H., Lin, T.Y., Vedantam, R., Gupta, S., Dollár, P., Zitnick, C.L.: Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325 (2015) DINOde: Continuous Vision-Text Alignment for OVSS 17

    2015 arXiv

  7. [15]

    Advances in neural information processing systems34, 17864–17875 (2021)

    Cheng, B., Schwing, A., Kirillov, A.: Per-pixel classification is not all you need for semantic segmentation. Advances in neural information processing systems34, 17864–17875 (2021)

    2021

  8. [16]

    In: European Conference on Computer Vision

    Cho, H., Yoon, S.H., Kweon, H., Yoon, K.J.: Finding meaning in points: Weakly supervised semantic segmentation for event cameras. In: European Conference on Computer Vision. pp. 266–286. Springer (2024)

    2024

  9. [17]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., Schiele, B.: The cityscapes dataset for semantic urban scene understanding. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3213–3223 (2016)

    2016

  10. [18]

    arXiv preprint arXiv:2309.16588 (2023)

    Darcet, T., Oquab, M., Mairal, J., Bojanowski, P.: Vision transformers need reg- isters. arXiv preprint arXiv:2309.16588 (2023)

    2023 arXiv

  11. [19]

    Advances in neural information processing systems32(2019)

    Dupont, E., Doucet, A., Teh, Y.W.: Augmented neural odes. Advances in neural information processing systems32(2019)

    2019

  12. [20]

    Everingham,M.,VanGool,L.,Williams,C.K.,Winn,J.,Zisserman,A.:Thepascal visualobjectclasses(voc)challenge.Internationaljournalofcomputervision88(2), 303–338 (2010)

    2010

  13. [21]

    arXiv preprint arXiv:2506.03706 (2025)

    Gandhamal, A., Sikdar, A., Sundaram, S.: Ov-coast: Cost aggregation with optimal transport for open-vocabulary semantic segmentation. arXiv preprint arXiv:2506.03706 (2025)

    2025 arXiv

  14. [22]

    In: European conference on computer vision

    Ghiasi,G.,Gu,X.,Cui,Y.,Lin,T.Y.:Scalingopen-vocabularyimagesegmentation with image-level labels. In: European conference on computer vision. pp. 540–557. Springer (2022)

    2022

  15. [23]

    In: 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

    Hajimiri, S., Ayed, I.B., Dolz, J.: Pay attention to your neighbours: Training-free open-vocabulary semantic segmentation. In: 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). pp. 5061–5071. IEEE (2025)

    2025

  16. [24]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Jose, C., Moutakanni, T., Kang, D., Baldassarre, F., Darcet, T., Xu, H., Li, D., Szafraniec, M., Ramamonjisoa, M., Oquab, M., et al.: Dinov2 meets text: A unified framework for image-and pixel-level vision-language alignment. In: Proceedings of the Computer Vision and Pattern ...

    2025

  17. [25]

    In: European Conference on Computer Vision

    Kang, D., Cho, M.: In defense of lazy visual grounding for open-vocabulary se- mantic segmentation. In: European Conference on Computer Vision. pp. 143–164. Springer (2024)

    2024

  18. [26]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Kim, J., Kwon, H., Kweon, H., Yoon, K.J.: Bootstrapping video semantic segmen- tation model via distillation-assisted test-time adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 10766– 10777 (2026)

    2026

  19. [27]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y., et al.: Segment anything. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 4015–4026 (2023)

    2023

  20. [28]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Kweon, H., Yoon, S.H., Kim, H., Park, D., Yoon, K.J.: Unlocking the potential of ordinary classifier: Class-specific adversarial erasing framework for weakly su- pervised semantic segmentation. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 69...

    2021

  21. [29]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Kweon, H., Yoon, S.H., Yoon, K.J.: Weakly supervised semantic segmentation via adversarial learning of classifier and reconstructor. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 11329– 11339 (2023)

    2023

  22. [30]

    In: European Conference on Computer Vision

    Kwon, H., Jeong, J., Yoon, S.H., Yoon, K.J.: Phase concentration and shortcut suppression for weakly supervised semantic segmentation. In: European Conference on Computer Vision. pp. 293–312. Springer (2024) 18 Yoon et al

    2024

  23. [31]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Lai, Z.: Exploring simple open-vocabulary semantic segmentation. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 30221–30230 (2025)

    2025

  24. [32]

    In: European Conference on Computer Vision

    Lan, M., Chen, C., Ke, Y., Wang, X., Feng, L., Zhang, W.: Clearclip: Decomposing clip representations for dense vision-language inference. In: European Conference on Computer Vision. pp. 143–160. Springer (2024)

    2024

  25. [33]

    In: European Conference on Computer Vision

    Lan, M., Chen, C., Ke, Y., Wang, X., Feng, L., Zhang, W.: Proxyclip: Proxy at- tention improves clip for open-vocabulary segmentation. In: European Conference on Computer Vision. pp. 70–88. Springer (2024)

    2024

  26. [34]

    In: International Conference on Learning Representations (2022),https://openreview.net/forum?id=RriDjddCLN

    Li, B., Weinberger, K.Q., Belongie, S., Koltun, V., Ranftl, R.: Language-driven semantic segmentation. In: International Conference on Learning Representations (2022),https://openreview.net/forum?id=RriDjddCLN

    2022

  27. [35]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition

    Liang, F., Wu, B., Dai, X., Li, K., Zhao, Y., Zhang, H., Zhang, P., Vajda, P., Mar- culescu, D.: Open-vocabulary semantic segmentation with mask-adapted clip. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition. pp. 7061–7070 (2023)

    2023

  28. [36]

    In: European conference on computer vision

    Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: European conference on computer vision. pp. 740–755. Springer (2014)

    2014

  29. [37]

    arXiv preprint arXiv:2210.02747 (2022)

    Lipman, Y., Chen, R.T., Ben-Hamu, H., Nickel, M., Le, M.: Flow matching for generative modeling. arXiv preprint arXiv:2210.02747 (2022)

    2022 arXiv

  30. [38]

    arXiv preprint arXiv:2209.03003 (2022)

    Liu, X., Gong, C., Liu, Q.: Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003 (2022)

    2022 arXiv

  31. [39]

    Advances in Neural Information Processing Systems33, 17548–17558 (2020)

    Lou, A., Lim, D., Katsman, I., Huang, L., Jiang, Q., Lim, S.N., De Sa, C.M.: Neural manifold ordinary differential equations. Advances in Neural Information Processing Systems33, 17548–17558 (2020)

    2020

  32. [40]

    In: International Conference on Machine Learning

    Luo, H., Bao, J., Wu, Y., He, X., Li, T.: Segclip: Patch aggregation with learnable centers for open-vocabulary semantic segmentation. In: International Conference on Machine Learning. pp. 23033–23044. PMLR (2023)

    2023

  33. [41]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Luo, J., Khandelwal, S., Sigal, L., Li, B.: Emergent open-vocabulary semantic segmentation from off-the-shelf vision-language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 4029– 4040 (2024)

    2024

  34. [42]

    Advances in neural information processing systems33, 2503–2515 (2020)

    Mathieu, E., Nickel, M.: Riemannian continuous normalizing flows. Advances in neural information processing systems33, 2503–2515 (2020)

    2020

  35. [43]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Mottaghi, R., Chen, X., Liu, X., Cho, N.G., Lee, S.W., Fidler, S., Urtasun, R., Yuille, A.: The role of context for object detection and semantic segmentation in the wild. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 891–898 (2014)

    2014

  36. [44]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Mukhoti, J., Lin, T.Y., Poursaeed, O., Wang, R., Shah, A., Torr, P.H., Lim, S.N.: Open vocabulary semantic segmentation with patch aligned contrastive learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 19413–19423 (2023)

    2023

  37. [45]

    arXiv preprint arXiv:2603.14482 (2026)

    Mur-Labadia, L., Muckley, M., Bar, A., Assran, M., Sinha, K., Rabbat, M., Le- Cun, Y., Ballas, N., Bardes, A.: V-jepa 2.1: Unlocking dense features in video self-supervised learning. arXiv preprint arXiv:2603.14482 (2026)

    2026 arXiv

  38. [46]

    arXiv preprint arXiv:2304.07193 (2023) DINOde: Continuous Vision-Text Alignment for OVSS 19

    Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al.: Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 (2023) DINOde: Continuous Vision-Text Alignment...

    2023 arXiv

  39. [47]

    In: International conference on machine learning

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PMLR (2021)

    2021

  40. [48]

    In: Inter- national conference on machine learning

    Rezende, D., Mohamed, S.: Variational inference with normalizing flows. In: Inter- national conference on machine learning. pp. 1530–1538. PMLR (2015)

    2015

  41. [49]

    In: International Conference on Medical image computing and computer-assisted intervention

    Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedi- cal image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. pp. 234–241. Springer (2015)

    2015

  42. [50]

    In: Proceed- ings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Sharma, P., Ding, N., Goodman, S., Soricut, R.: Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In: Proceed- ings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 2556–25...

    2018

  43. [51]

    Advances in Neural Information Processing Systems35, 33754–33767 (2022)

    Shin,G.,Xie,W.,Albanie,S.:Reco:Retrieveandco-segmentforzero-shottransfer. Advances in Neural Information Processing Systems35, 33754–33767 (2022)

    2022

  44. [52]

    arXiv preprint arXiv:2109.14279 (2021)

    Siméoni, O., Puy, G., Vo, H.V., Roburin, S., Gidaris, S., Bursuc, A., Pérez, P., Marlet, R., Ponce, J.: Localizing objects with self-supervised transformers and no labels. arXiv preprint arXiv:2109.14279 (2021)

    2021 arXiv

  45. [53]

    arXiv preprint arXiv:2508.10104 (2025)

    Siméoni, O., Vo, H.V., Seitzer, M., Baldassarre, F., Oquab, M., Jose, C., Khali- dov, V., Szafraniec, M., Yi, S., Ramamonjisoa, M., et al.: Dinov3. arXiv preprint arXiv:2508.10104 (2025)

    2025 arXiv

  46. [54]

    In: proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Sun, G., Liu, Y., Ding, H., Probst, T., Van Gool, L.: Coarse-to-fine feature mining for video semantic segmentation. In: proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 3126–3137 (2022)

    2022

  47. [55]

    IEEE Transactions on Pattern Analysis and Machine Intelligence (2024)

    Sun, G., Liu, Y., Ding, H., Wu, M., Van Gool, L.: Learning local and global tem- poral contexts for video semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence (2024)

    2024

  48. [56]

    In: Proceed- ings of the IEEE/CVF International Conference on Computer Vision

    Sun, L., Cao, J., Xie, J., Jiang, X., Pang, Y.: Cliper: Hierarchically improving spa- tial representation of clip for open-vocabulary semantic segmentation. In: Proceed- ings of the IEEE/CVF International Conference on Computer Vision. pp. 23199– 23209 (2025)

    2025

  49. [57]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Sun, S., Li, R., Torr, P., Gu, X., Li, S.: Clip as rnn: Segment countless visual concepts without training endeavor. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 13171–13182 (2024)

    2024

  50. [58]

    In: European conference on computer vision

    Wang, F., Mei, J., Yuille, A.: Sclip: Rethinking self-attention for dense vision- language inference. In: European conference on computer vision. pp. 315–332. Springer (2024)

    2024

  51. [59]

    IEEE transactions on pattern analysis and machine intelligence45(12), 15790–15801 (2023)

    Wang, Y., Shen, X., Yuan, Y., Du, Y., Li, M., Hu, S.X., Crowley, J.L., Vaufreydaz, D.: Tokencut: Segmenting objects in images and videos with self-supervised trans- former and normalized cut. IEEE transactions on pattern analysis and machine intelligence45(12), 15790–15801 (2023)

    2023

  52. [60]

    Wysoczańska, M., Ramamonjisoa, M., Trzciński, T., Siméoni, O.: Clip-diy: Clip denseinferenceyieldsopen-vocabularysemanticsegmentationfor-free.In:Proceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 1403–1413 (2024)

    2024

  53. [61]

    In: European Conference on Computer Vision

    Wysoczańska, M., Siméoni, O., Ramamonjisoa, M., Bursuc, A., Trzciński, T., Pérez, P.: Clip-dinoiser: Teaching clip a few dino tricks for open-vocabulary se- mantic segmentation. In: European Conference on Computer Vision. pp. 320–337. Springer (2024) 20 Yoon et al

    2024

  54. [62]

    Advances in neural information processing systems34, 12077–12090 (2021)

    Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J.M., Luo, P.: Segformer: Simple and efficient design for semantic segmentation with transformers. Advances in neural information processing systems34, 12077–12090 (2021)

    2021

  55. [63]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Xie, J., Hou, X., Ye, K., Shen, L.: Clims: Cross language image matching for weakly supervised semantic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 4483–4492 (2022)

    2022

  56. [64]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Xu, J., De Mello, S., Liu, S., Byeon, W., Breuel, T., Kautz, J., Wang, X.: Groupvit: Semantic segmentation emerges from text supervision. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 18134– 18144 (2022)

    2022

  57. [65]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Xu, J., Liu, S., Vahdat, A., Byeon, W., Wang, X., De Mello, S.: Open-vocabulary panoptic segmentation with text-to-image diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 2955–2966 (2023)

    2023

  58. [66]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition

    Xu, J., Hou, J., Zhang, Y., Feng, R., Wang, Y., Qiao, Y., Xie, W.: Learning open- vocabulary semantic segmentation models from natural language supervision. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition. pp. 2935–2944 (2023)

    2023

  59. [67]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Xu, L., Ouyang, W., Bennamoun, M., Boussaid, F., Xu, D.: Multi-class token transformer for weakly supervised semantic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 4310–4319 (2022)

    2022

  60. [68]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Xu, M., Zhang, Z., Wei, F., Hu, H., Bai, X.: Side adapter network for open- vocabulary semantic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 2945–2954 (2023)

    2023

  61. [69]

    In: European conference on computer vision

    Xu, M., Zhang, Z., Wei, F., Lin, Y., Cao, Y., Hu, H., Bai, X.: A simple baseline for open-vocabulary semantic segmentation with pre-trained vision-language model. In: European conference on computer vision. pp. 736–753. Springer (2022)

    2022

  62. [70]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Yi, M., Cui, Q., Wu, H., Yang, C., Yoshie, O., Lu, H.: A simple framework for text- supervised semantic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 7071–7080 (2023)

    2023

  63. [71]

    In: European conference on computer vision

    Yoon, S.H., Kweon, H., Cho, J., Kim, S., Yoon, K.J.: Adversarial erasing frame- work via triplet with gated pyramid pooling layer for weakly supervised semantic segmentation. In: European conference on computer vision. pp. 326–344. Springer (2022)

    2022

  64. [72]

    In: European Conference on Computer Vision

    Yoon, S.H., Kwon, H., Jeong, J., Park, D., Yoon, K.J.: Diffusion-guided weakly supervised semantic segmentation. In: European Conference on Computer Vision. pp. 393–411. Springer (2024)

    2024

  65. [73]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Yoon, S.H., Kwon, H., Kim, H., Yoon, K.J.: Class tokens infusion for weakly su- pervised semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3595–3605 (2024)

    2024

  66. [74]

    In: European conference on computer vision

    Yu, L., Poirson, P., Yang, S., Berg, A.C., Berg, T.L.: Modeling context in refer- ring expressions. In: European conference on computer vision. pp. 69–85. Springer (2016)

    2016

  67. [75]

    Advances in Neural Information Processing Systems36, 32215–32234 (2023)

    Yu, Q., He, J., Deng, X., Shen, X., Chen, L.C.: Convolutions die hard: Open- vocabulary segmentation with single frozen convolutional clip. Advances in Neural Information Processing Systems36, 32215–32234 (2023)

    2023

  68. [76]

    Yuan, Y., Chen, X., Wang, J.: Object-contextual representations for semantic seg- mentation.In:ComputerVision–ECCV2020:16thEuropeanConference,Glasgow, UK, August 23–28, 2020, Proceedings, Part VI 16. pp. 173–190. Springer (2020) DINOde: Continuous Vision-Text Alignment for OVSS 21

    2020

  69. [77]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Zhai, X., Mustafa, B., Kolesnikov, A., Beyer, L.: Sigmoid loss for language im- age pre-training. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 11975–11986 (2023)

    2023

  70. [78]

    Zhang, H., Li, F., Zou, X., Liu, S., Li, C., Yang, J., Zhang, L.: A simple framework foropen-vocabularysegmentationanddetection.In:ProceedingsoftheIEEE/CVF International Conference on Computer Vision. pp. 1020–1031 (2023)

    2023

  71. [79]

    In: International Conference on Machine Learning (2023)

    Zheng Ding, Jieke Wang, Z.T.: Open-vocabulary universal image segmentation with maskclip. In: International Conference on Machine Learning (2023)

    2023

  72. [80]

    Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., Torralba, A.: Scene parsing throughade20kdataset.In:ProceedingsoftheIEEEconferenceoncomputervision and pattern recognition. pp. 633–641 (2017)

    2017

  73. [81]

    International journal of computer vision127(3), 302–321 (2019)

    Zhou, B., Zhao, H., Puig, X., Xiao, T., Fidler, S., Barriuso, A., Torralba, A.: Se- mantic understanding of scenes through the ade20k dataset. International journal of computer vision127(3), 302–321 (2019)

    2019

  74. [82]

    In: European conference on computer vision

    Zhou, C., Loy, C.C., Dai, B.: Extract free dense labels from clip. In: European conference on computer vision. pp. 696–712. Springer (2022)

    2022

  75. [83]

    arXiv preprint arXiv:2111.07832 (2021)

    Zhou, J., Wei, C., Wang, H., Shen, W., Xie, C., Yuille, A., Kong, T.: ibot: Image bert pre-training with online tokenizer. arXiv preprint arXiv:2111.07832 (2021)

    2021 arXiv

  76. [84]

    In: Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition

    Zou, X., Dou, Z.Y., Yang, J., Gan, Z., Li, L., Li, C., Dai, X., Behl, H., Wang, J., Yuan, L., et al.: Generalized decoding for pixel, image, and language. In: Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 15116–15127 (2023) 22 Yoon et...

    2023

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.