Pith. sign in

REVIEW 5 major objections 8 minor 60 references

Towards Open-Vocabulary Video Semantic Segmentation

T0 review · 5 major / 8 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Adding temporal context to open-vocabulary segmentation lifts unseen-class mIoU to 17.99% on VSPW, beating image-based methods by ~4 points.

desk verdict Defines a genuinely new task and a sensible baseline, but the evaluation protocol (arbitrary split, input masking) and implausible Cityscapes numbers make the headline 4-point gain unproven. read the letter →

arxiv 2412.09329 v1 pith:G4XRBRVM submitted 2024-12-12 cs.MM cs.AI

Xinhao Li , Yun Liu , Guolei Sun , Min Wu , Le Zhang , Ce Zhu This is my paper · ORCID
classification cs.MMcs.AI
keywords open-vocabularyvideosemanticsegmentationzero-shottemporalcontextfusionvision-languagemodelssceneparsingcross-datasetgeneralizationcostvolumetextencoding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces open-vocabulary video semantic segmentation (OV-VSS) as a distinct task: labeling every pixel in each video frame for categories the model has never seen during training, not just objects but also background classes. To show the task is tractable, the authors propose OV2VSS, a video-native baseline that fuses features across neighboring frames, pulls long-range context from a randomly selected distant frame, and refines CLIP text embeddings with video visual features. On the VSPW benchmark, with the first 80 classes as seen and the remaining 44 as unseen, OV2VSS attains 17.99% mIoU on unseen classes, about 4 points above the strongest image-based open-vocabulary segmenter. The same model, trained only on VSPW, also transfers directly to Cityscapes and segments novel urban categories without fine-tuning. If these numbers hold, temporal context and video-aware text encoding are real ingredients for open-world video understanding.

What carries the argument

The central mechanism is OV2VSS's spatio-temporal feature aggregation built on top of CLIP's frozen encoders. The Spatial-Temporal Context Fusion module gradually accumulates cross-frame attention maps from the closest neighboring frame outward, using multi-scale affinity aggregation to exchange information across feature scales; the Random Frame Enhancement module then cross-attends the target frame's fused features to a context representation from a temporally distant frame; and the Video Text Encoding module refines CLIP's text embeddings with multi-head self-attention over video visual features before computing a cosine-similarity cost volume with image features, which is concatenated with shallow backbone features for position awareness.

What would settle it

Re-run the VSPW unseen-class evaluation with a split that holds out entire semantic super-categories (e.g., all animal or all vehicle classes) instead of the last 44 categories in the list, and randomize the class order; if the 17.99% mIoU and the ~4-point lead over FreeSeg shrink substantially, the claimed zero-shot advantage is an artifact of the easy split.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that open-vocabulary video semantic segmentation is a meaningful benchmark and that a video-native architecture can substantially outperform per-frame image methods at zero-shot generalization. The central evidence is the VSPW evaluation: with the category list split 80 seen / 44 unseen and unseen-class pixels masked out during training, OV2VSS reaches 17.99% mIoU on unseen classes, versus 13.13% for FreeSeg and 6.56% for SAN on the same backbones and protocol. The paper also shows cross-dataset transfer to Cityscapes, where the VSPW-trained model achieves 27.65% mIoU on ResNet-101, well above image-based comparisons. The authors attribute the gains to three video-specific components: spatial-temporal context fusion, random-frame enhancement, and video text encoding, with ablations showing each contributes.

Load-bearing premise

The evaluation assumes that splitting VSPW's 124 classes by category order and masking out the last 44 in training frames produces a fair measure of open-vocabulary generalization; if those unseen classes are visually similar to the seen ones, or if the masking removes signal the model needs, the reported 17.99% mIoU overstates zero-shot capability.

Editorial extensions

If this is right

  • Video semantic segmentation should be treated as a distinct open-vocabulary task, since temporal context measurably improves accuracy on novel classes relative to per-frame image methods.
  • A single temporally distant frame, randomly chosen during training, can provide useful long-range context without processing the whole video.
  • Refining CLIP text embeddings with visual features from the video improves pixel-text alignment compared with using frozen text embeddings.
  • A model trained on 80 VSPW classes can segment novel classes in Cityscapes without any fine-tuning, supporting cross-dataset zero-shot deployment.
  • Pixel-level open-vocabulary labeling of both objects and background regions is achievable in video, going beyond instance-centric open-vocabulary approaches.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A stricter comparison would ablate the video text encoding while keeping temporal fusion, and test a per-frame image method that receives the same multiple frames (e.g., ensembling frame predictions); the paper's ablations do not isolate the temporal gain against the exact strongest baseline.
  • Because the unseen classes are masked in training, the model never sees their appearance; a stress test with semantically disjoint held-out super-categories would tell whether the gain comes from true open-vocabulary recognition or from visual similarity between seen and unseen classes.
  • The video text encoding idea is likely transferable to other video-language grounding tasks, such as open-vocabulary video instance segmentation or dense video captioning.
  • The random-frame selection could be made content-aware (for example, choosing the frame with the largest feature change) to give even stronger long-range context; the current design uses a uniformly random choice.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 8 minor

Summary. The manuscript introduces the task of open-vocabulary video semantic segmentation (OV-VSS) and proposes OV2VSS, a baseline that extends per-frame CLIP-based open-vocabulary segmentation with three video-specific components: a spatial-temporal context fusion module that progressively aggregates features from neighboring frames, a random frame enhancement module that injects long-range context from a distant frame, and a video text encoding module that refines CLIP text features with visual context. Experiments on VSPW and Cityscapes report an unseen-class mIoU of 17.99% on VSPW with a ViT-B2 backbone, a claimed 4-percentage-point improvement over image-based open-vocabulary methods, and large gains in a cross-dataset Cityscapes evaluation. The paper also presents ablations of each module and a complexity analysis.

Significance. If the claims are supported, the paper would be a useful first step for OV-VSS: it identifies a real gap, proposes a simple and plausible temporal extension of image-level open-vocabulary segmenters, and shows large apparent gains. The manuscript is generally non-circular: the reported unseen-class performance is a measured quantity, and the main external dependence is on CLIP's pretrained representations, which is an input assumption rather than a fitted constant. The ablation design is informative, and the authors commit to releasing source code, which would aid reproducibility. However, the current evidence is weakened by the non-standard evaluation protocol and by internal inconsistencies in the reported cross-dataset numbers, so the significance of the specific state-of-the-art claim is not yet established.

major comments (5)
  1. [Section IV-C, Evaluation Protocol] The evaluation protocol rests on an arbitrary 80/44 split defined by VSPW's category order rather than on a semantic or super-category split. Prior open-vocabulary segmentation evaluations, such as Zegformer [40], deliberately separate super-categories to avoid near-duplicate classes across the seen/unseen boundary. The manuscript gives no analysis of the last 44 classes, such as class frequency, visual similarity to the first 80, or per-class IoU, so the reported 17.99% unseen mIoU and the '4% improvement' are not yet comparable to established open-vocabulary protocols. This assumption is load-bearing because the entire state-of-the-art comparison in Table I depends on the split being a fair test. Please validate the split with class statistics and at least one alternative split, or adopt a standard protocol.
  2. [Section IV-C, Evaluation Protocol] The protocol masks invisible-class pixels in the input video frames before training, which is non-standard. In open-vocabulary segmentation the input image is normally kept intact and only labels for unseen classes are ignored; masking creates a train/test distribution shift because test frames contain the full scene. If the masking was applied only to OV2VSS, the comparison in Table I is confounded; if applied to all methods, the absolute numbers are not directly comparable with the existing image-based open-vocabulary literature. Please report results both with and without input masking, and with the standard label-only masking, so that the effect of this design choice can be assessed.
  3. [Table II, Cross-Dataset Results] The Cityscapes cross-dataset table contains internal inconsistencies that need to be resolved. FreeSeg is reported with mIoU 8.37, fwIoU 3.19, and pAcc 6.07; a pixel accuracy of 6.07% is implausible for Cityscapes, where even a constant prediction toward a dominant class would score substantially higher. In addition, the ordering of methods reverses relative to VSPW: SAN improves from 6.56 to 14.68 mIoU while FreeSeg drops from 13.13 to 8.37, despite FreeSeg being much stronger on VSPW. Please verify the numbers, state precisely which of the Cityscapes classes are treated as unseen, and specify how unlabeled pixels are handled in each metric.
  4. [Section I and Table I, Backbone Control] The headline 4% improvement is presented without consistently controlling for backbone. The 17.99% result uses the ViT-B2 backbone, while the image-based FreeSeg result of 13.13% uses ResNet-101. The same-backbone ResNet-101 comparison, Ours 17.22 vs. FreeSeg 13.13, is the relevant one for the headline claim and should be presented as such, or ViT-B2 numbers should be provided for all baselines so that backbone differences are not entangled with method differences.
  5. [Section IV and Table III, Statistical Reliability] All experiments appear to be single runs with no error bars or significance testing, and hyperparameters such as the loss weights alpha and beta in Eq. (15) are selected using the same split on which the final numbers are reported. The differences between ablation configurations, for example 14.88% versus 17.99% mIoU, could in principle be within run-to-run variation. Please report multiple seeds or, at minimum, state explicitly that all numbers come from a single run and that no variance estimate is available.
minor comments (8)
  1. [Title and Abstract] There is a typo in the title and abstract: 'V ocabulary' should be 'Vocabulary', and 'out experiments' in Section IV.A should read 'our experiments'.
  2. [Section IV.A and IV.C] The number of VSPW validation frames is reported as 24,392 images in Section IV.C but as 24,502 frames in Section IV.A; please reconcile the two numbers and clarify whether the evaluation is on the validation set or the test set, since both terms are used.
  3. [Section IV.F, Loss Ratio Ablation] In the ablation text for the loss ratio, the sentence 'Keeping alpha at 1 and reducing beta to 0.1' is repeated; the second occurrence should refer to beta = 10, matching Table III.
  4. [Section III.A, Eq. (3)] Equation (3) defines Q_ti as a function of U_ti while K_past and V_past are functions of D_past; the feature-source notation should be made consistent so the reader can follow which feature maps are projected into queries, keys, and values.
  5. [Section IV.B] The default values of the loss weights alpha and beta in Eq. (15) are not stated in the experimental settings; they are only implied by the ablation table. Please specify the default values.
  6. [Section IV.A, Cityscapes Description] The statement that 'only the 20th frame is annotated' for Cityscapes should be checked against the dataset documentation, since the annotated frame index is commonly described differently across versions of the dataset.
  7. [Section V] The future-work paragraph contains a broken citation placeholder '[57 ?]'; this should be corrected before publication.
  8. [Figure 5] Figure 5 appears only as a caption in the provided text; please ensure the actual figure is included in the final version.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the unseen-class mIoU is a measured empirical result under a stated protocol; the only self-citation is architectural and non-load-bearing.

full rationale

The central claim that OV2VSS improves zero-shot VSPW segmentation by about 4 mIoU over image-based methods is an empirical measurement, not a quantity fitted by construction. The reported 17.99% unseen-class mIoU comes from a defined protocol: the first 80 VSPW classes are used as seen classes, the last 44 as unseen classes, and unseen-class pixels are masked from the training inputs. All compared methods are trained and evaluated under that same protocol, so the reported gain is a direct empirical difference rather than an identity. The only self-citation with method-level weight is [34], used as inspiration for the multi-scale affinity aggregation in Eq. (5); that work is independently published and is not invoked as a uniqueness theorem, nor does it supply the benchmark numbers. The reliance on CLIP embeddings is an external input assumption, not a circular step. Concerns that the 80/44 order split is arbitrary, that input masking creates a train/test distribution shift, and that the single-run results lack error bars are evaluation-fairness and robustness risks, but none of them makes the derivation equivalent to its own inputs.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim depends on four hand-tuned hyperparameters and protocol choices (loss weights, temporal gaps, and the class split), none of which are derived from first principles. The method also assumes CLIP's embeddings transfer to video and that the arbitrary 80/44 split is a faithful open-vocabulary test. No new physical or conceptual entities are introduced.

free parameters (4)
  • loss weight alpha = 1 (ablated over 0.1, 1, 10)
    The main CE loss weight is a hyperparameter tuned on the validation set; final choice affects reported mIoU (14.88 to 17.99 across settings).
  • loss weight beta = 1 (ablated over 0.1, 1, 10)
    Auxiliary loss weight; tuning changes mIoU from 13.78 to 17.99.
  • temporal gaps = 3, 6, 9
    Distances between adjacent frames and target frame are hand-selected; no sensitivity analysis is provided.
  • visible class subset = first 80 of 124 VSPW classes
    The train/test class split is chosen by category order, not by semantic super-category; this drives the difficulty of the open-vocabulary evaluation.
assumptions (3)
  • domain assumption CLIP image and text embeddings provide a shared visual-semantic space that transfers from static images to video frames.
    The method relies on CLIP's zero-shot capability to recognize novel classes; stated in Section III.C and used throughout.
  • ad hoc to paper The first-80/last-44 VSPW class split is a fair open-vocabulary protocol.
    Section IV-C justifies this protocol as avoiding manual super-category selection, but it introduces arbitrary ordering and input masking.
  • standard math Standard attention and cost-volume operations are stable and differentiable in the proposed architecture.
    The method uses standard MHSA, cross-attention, and cosine-similarity cost volumes without new mathematical machinery.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Open-Vocabulary Video Semantic Segmentation." pith.science (2026). https://pith.science/paper/G4XRBRVM

@misc{pith2026241209329,
  author       = {Pith},
  title        = {Pith review of: Towards Open-Vocabulary Video Semantic Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G4XRBRVM}},
  note         = {Machine review of arXiv:2412.09329}
}
read the original abstract

Semantic segmentation in videos has been a focal point of recent research. However, existing models encounter challenges when faced with unfamiliar categories. To address this, we introduce the Open Vocabulary Video Semantic Segmentation (OV-VSS) task, designed to accurately segment every pixel across a wide range of open-vocabulary categories, including those that are novel or previously unexplored. To enhance OV-VSS performance, we propose a robust baseline, OV2VSS, which integrates a spatial-temporal fusion module, allowing the model to utilize temporal relationships across consecutive frames. Additionally, we incorporate a random frame enhancement module, broadening the model's understanding of semantic context throughout the entire video sequence. Our approach also includes video text encoding, which strengthens the model's capability to interpret textual information within the video context. Comprehensive evaluations on benchmark datasets such as VSPW and Cityscapes highlight OV-VSS's zero-shot generalization capabilities, especially in handling novel categories. The results validate OV2VSS's effectiveness, demonstrating improved performance in semantic segmentation tasks across diverse video datasets.

Figures

Figures reproduced from arXiv: 2412.09329 by the authors.

Figure 1
Figure 1. Comparison of VSS and OV-VSS: In traditional VSS, the model [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The comparison between our method and image-based methods on the VSPW dataset, from top to bottom, includes the original frame, image method, [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Overall structure of OV2VSS. Our approach primarily leverages three modules: the spatial-temporal information fusion module, which integrates spatial-temporal details from the video; the random frame enhancement module, which acquires contextual information from a randomly selected frame; the video text encoding, which utilizes text supervision in the training process. A denotes attention, + denotes element-wise add… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The architecture of the Spatio-Temporal Context Fusion module, which [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: The architecture of the Random Frame Enhancement module utilizes [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Qualitative results on the VSPW dataset. From top to bottom: Original frames, segmentation results from the image-based method (FreeSeg), [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Qualitative results on CityScapes datasets. We conduct inference on Cityscapes using the model trained on VSPW without further training. Although [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 54 canonical work pages

  1. [40]

    Decoupling zero-shot semantic segmentation,

    J. Ding, N. Xue, G.-S. Xia, and D. Dai, “Decoupling zero-shot semantic segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 11 583–11 592

    2022

  2. [1]

    Mining contextual information beyond image for semantic segmentation,

    Z. Jin, T. Gong, D. Yu, Q. Chu, J. Wang, C. Wang, and J. Shao, “Mining contextual information beyond image for semantic segmentation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 7231–7241

    work page 2021

  3. [2]

    ISNet: Integrate image-level and semantic-level context for semantic seg- mentation,

    Z. Jin, B. Liu, Q. Chu, and N. Yu, “ISNet: Integrate image-level and semantic-level context for semantic seg- mentation,” in Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision , 2021, pp. 7189– 7198

    work page 2021

  4. [3]

    Boundary-guided lightweight semantic segmen- tation with multi-scale semantic context,

    Q. Zhou, L. Wang, G. Gao, K. Bin, W. Ou, and H. Lu, “Boundary-guided lightweight semantic segmen- tation with multi-scale semantic context,” IEEE Transac- tions on Multimedia , 2024

    work page 2024

  5. [4]

    Fbsnet: A fast bilateral symmetrical network for real-time se- mantic segmentation,

    G. Gao, G. Xu, J. Li, Y . Yu, H. Lu, and J. Yang, “Fbsnet: A fast bilateral symmetrical network for real-time se- mantic segmentation,” IEEE Transactions on Multimedia, vol. 25, pp. 3273–3283, 2022

    work page 2022

  6. [5]

    Semantic segmen- tation guided pixel fusion for image retargeting,

    B. Yan, X. Niu, B. Bare, and W. Tan, “Semantic segmen- tation guided pixel fusion for image retargeting,” IEEE Transactions on Multimedia, vol. 22, no. 3, pp. 676–687, 2019

    work page 2019

  7. [6]

    Difference-aware distillation for semantic segmenta- tion,

    J. Gou, X. Zhou, L. Du, Y . Zhan, W. Chen, and Z. Yi, “Difference-aware distillation for semantic segmenta- tion,” IEEE Transactions on Multimedia , 2024

    work page 2024

  8. [7]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision,” in Proceedings of the Interna- tional Conference on Machine Learning , 2021, pp. 8748– 8763

    work page 2021

Show all 60 references
  1. [8]

    Side adapter network for open-vocabulary semantic segmen- tation,

    M. Xu, Z. Zhang, F. Wei, H. Hu, and X. Bai, “Side adapter network for open-vocabulary semantic segmen- tation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 2945–2954

    2023

  2. [9]

    Scaling open- vocabulary image segmentation with image-level labels,

    G. Ghiasi, X. Gu, Y . Cui, and T.-Y . Lin, “Scaling open- vocabulary image segmentation with image-level labels,” in Proceedings of the European Conference on Computer Vision, 2022, pp. 540–557

    2022

  3. [10]

    Towards open-vocabulary video instance segmentation,

    H. Wang, C. Yan, S. Wang, X. Jiang, X. Tang, Y . Hu, W. Xie, and E. Gavves, “Towards open-vocabulary video instance segmentation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 4057–4066

    2023

  4. [11]

    Openvis: Open-vocabulary video instance segmentation,

    P. Guo, T. Huang, P. He, X. Liu, T. Xiao, Z. Chen, and W. Zhang, “Openvis: Open-vocabulary video instance segmentation,” arXiv preprint arXiv:2305.16835 , 2023

    2023 arXiv

  5. [12]

    VSPW: A large-scale dataset for video scene parsing in the wild,

    J. Miao, Y . Wei, Y . Wu, C. Liang, G. Li, and Y . Yang, “VSPW: A large-scale dataset for video scene parsing in the wild,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 4133–4143

    2021

  6. [13]

    The Cityscapes dataset for semantic urban scene under- standing,

    M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. En- zweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele, “The Cityscapes dataset for semantic urban scene under- standing,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2016, pp. 3213–3223

    2016

  7. [14]

    In- door segmentation and support inference from RGBD images,

    N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, “In- door segmentation and support inference from RGBD images,” in Proceedings of the European Conference on Computer Vision, 2012, pp. 746–760

    2012

  8. [15]

    Segmentation and recognition using structure from mo- tion point clouds,

    G. J. Brostow, J. Shotton, J. Fauqueur, and R. Cipolla, “Segmentation and recognition using structure from mo- tion point clouds,” in Proceedings of the European Con- ference on Computer Vision , 2008, pp. 44–57

    2008

  9. [16]

    Low-latency video semantic segmentation,

    Y . Li, J. Shi, and D. Lin, “Low-latency video semantic segmentation,” in Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , 2018, pp. 5997–6005

    2018

  10. [17]

    Clockwork convnets for video semantic segmentation,

    E. Shelhamer, K. Rakelly, J. Hoffman, and T. Darrell, “Clockwork convnets for video semantic segmentation,” in Proceedings of the European Conference on Computer Vision, 2016, pp. 852–868

    2016

  11. [18]

    Budget-aware deep semantic video segmentation,

    B. Mahasseni, S. Todorovic, and A. Fern, “Budget-aware deep semantic video segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017, pp. 1029–1038

    2017

  12. [19]

    Deep feature flow for video recognition,

    X. Zhu, Y . Xiong, J. Dai, L. Yuan, and Y . Wei, “Deep feature flow for video recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017, pp. 2349–2358

    2017

  13. [20]

    Dynamic video segmentation network,

    Y .-S. Xu, T.-J. Fu, H.-K. Yang, and C.-Y . Lee, “Dynamic video segmentation network,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 6556–6565

    2018

  14. [21]

    Accel: A correc- tive fusion network for efficient semantic segmentation on video,

    S. Jain, X. Wang, and J. E. Gonzalez, “Accel: A correc- tive fusion network for efficient semantic segmentation on video,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 8866–8875

    2019

  15. [22]

    Temporally distributed networks for fast video semantic segmentation,

    P. Hu, F. Caba, O. Wang, Z. Lin, S. Sclaroff, and F. Per- azzi, “Temporally distributed networks for fast video semantic segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 8818–8827

    2020

  16. [23]

    Efficient se- mantic video segmentation with per-frame inference,

    Y . Liu, C. Shen, C. Yu, and J. Wang, “Efficient se- mantic video segmentation with per-frame inference,” in Proceedings of the European Conference on Computer Vision, 2020, pp. 352–368

    2020

  17. [24]

    Local memory attention for fast video semantic seg- mentation,

    M. Paul, M. Danelljan, L. Van Gool, and R. Timofte, “Local memory attention for fast video semantic seg- mentation,” in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems , 2021, pp. 1102–1109

    2021

  18. [25]

    Feature space optimization for semantic video segmentation,

    A. Kundu, V . Vineet, and V . Koltun, “Feature space optimization for semantic video segmentation,” in Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2016, pp. 3168–3175

    2016

  19. [26]

    Video semantic segmentation via sparse tem- poral transformer,

    J. Li, W. Wang, J. Chen, L. Niu, J. Si, C. Qian, and L. Zhang, “Video semantic segmentation via sparse tem- poral transformer,” in Proceedings of the ACM Interna- tional Conference on Multimedia , 2021, pp. 59–68. 11

    2021

  20. [27]

    Semantic video CNNs through representation warping,

    R. Gadde, V . Jampani, and P. V . Gehler, “Semantic video CNNs through representation warping,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2017, pp. 4453–4462

    2017

  21. [28]

    Video scene parsing with predictive feature learning,

    X. Jin, X. Li, H. Xiao, X. Shen, Z. Lin, J. Yang, Y . Chen, J. Dong, L. Liu, Z. Jie et al. , “Video scene parsing with predictive feature learning,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2017, pp. 5580–5588

    2017

  22. [29]

    Surveillance video parsing with single frame supervi- sion,

    S. Liu, C. Wang, R. Qian, H. Yu, R. Bao, and Y . Sun, “Surveillance video parsing with single frame supervi- sion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2017, pp. 413–421

    2017

  23. [30]

    Semantic video seg- mentation by gated recurrent flow propagation,

    D. Nilsson and C. Sminchisescu, “Semantic video seg- mentation by gated recurrent flow propagation,” in Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2018, pp. 6819–6828

    2018

  24. [31]

    Improving semantic segmen- tation via video propagation and label relaxation,

    Y . Zhu, K. Sapra, F. A. Reda, K. J. Shih, S. Newsam, A. Tao, and B. Catanzaro, “Improving semantic segmen- tation via video propagation and label relaxation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 8856–8865

    2019

  25. [32]

    AuxAdapt: Stable and efficient test-time adaptation for temporally consistent video semantic segmentation,

    Y . Zhang, S. Borse, H. Cai, and F. Porikli, “AuxAdapt: Stable and efficient test-time adaptation for temporally consistent video semantic segmentation,” in Proceedings of the IEEE Winter Conference on Applications of Com- puter Vision, 2022, pp. 2339–2348

    2022

  26. [33]

    Coarse-to-fine feature mining for video semantic seg- mentation,

    G. Sun, Y . Liu, H. Ding, T. Probst, and L. Van Gool, “Coarse-to-fine feature mining for video semantic seg- mentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 3126–3137

    2022

  27. [34]

    Mining relations among cross-frame affini- ties for video semantic segmentation,

    G. Sun, Y . Liu, H. Tang, A. Chhatkuli, L. Zhang, and L. Van Gool, “Mining relations among cross-frame affini- ties for video semantic segmentation,” in Proceedings of the European Conference on Computer Vision , 2022, pp. 522–539

    2022

  28. [35]

    Learning local and global temporal contexts for video semantic segmentation,

    G. Sun, Y . Liu, H. Ding, M. Wu, and L. Van Gool, “Learning local and global temporal contexts for video semantic segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2024

    2024

  29. [36]

    Video K-Net: A simple, strong, and unified baseline for video segmentation,

    X. Li, W. Zhang, J. Pang, K. Chen, G. Cheng, Y . Tong, and C. C. Loy, “Video K-Net: A simple, strong, and unified baseline for video segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 18 847–18 857

    2022

  30. [37]

    Tube-Link: A flexible cross tube baseline for universal video segmentation,

    X. Li, H. Yuan, W. Zhang, G. Cheng, J. Pang, and C. C. Loy, “Tube-Link: A flexible cross tube baseline for universal video segmentation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 13 923–13 933

    2023

  31. [38]

    Mask propagation for efficient video seman- tic segmentation,

    Y . Weng, M. Han, H. He, M. Li, L. Yao, X. Chang, and B. Zhuang, “Mask propagation for efficient video seman- tic segmentation,” in Proceedings of the Conference on Neural Information Processing Systems , 2023, pp. 7170– 7183

    2023

  32. [39]

    Scaling up visual and vision-language representation learning with noisy text supervision,

    C. Jia, Y . Yang, Y . Xia, Y .-T. Chen, Z. Parekh, H. Pham, Q. Le, Y .-H. Sung, Z. Li, and T. Duerig, “Scaling up visual and vision-language representation learning with noisy text supervision,” in Proceedings of the Interna- tional Conference on Machine Learning , 2021, pp. 4...

    2021

  33. [41]

    P2T: Pyramid pooling transformer for scene understanding,

    Y .-H. Wu, Y . Liu, X. Zhan, and M.-M. Cheng, “P2T: Pyramid pooling transformer for scene understanding,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 11, pp. 12 760–12 771, 2022

    2022

  34. [42]

    Object-contextual rep- resentations for semantic segmentation,

    Y . Yuan, X. Chen, and J. Wang, “Object-contextual rep- resentations for semantic segmentation,” in Proceedings of the European Conference on Computer Vision , 2020, pp. 173–190

    2020

  35. [43]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Proceedings of the Con- ference on Neural Information Processing Systems , 2017, pp. 5998–6008

    2017

  36. [44]

    CAT-Seg: Cost aggregation for open-vocabulary semantic segmentation,

    S. Cho, H. Shin, S. Hong, S. An, S. Lee, A. Arnab, P. H. Seo, and S. Kim, “CAT-Seg: Cost aggregation for open-vocabulary semantic segmentation,” arXiv preprint arXiv:2303.11797, 2023

    2023 arXiv

  37. [45]

    Open- vocabulary semantic segmentation with decoupled one- pass network,

    C. Han, Y . Zhong, D. Li, K. Han, and L. Ma, “Open- vocabulary semantic segmentation with decoupled one- pass network,” in Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision , 2023, pp. 1086– 1096

    2023

  38. [46]

    A simple baseline for open-vocabulary semantic segmentation with pre-trained vision-language model,

    M. Xu, Z. Zhang, F. Wei, Y . Lin, Y . Cao, H. Hu, and X. Bai, “A simple baseline for open-vocabulary semantic segmentation with pre-trained vision-language model,” in Proceedings of the European Conference on Computer Vision, 2022, pp. 736–753

    2022

  39. [47]

    FreeSeg: Unified, universal and open-vocabulary image segmentation,

    J. Qin, J. Wu, P. Yan, M. Li, R. Yuxi, X. Xiao, Y . Wang, R. Wang, S. Wen, X. Pan et al. , “FreeSeg: Unified, universal and open-vocabulary image segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 19 446–19 455

    2023

  40. [48]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, P. Doll ´ar, and R. Girshick, “Segment anything,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 4015–4026

    2023

  41. [49]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weis- senborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Min- derer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” in Proceedings of the International...

    2021

  42. [50]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778

    2016

  43. [51]

    Adam: A method for stochas- tic optimization,

    D. P. Kingma and J. L. Ba, “Adam: A method for stochas- tic optimization,” in Proceedings of the International 12 Conference on Learning Representations , 2014, pp. 1– 15

    2014

  44. [52]

    Deep-irtarget: An automatic target detector in infrared imagery using dual-domain feature extraction and allo- cation,

    R. Zhang, L. Xu, Z. Yu, Y . Shi, C. Mu, and M. Xu, “Deep-irtarget: An automatic target detector in infrared imagery using dual-domain feature extraction and allo- cation,” IEEE Transactions on Multimedia , vol. 24, pp. 1735–1749, 2021

    2021

  45. [53]

    Semantic scene completion from a single depth image,

    S. Song, F. Yu, A. Zeng, A. X. Chang, M. Savva, and T. Funkhouser, “Semantic scene completion from a single depth image,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017, pp. 1746–1754

    2017

  46. [54]

    Con- trastive boundary learning for point cloud segmentation,

    L. Tang, Y . Zhan, Z. Chen, B. Yu, and D. Tao, “Con- trastive boundary learning for point cloud segmentation,” in Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , 2022, pp. 8489– 8499

    2022

  47. [55]

    Learning from noisy labels with deep neural networks: A survey,

    H. Song, M. Kim, D. Park, Y . Shin, and J.-G. Lee, “Learning from noisy labels with deep neural networks: A survey,” IEEE Transactions on Neural Networks and Learning Systems, vol. 34, no. 11, pp. 8135–8153, 2022

    2022

  48. [56]

    Cognition-driven structural prior for instance- dependent label transition matrix estimation,

    R. Zhang, Z. Cao, S. Yang, L. Si, H. Sun, L. Xu, and F. Sun, “Cognition-driven structural prior for instance- dependent label transition matrix estimation,” IEEE Transactions on Neural Networks and Learning Systems , 2024

    2024

  49. [57]

    Feature modulation transformer: Cross-refinement of global representation via high-frequency prior for image super-resolution,

    A. Li, L. Zhang, Y . Liu, and C. Zhu, “Feature modulation transformer: Cross-refinement of global representation via high-frequency prior for image super-resolution,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 12 514–12 524

    2023

  50. [58]

    Panet: Few-shot image semantic segmentation with prototype alignment,

    K. Wang, J. H. Liew, Y . Zou, D. Zhou, and J. Feng, “Panet: Few-shot image semantic segmentation with prototype alignment,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2019, pp. 9197–9206

    2019

  51. [59]

    Part-aware correlation networks for few-shot learning,

    R. Zhang, J. Tan, Z. Cao, L. Xu, Y . Liu, L. Si, and F. Sun, “Part-aware correlation networks for few-shot learning,” IEEE Transactions on Multimedia , 2024. Xinhao Li received his B.E. from SouthWest Jiao- tong University (SWJTU), Chengdu, China, in 2022. He has been working ...

    2024

  52. [1998]

    He was with Nanyang Technological Univer- sity, Singapore, for 14 years from 1998 to 2012, where he was a Research Fellow, a Program Manager, an Assistant Pro- fessor, and then promoted to an Associate Professor in 2005. He has been with University of Electronic Science and Te...

    1998

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.