Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

FGAseg: Fine-Grained Pixel-Text Alignment for Open-Vocabulary Semantic Segmentation

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FGAseg turns CLIP's coarse image-level alignment into pixel-level alignment and reports higher mIoU than prior open-vocabulary segmentation methods on five benchmarks.

desk verdict A plausible OVS recombination whose SOTA claim is currently undermined by internal numerical contradictions; fix the tables and it could be a citable incremental contribution. read the letter →

arxiv 2501.00877 v2 pith:NFLX4AUZ submitted 2025-01-01 cs.CV

Bingyu Li , Da Zhang , Zhiyuan Zhao , Junyu Gao , Xuelong Li This is my paper · ORCID
classification cs.CV
keywords open-vocabularysemanticsegmentationvision-languagemodelsCLIPpixel-textalignmentpseudo-maskscategoryboundarycross-modalattentionfine-grained
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes FGAseg, a model for open-vocabulary semantic segmentation that turns CLIP's image-level vision-text alignment into fine-grained pixel-text alignment. The model adds a Pixel-Text Alignment Transformer, which lets text tokens attend to position-encoded image tokens, and a Text-Pixel Alignment Loss that supervises a local alignment map with ground-truth masks. It also treats cosine and convolution-based similarity matrices as pseudo-masks that carry category boundary information through the decoder. The authors report that FGAseg beats prior open-vocabulary segmentation methods on A-847, PC-459, A-150, PC-59, and PAS-20, reaching 16.3 mIoU on A-847 and 23.9 on PC-459 with a ViT-L/14 backbone while training only on COCO-Stuff.

What carries the argument

The central object is a pair of similarity maps. The global map $S_g$ is the cosine similarity between aligned text embeddings and image features; the local map $S_l$ comes from sliding text-derived convolution kernels over the vision feature map, then normalizing the kernels with softmax. Fused, these maps act as a pseudo-mask that feeds class aggregation (a two-layer cross-attention using text guidance as query) and spatial aggregation (a Swin Transformer module). The second mechanism is the Pixel-Text Alignment Transformer (P2Tformer): a cross-attention layer in which text tokens query flattened, position-encoded image tokens, with a trainable scale $\gamma$ blending the attended output into the original text embeddings. A Text-Pixel Alignment Loss computes mean squared error between an upsampled local alignment map and the ground-truth mask, steering the vision encoder toward pixel-level alignment without destroying the pretrained image-text alignment.

What would settle it

Measure the mIoU drop on A-847 when the pseudo-mask branch is disabled at inference while keeping the aligned text embeddings; if the drop is small, the pseudo-mask mechanism is not the source of the reported gains. Alternatively, compare the pseudo-masks against ground-truth boundaries on a held-out set and check whether per-class agreement predicts per-class mIoU, since low agreement would contradict the paper's explanation.

Watch

Extended reading notes

Core claim

FGAseg claims that the remaining gap between CLIP's pretraining and segmentation can be closed by a bidirectional alignment mechanism plus category-boundary supplementation, and that this is enough to outperform existing open-vocabulary segmentation models without external datasets or auxiliary backbones. The Pixel-Level Alignment module produces text embeddings that have looked at image content via cross-attention, and the Text-Pixel Alignment Loss forces local vision features to match ground-truth masks. The Category Supplementation Propagation module fuses a global cosine-similarity map with a local convolution-based similarity map into an optimizable pseudo-mask that guides class and spatial aggregation in the decoder. On five benchmarks, these components together yield higher mIoU than prior methods, with larger gains when scaling from ViT-B/16 to ViT-L/14.

Load-bearing premise

The model's performance depends on the cosine and convolution similarity maps from CLIP being reliable guides to category boundaries for unseen classes; if those maps are noisy, the aggregation steps will amplify the noise rather than refine the mask.

Editorial extensions

If this is right

  • Open-vocabulary segmentation can be improved on existing benchmarks without collecting new labeled data or adding auxiliary backbones, since the gains come from refining CLIP's own features.
  • The pseudo-mask propagation supplies boundary cues that benefit high-vocabulary benchmarks in particular: the method's largest relative gains appear on A-847 (16.3 mIoU) and PC-459 (23.9 mIoU).
  • The Top-K class selection at inference preserves most of the mIoU while cutting per-image time, so the accuracy gains do not have to come at the cost of real-time use.
  • The method scales with the vision backbone: switching from ViT-B/16 to ViT-L/14 raises mIoU on every reported dataset, suggesting the alignment components integrate cleanly with larger CLIP encoders.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the same similarity-map-as-pseudo-mask recipe could be lifted to other dense prediction tasks that need open-vocabulary grounding, such as referring-expression segmentation or open-vocabulary object detection, where boundary quality is also a bottleneck.
  • Beyond the paper: the paper does not measure pseudo-mask quality, so an immediate test is to compute the agreement of $S_g$ and $S_l$ with human-annotated boundaries on a held-out set and correlate that agreement with per-class mIoU.
  • Beyond the paper: the reliance on a fixed text prompt template ('a photo of a {class}') is likely a sensitivity point; evaluating the model with multiple prompt paraphrases could reveal how much of the reported gain depends on prompt wording.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes FGAseg, an open-vocabulary semantic segmentation approach that refines CLIP's image-level vision-text alignment through a Pixel-Level Alignment module, consisting of a Pixel-Text Alignment Transformer (P2Tformer) and a Text-Pixel Alignment Loss (T2Ploss). To preserve category boundaries, the method also introduces a Category Supplementation Propagation module that treats global cosine-similarity and local convolution-similarity maps as pseudo-masks during forward propagation. The model is trained end-to-end on COCO-Stuff and evaluated on A-847, PC-459, A-150, PC-59, and PAS-20, where the authors report consistent improvements over prior methods using ViT-B/16 and ViT-L/14 backbones, without external datasets or auxiliary backbones. The paper includes several ablation studies on the alignment module, kernel sizes, decoder variants, and training strategies.

Significance. If the reported numbers are correct, FGAseg provides a competitive and relatively simple recipe for open-vocabulary segmentation, achieving gains on multiple benchmarks while avoiding external datasets and auxiliary backbones. The architectural ideas - cross-modal pixel-text attention, a dedicated alignment loss, and using similarity maps as boundary-preserving pseudo-masks - are clear and reasonably motivated. The paper also includes extensive ablations and comparisons, which strengthens the empirical case. However, the overall credibility is undermined by several internal inconsistencies in the reported mIoU values and hyperparameter claims; these must be resolved before the stated state-of-the-art result can be assessed.

major comments (4)
  1. [§IV.C and Table I] The main comparison text and Table I report different numbers for the same ViT-L/14 configuration: the text states mIoU of 37.7, 63.1, and 96.9 for A-150, PC-59, and PAS-20, while Table I lists 37.9, 63.4, and 97.1 for those same benchmarks. Because the paper's central claim is that FGAseg consistently outperforms existing methods, the reader cannot determine which of these values is the actual result. Please correct the discrepancy and specify which numbers are used in the claims.
  2. [Table II vs. Table III(a)] The baseline configurations supposed to be the same are not consistent across the ablation tables. In Table II, the row with only the P2Tformer reports A-847 mIoU of 11.6 and the no-module baseline reports 11.1, while Table III(a) reports 8.5 for 'w/o P2Tformer' and 9.0 for 'w/ P2Tformer' on the same dataset and backbone. These differences of about 2.5-2.6 mIoU are too large to be ignored, and they directly affect the conclusion that the P2Tformer substantially improves alignment. Please reconcile the baselines and explain the cause of the discrepancy.
  3. [§IV.D.4 and Table IV(a)] The text states that a kernel size of 9 consistently achieves the highest mIoU, but Table IV(a) shows that kernel size 11 gives the best A-847 (12.1) and A-150 (32.0) scores and ties for the best PC-459 (19.4), while kernel 9 is not the maximum on any of these three sets. Moreover, §IV.D.1 sets the default kernel size to 3, yet the paper does not specify which kernel size is used in the final configuration for Table I. Please clarify the actual kernel size used in the main experiments and correct the description of the kernel-size dependency.
  4. [§III.B.2, Eq. (8)] The alignment loss Lalign is defined as an MSE between Oalign in R^{B x T x H x W} and M in R^{B x H x W}. As written, this subtraction is dimensionally incompatible unless M is broadcast or is actually a one-hot tensor with T channels. The text mentions 'a binary mask M' without specifying how it is expanded or whether the loss is computed per class. Please define the exact target representation and the reduction operation; otherwise the loss cannot be reproduced from the description.
minor comments (4)
  1. [Abstract, §I, and Fig. 7 caption] The abstract and Section I state 'The code is here' without providing a URL or repository reference, and the Figure 7 caption says 'None of the datasets were seen by the model during training' despite the model being trained on COCO-Stuff and the figure displaying COCO-derived images. Please provide a working code link and clarify the relationship between COCO-Stuff training and the displayed COCO images.
  2. [§IV.D.1] The text refers to 'the number of layers Nd in the P2Transformer', while elsewhere the module is called P2Tformer. Please use a single consistent notation throughout the paper.
  3. [Table VI] In Table VI, the time for Top-N=16 on A-847 is reported as 256 ms, which is lower than the times for Top-N=4 (327 ms) and Top-N=8 (399 ms); this non-monotonic entry appears to be a typo and should be verified and corrected.
  4. [§IV.D.2 and Table II] The text says that incorporating T2Ploss results in 'slight improvements', but Table II shows that adding T2Ploss increases A-847 from 11.6 to 12.1 while PC-459 decreases from 18.4 to 18.3. The narrative should reflect this mixed outcome rather than uniformly implying improvement.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: all reported gains are empirical benchmark numbers obtained with standard training losses and external test sets; the flagged internal inconsistencies are reproducibility concerns, not circular reasoning.

full rationale

FGAseg does not derive its central claim from the claim itself or from an author-supplied uniqueness theorem. The method is trained on COCO-Stuff with ground-truth masks via Lce, Lalign, and Lauxi (Eqs. 8, 11-13), and evaluated on external OVS benchmarks (A-847, PC-459, A-150, PC-59, PAS-20). The pseudo-masks Sg and Sl (Eqs. 9-10) are computed from the model's own vision/text features and used only as intermediate guidance in the decoder; they are not targets of the final evaluation and their usefulness is judged by held-out mIoU, so no prediction reduces to a fitted parameter. No parameter is fitted to the test benchmarks and then renamed a prediction. The self-citations [21], [22], [38] are background references and are not load-bearing. There is no ansatz smuggled in via citation and no known result renamed as new. The manuscript does contain internal numerical inconsistencies (Section IV.C text gives ViT-L/14 37.7/63.1/96.9 while Table I lists 37.9/63.4/97.1; the no-module baseline is 11.1 in Table II but 8.5 in Table III(a); Section IV.D.1 sets the default kernel to 3 while Section IV.D.4 claims 9 is optimal although Table IV(a) shows 11 higher on several datasets) and an unavailable code link, but these are correctness/reproducibility issues, not circularity. Hence score 0.

Assumptions & free parameters 8 free parameters · 4 assumptions · 0 invented entities

No new physical or conceptual entities are introduced; the modules are compositions of existing attention and convolution operations.

free parameters (8)
  • lambda_align = 0.02
    Weight for the text-pixel alignment loss; chosen by ablation on validation sets (Table III(d)).
  • lambda_auxi = 0.2
    Weight for the auxiliary loss; set in Section IV.B and not ablated.
  • number_of_p2tformer_layers_N = 1
    Default number of P2Tformer layers; ablation shows N=1 balances cost and performance.
  • decoder_layers_Nd = 3
    Decoder depth set in Section IV.B; not ablated.
  • kernel_size_K = 3 (text claims optimal is 9)
    Kernel size for convolution in LCS; default 3 in Section IV.D.1 but Table IV(a) and text suggest 9 is best, a contradiction.
  • gamma = trainable
    Learnable scaling for P2Tformer output; ablation shows trainable gamma is best.
  • vision_guidance_layers = ViT-L: 7,15; ViT-B: 3,7
    Choice of CLIP intermediate layers used as vision guidance.
  • top_k = 1 for inference
    Number of classes kept during inference; ablation shows Top-N=1 retains mIoU with less time.
assumptions (4)
  • domain assumption CLIP image and text features contain sufficient semantic information to be aligned at pixel level via fine-tuning
    The method builds on the premise that CLIP's representations are transferable to pixel-level tasks despite image-level pretraining.
  • domain assumption Ground-truth masks are available for training on COCO-Stuff and provide correct supervision for T2Ploss
    Lalign uses MSE between the convolved similarity map and the binary ground-truth mask (Eq. 8).
  • ad hoc to paper Cosine and convolution similarity maps serve as accurate pseudo-masks for category boundaries
    This is the central mechanism of the Category Supplementation Propagation module; no independent evidence is provided beyond the method's own performance.
  • domain assumption Evaluation on A-847, PC-459, A-150, PC-59, PAS-20 measures open-vocabulary generalization when trained only on COCO-Stuff
    Standard protocol, but the assumption that these benchmarks reflect true open-vocabulary ability is a modeling choice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FGAseg: Fine-Grained Pixel-Text Alignment for Open-Vocabulary Semantic Segmentation." pith.science (2026). https://pith.science/paper/NFLX4AUZ

@misc{pith2026250100877,
  author       = {Pith},
  title        = {Pith review of: FGAseg: Fine-Grained Pixel-Text Alignment for Open-Vocabulary Semantic Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NFLX4AUZ}},
  note         = {Machine review of arXiv:2501.00877}
}
read the original abstract

Open-vocabulary segmentation aims to identify and segment specific regions and objects based on text-based descriptions. A common solution is to leverage powerful vision-language models (VLMs), such as CLIP, to bridge the gap between vision and text information. However, VLMs are typically pretrained for image-level vision-text alignment, focusing on global semantic features. In contrast, segmentation tasks require fine-grained pixel-level alignment and detailed category boundary information, which VLMs alone cannot provide. As a result, information extracted directly from VLMs can't meet the requirements of segmentation tasks. To address this limitation, we propose FGAseg, a model designed for fine-grained pixel-text alignment and category boundary supplementation. The core of FGAseg is a Pixel-Level Alignment module that employs a cross-modal attention mechanism and a text-pixel alignment loss to refine the coarse-grained alignment from CLIP, achieving finer-grained pixel-text semantic alignment. Additionally, to enrich category boundary information, we introduce the alignment matrices as optimizable pseudo-masks during forward propagation and propose Category Information Supplementation module. These pseudo-masks, derived from cosine and convolutional similarity, provide essential global and local boundary information between different categories. By combining these two strategies, FGAseg effectively enhances pixel-level alignment and category boundary information, addressing key challenges in open-vocabulary segmentation. Extensive experiments demonstrate that FGAseg outperforms existing methods on open-vocabulary semantic segmentation benchmarks.

Figures

Figures reproduced from arXiv: 2501.00877 by the authors.

Figure 1
Figure 1. Comparison of Image-Level Pretraining and Pixel-Level Align￾ment. (a) Image-Level Pretraining aligns image and text embeddings via con￾trastive learning, while (b) Pixel-Level Alignment incorporates a pixel-level transformer, alignment loss and category information supplement to achieve finer-grained alignment, bridging the gap for open-vocabulary segmentation (OVS). also hinders their broader applicability due to c… view at source ↗
Figure 2
Figure 2. Overall architecture of FGAseg. (a) Pixel-Level Alignment Module: The P2Tformer aligns tokens in a pixel-text manner, while the T2Ploss enforces precise text-pixel alignment through local alignment. (b) Global Category Supplementation (GCS) and Local Category Supplementation (LCS) provide category boundary information as pseudo-masks for guidance. (c) Global and Local Category Supplementation Propagation incorporate… view at source ↗
Figure 3
Figure 3. Pixel-Level Alignment Module. This module refines pixel-text alignment by using multi-head attention and MLP layers across multiple P2Tformer layers. Vision and text tokens are processed to capture cross￾modal correlations, and the result is scaled by a learnable parameter γ, to balance the modalities. Aligned text tokens slide over vision tokens, creating alignment matrices for computing alignment loss (Lalign) via… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Illustration of (a) Global Category Supplementation(GCS) and (b) Local Category Supplementation (LCS). (a) Global category information is obtained by computing the cosine similarity,(b) Local category information is obtained by applying a text convolutional kernel to s…
Figure 5
Figure 5. Figure 5: Decoder details. (a) During training, the local similarity matrix is processed through concatenation, a linear layer, and a convolutional layer, with an auxiliary loss component. (b) In inference, TopK selection is applied to the local similarity matrix, followed by si…
Figure 6
Figure 6. Figure 6: Visualization of pseudo-masks. The first row represents global category supplementation (24×24 resolution), while the second row represents local category supplementation (96×96 resolution). We illustrate several successful pseudo label masks (in black) and cases that …
Figure 7
Figure 7. Figure 7: Segmentation case visualization. The data is sourced from the COCO dataset and the ADE20k dataset. None of the datasets were seen by the model during training, and the data labels are consistent with the respective datasets. (mIoU) across all datasets, indicating its e…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. EOVSAM: Efficient Open-Vocabulary Segmentation with SAM 3 in One Pass

    cs.CV 2026-08 conditional novelty 6.0 of 10

    By removing SAM 3's text prompts and pooling SigLIP features with decoder attention maps, EOVSAM performs open-vocabulary segmentation in one pass, beating vanilla SAM 3 in accuracy while running up to 338× faster.

Reference graph

Works this paper leans on

49 extracted references · 42 canonical work pages · cited by 1 Pith paper

  1. [1]

    Xian, Y ., Choudhury, S., He, Y ., Schiele, B., & Akata, Z. (2019). Semantic projection network for zero-and few-label semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 8256–8265)

    work page 2019

  2. [2]

    Bucher, M., Vu, T.-H., Cord, M., & P ´erez, P. (2019). Zero-shot semantic segmentation. Advances in Neural Information Processing Systems , 32

    work page 2019

  3. [3]

    Q., Belongie, S., Koltun, V ., & Ranftl, R

    Li, B., Weinberger, K. Q., Belongie, S., Koltun, V ., & Ranftl, R. (2022). Language-driven semantic segmentation. arXiv preprint arXiv:2201.03546

    arXiv 2022

  4. [4]

    H., Ding, H., Liu, J., Wang, Y ., Tang, Y ., Yang, Y ., Feng, J., Zhao, Y ., et al

    Han, K., Liu, Y ., Liew, J. H., Ding, H., Liu, J., Wang, Y ., Tang, Y ., Yang, Y ., Feng, J., Zhao, Y ., et al. (2023). Global knowledge calibration for fast open-vocabulary segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 797–807)

    work page 2023

  5. [5]

    Xu, J., De Mello, S., Liu, S., Byeon, W., Breuel, T., Kautz, J., & Wang, X. (2022). Groupvit: Semantic segmentation emerges from text supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 18134–18144)

    work page 2022

  6. [6]

    Ding, J., Xue, N., Xia, G.-S., & Dai, D. (2022). Decoupling zero-shot semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 11583–11592)

    work page 2022

  7. [7]

    H., & Kim, S

    Cho, S., Shin, H., Hong, S., Arnab, A., Seo, P. H., & Kim, S. (2024). Cat-seg: Cost aggregation for open-vocabulary semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 4113–4123)

    work page 2024

  8. [8]

    Xu, M., Zhang, Z., Wei, F., Hu, H., & Bai, X. (2023). Side adapter network for open-vocabulary semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 2945–2954)

    work page 2023

Show all 49 references
  1. [9]

    Xu, M., Zhang, Z., Wei, F., Lin, Y ., Cao, Y ., Hu, H., & Bai, X. (2022). A simple baseline for open-vocabulary semantic segmentation with pre- trained vision-language model. In European Conference on Computer Vision (pp. 736–753). Springer

    2022

  2. [10]

    Ghiasi, G., Gu, X., Cui, Y ., & Lin, T.-Y . (2022). Scaling open-vocabulary image segmentation with image-level labels. In European Conference on Computer Vision (pp. 540–557). Springer

    2022

  3. [11]

    Han, C., Zhong, Y ., Li, D., Han, K., & Ma, L. (2023). Open-vocabulary semantic segmentation with decoupled one-pass network. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 1086–1096)

    2023

  4. [12]

    Mukhoti, J., Lin, T.-Y ., Poursaeed, O., Wang, R., Shah, A., Torr, P. H. S., & Lim, S.-N. (2023). Open vocabulary semantic segmentation with patch aligned contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 19413–19423)

    2023

  5. [13]

    Liang, F., Wu, B., Dai, X., Li, K., Zhao, Y ., Zhang, H., Zhang, P., Vajda, P., & Marculescu, D. (2023). Open-vocabulary semantic segmentation with mask-adapted clip. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 7061–7070)

    2023

  6. [14]

    Sharma, P., Ding, N., Goodman, S., & Soricut, R. (2018). Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 2...

    2018

  7. [15]

    A., Friedland, G., Elizalde, B., Ni, K., Poland, D., Borth, D., & Li, L.-J

    Thomee, B., Shamma, D. A., Friedland, G., Elizalde, B., Ni, K., Poland, D., Borth, D., & Li, L.-J. (2016). Yfcc100m: The new data in multimedia research. Communications of the ACM , 59(2), 64–73. ACM New York, NY , USA

    2016

  8. [16]

    Kirillov, A., He, K., Girshick, R., Rother, C., & Doll ´ar, P. (2019). Panoptic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 9404–9413)

    2019

  9. [17]

    Pont-Tuset, J., Uijlings, J., Changpinyo, S., Soricut, R., & Ferrari, V . (2020). Connecting vision and language with localized narratives. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part V 16 (pp. 647–664). Springer. 12

    2020

  10. [18]

    S., & Pang, Y

    Xie, B., Cao, J., Xie, J., Khan, F. S., & Pang, Y . (2024). Sed: A simple encoder-decoder for open-vocabulary semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 3426–3436)

    2024

  11. [19]

    Ding, Z., Wang, J., & Tu, Z. (2022). Open-V ocabulary Panoptic Seg- mentation MaskCLIP. arXiv preprint arXiv:2208.01343

    2022 arXiv

  12. [20]

    Wang, X., Li, S., Kallidromitis, K., Kato, Y ., Kozuka, K., & Darrell, T. (2024). Hierarchical open-vocabulary universal image segmentation. Advances in Neural Information Processing Systems , 36

    2024

  13. [21]

    Li, B., Zhang, D., Zhao, Z., Gao, J., & Li, X. (2024). StitchFusion: Weaving Any Visual Modalities to Enhance Multimodal Semantic Seg- mentation. arXiv preprint arXiv:2408.01343

    2024 arXiv

  14. [22]

    Li, B., Zhang, D., Zhao, Z., Gao, J., & Li, X. (2024). U3M: Unbiased Multiscale Modal Fusion Model for Multimodal Semantic Segmentation. arXiv preprint arXiv:2405.15365

    2024 arXiv

  15. [23]

    Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2017). Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Transactions on Pattern Analysis and Machine Intelligence , 40(4), 834–848. IEEE

    2017

  16. [24]

    Zhang, J., Liu, H., Yang, K., Hu, X., Liu, R., & Stiefelhagen, R. (2023). CMX: Cross-modal fusion for RGB-X semantic segmentation with trans- formers. IEEE Transactions on Intelligent Transportation Systems

    2023

  17. [25]

    Zhang, D., Wang, F., Ning, L., Zhao, Z., Gao, J., & Li, X. (2024). Integrating SAM with Feature Interaction for Remote Sensing Change Detection. IEEE Transactions on Geoscience and Remote Sensing

    2024

  18. [26]

    Yu, Q., He, J., Deng, X., Shen, X., & Chen, L.-C. (2023). Convolutions die hard: Open-vocabulary segmentation with single frozen convolutional clip. Advances in Neural Information Processing Systems , 36, 32215– 32234

    2023

  19. [27]

    Xu, J., Liu, S., Vahdat, A., Byeon, W., Wang, X., & De Mello, S. (2023). Open-vocabulary panoptic segmentation with text-to-image diffu- sion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 2955–2966)

    2023

  20. [28]

    W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., & others

    Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., & others. (2021). Learning transferable visual models from natural language supervision. In International Conference on Machine Learning (pp. 8748–8763). PMLR

    2021

  21. [29]

    Jia, C., Yang, Y ., Xia, Y ., Chen, Y .-T., Parekh, Z., Pham, H., Le, Q., Sung, Y .-H., Li, Z., & Duerig, T. (2021). Scaling up visual and vision-language representation learning with noisy text supervision. In International Conference on Machine Learning (pp. 4904–4916). PMLR

    2021

  22. [30]

    Long, J., Shelhamer, E., & Darrell, T. (2015). Fully convolutional net- works for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3431–3440)

    2015

  23. [31]

    Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In Medical Image Com- puting and Computer-Assisted Intervention–MICCAI 2015: 18th Interna- tional Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part...

    2015

  24. [32]

    Badrinarayanan, V ., Kendall, A., & Cipolla, R. (2017). SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12), 2481– 2495

    2017

  25. [33]

    He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 770–778)

    2016

  26. [34]

    Dosovitskiy, A. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929

    2020 arXiv

  27. [35]

    M., & Luo, P

    Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J. M., & Luo, P. (2021). SegFormer: Simple and efficient design for semantic segmentation with transformers. Advances in Neural Information Processing Systems , 34, 12077–12090

    2021

  28. [36]

    Devlin, J. (2018). BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

    2018 arXiv

  29. [37]

    Tan, M., & Le, Q. (2019). EfficientNet: Rethinking model scaling for convolutional neural networks. In Proceedings of the International Conference on Machine Learning (pp. 6105–6114). PMLR

    2019

  30. [38]

    Z. Hu, J. Gao, Y . Yuan, and X. Li, ”Contrastive Tokens and Label Acti- vation for Remote Sensing Weakly Supervised Semantic Segmentation,” IEEE Transactions on Geoscience and Remote Sensing , 2024

    2024

  31. [39]

    Lin, B., Xie, J., Li, C., & Qu, Y . (2018). Deeptongue: Tongue segmen- tation via ResNet. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 1035–1039). IEEE

    2018

  32. [40]

    G., Kirillov, A., & Girdhar, R

    Cheng, B., Misra, I., Schwing, A. G., Kirillov, A., & Girdhar, R. (2022). Masked-attention mask transformer for universal image segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 1290–1299). IEEE

    2022

  33. [41]

    Cheng, B., Schwing, A., & Kirillov, A. (2021). Per-pixel classification is not all you need for semantic segmentation. Advances in Neural Information Processing Systems , 34, 17864–17875

    2021

  34. [42]

    C., & Dai, B

    Zhou, C., Loy, C. C., & Dai, B. (2022). Extract free dense labels from CLIP. In European Conference on Computer Vision (pp. 696–712). Springer

    2022

  35. [43]

    Jiao, S., Zhu, H., Huang, J., Zhao, Y ., Wei, Y ., & Shi, H. (2025). Collaborative vision-text representation optimizing for open-vocabulary segmentation. In Proceedings of the European Conference on Computer Vision (pp. 399–416)

    2025

  36. [44]

    M., Xie, J., Nie, J., Yang, A., & Pang, Y

    Xie, B., Cao, J., Anwer, R. M., Xie, J., Nie, J., Yang, A., & Pang, Y . (2024). Multi-query and multi-level enhanced network for semantic segmentation. Pattern Recognition, 156, 110777. Elsevier

    2024

  37. [45]

    Zhu, C., Li, L., Wu, Y ., & Sun, Z. (2024). Saswot: Real-time semantic segmentation architecture search without training. In Proceedings of the AAAI Conference on Artificial Intelligence (V ol. 38, No. 7, pp. 7722– 7730)

    2024

  38. [46]

    J., Han, D., & Hwang, W

    Na, J., Ha, J.-W., Chang, H. J., Han, D., & Hwang, W. (2024). Switching temporary teachers for semi-supervised semantic segmentation. Advances in Neural Information Processing Systems , 36

    2024

  39. [47]

    Zhao, T., Chen, Y ., Wu, Y ., Liu, T., Du, B., Xiao, P., Qiu, S., Yang, H., Li, G., Yang, Y ., & others. (2024). Improving Bird’s Eye View Semantic Segmentation by Task Decomposition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 155...

    2024

  40. [48]

    Xu, W., Wang, C., Feng, X., Xu, R., Huang, L., Zhang, Z., Guo, L., & Xu, S. (2024). Generalization boosted adapter for open-vocabulary segmentation. IEEE Transactions on Circuits and Systems for Video Technology. IEEE

    2024

  41. [49]

    Shan, X., Wu, D., Zhu, G., Shao, Y ., Sang, N., & Gao, C. (2024). Open- vocabulary semantic segmentation with image embedding balancing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 28412–28421). Bingyu Li is pursuing the Ph.D. degr...

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.