Pith. sign in

REVIEW 4 major objections 4 minor 44 references

Revisiting Audio-Visual Segmentation with Vision-Centric Transformer

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Vision-Centric Transformer sets new best results in audio-visual segmentation.

desk verdict Vision-centric queries with prototype prompting give a real architectural idea and a large AVSS gain, but the universal SOTA claim rests on an unablated Semantic-SAM prior and one thin 0.4 MJ margin on MS3. read the letter →

arxiv 2506.23623 v1 pith:TZW6GB6Q submitted 2025-06-30 cs.CV

Shaofei Huang , Rui Ling , Tianrui Hui , Hongyu Li , Xu Zhou , Shifeng Zhang , Si Liu , Richang Hong
show 1 more author
Meng Wang
This is my paper
classification cs.CV
keywords audio-visualsegmentationvision-centrictransformerobjectqueriesaudioprototypepromptingpixelcontextgroupingmixeddisambiguationdensepredictionAVSBench
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that audio-visual segmentation should start from object queries derived from visual regions rather than from the audio mixture. Previous transformer methods derive their queries from audio, and the paper contends that this makes them confused by mixed sounds and prone to losing visual detail needed for precise masks. It proposes a Vision-Centric Transformer (VCT) whose queries are born from grouped pixel regions, primed with audio event prototypes, and refined by alternately attending to audio and fine-grained visual features. This design is reported to reach the best mean IoU and F-score on all three AVSBench subsets, including 51.2 mean IoU on the hardest semantic subset with a Swin-B backbone at 384x384 resolution. If true, the result would mean the source of the query—vision rather than audio—is what limits current audio-visual segmentation systems.

What carries the argument

The load-bearing object is the Vision-Centric Transformer (VCT), whose query is a vector initialized from a spatially grouped visual region rather than from embedded audio. Its companion module, Prototype Prompted Query Generation (PPQG), builds these queries in three steps: a convolution and MLP aggregation compresses the high-resolution feature into visual embeddings; a cross-attention layer lets those embeddings read from K learnable audio prototypes, each pushed by a prototype-audio contrastive loss to encode one audio event category; and a Gumbel-Softmax hard assignment groups pixel context into the queries so different queries lock onto different regions. The decoder then alternates an audio information fetching block, where each query cross-attends with the full audio feature as key and value, with three visual information enhancement blocks that attend to progressively finer visual features under the previous layer's predicted mask. This combination carries the argument because the vision-derived queries provide the spatial anchor and the audio prototypes provide the category prior that together are supposed to factorize the mixed audio.

What would settle it

Build a synthetic or annotated set where two visible sources produce overlapping sounds and an off-screen source is mixed into the same audio frame; compare VCT masks under the full mixture against masks obtained with each source isolated. A large drop in per-source mean IoU, or a shift in which object is segmented when the off-screen track is removed, would falsify the claim that vision-derived queries factorize the audio mixture.

Watch

Extended reading notes

Core claim

The central claim is that replacing audio-centric object queries with vision-derived queries that progressively fetch corresponding audio information resolves the two failure modes of prior audio-visual segmentation transformers. Concretely, the VCT computes queries from the highest-resolution visual feature, groups pixel contexts with a hard Gumbel-Softmax assignment so different queries attend to different image regions, and prompts those queries with learnable audio-category prototypes trained by a prototype-audio contrastive loss. The queries then pass through an iterative decoder in which each query first cross-attends to the full audio feature of the frame, then attends to three scales of visual features with mask-guided attention. The paper claims this lets each query pick out exactly the sound components belonging to the region it represents, separating multiple on-screen sources and rejecting off-screen noise, while preserving enough visual detail for accurate contours. On AVSBench, the method is reported to outperform all previous approaches on S4, MS3, and AVSS across ResNet-50, PVT-v2, and Swin-B backbones, including the largest gains on the hardest semantic subset.

Load-bearing premise

The method assumes that a query tied to one visual region can pull just that region's sound out of a mixed audio clip using attention over the whole audio, even when several on- and off-screen sources overlap; if the audio mixture cannot be separated that way, the design's central advantage disappears.

Editorial extensions

If this is right

  • The query-generation source matters as much as the fusion architecture: switching from audio-derived to vision-derived queries yields a reported gain from 33.2 to 35.2 mean IoU on the AVSS subset with the same backbone before any audio prompting is added.
  • The full PPQG recipe—visual embedding aggregation, hard pixel-context grouping, and audio-prototype prompting with the contrastive loss—adds progressively: 35.2, 35.8, 36.3, and finally 37.5 mean IoU on AVSS.
  • With a PVT-v2 backbone at 224x224, VCT reaches 44.7 mean IoU on AVSS, surpassing a previously reported Swin-B 384x384 method at 44.4, implying the architecture itself, not backbone strength or resolution, drives the improvement.
  • On the saturated S4 subset, the method still lifts mean IoU to 86.2 with Swin-B at 384x384, indicating gains come from hard cases rather than from inflating easy scores.
  • The logit-map visualization shows vision-derived queries distribute their focus across different image regions, whereas audio-derived queries cluster on a few salient objects; this spatial diversity is the mechanism behind the reported multi-source improvements.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An immediate extension would test VCT on a purposely hard mixture benchmark where two visible objects emit spectrally overlapping sounds and an off-screen source is present; the cross-attention factorization predicts per-object masks should remain separable, a claim the current AVSBench subsets may not fully stress.
  • The vision-centric query principle could transfer to other multimodal dense tasks that mix queries from a non-spatial modality, such as language-referenced video object segmentation, by anchoring queries in pixels and fetching the referring signal later.
  • The prototype-audio contrastive loss effectively turns audio prototypes into a learnable event vocabulary; one could repurpose it for open-vocabulary or weakly supervised audio event discovery, where prototypes are initialized from class names and matched to audio.
  • If the factorization property holds, the iterative audio-fetching block could be reused as a generic audio source separation front-end that is conditioned on visual regions rather than on global audio statistics.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper proposes VCT, a Transformer architecture for audio-visual segmentation in which object queries are generated from visual features by a Prototype Prompted Query Generation (PPQG) module and then refined through iterative audio-fetching and visual-enhancement decoder blocks. The authors argue that audio-centric query designs suffer from mixed-audio ambiguity and visual detail loss, and they report state-of-the-art MJ/MF scores on the S4, MS3, and AVSS subsets of AVSBench, including 51.2 MJ on AVSS with Swin-B at 384x384 resolution. The code is released.

Significance. The proposed design is well motivated, and the ablations in Table 2 show monotonic gains from the PPQG components on AVSS, which is a useful signal in favor of the vision-centric query formulation. The paper is clearly written, the code is public, and the reported AVSS improvements are substantial. However, the central SOTA claim is not yet fully supported: the main table reports results for VCT only with Swin-B despite text claiming multiple backbones, the Semantic-SAM prior is not ablated, and the MS3 margin over the previous best is 0.4 MJ with no variance information. These issues require additional experiments before the empirical claims can be assessed.

major comments (4)
  1. [Section 4.2 / Table 1] Section 4.2 says VCT is trained with ResNet-50, PVT-v2, and Swin-B backbones, and Section 4.3 claims "our method achieves even better MJ performance with a PVT-v2 backbone and a 224 x 224 image size compared to AVSBias using Swin-B". However, Table 1 lists VCT results only for Swin-B at 224 and 384; no VCT rows for ResNet-50 or PVT-v2 appear. The cross-backbone claim and the PVT-v2 sentence are therefore unsupported by the reported table. Please add the missing results or revise the claims.
  2. [Section 4.2 / Tables 2-4] Section 4.2 states that the Semantic-SAM segmentation prior is fused into the visual feature "in the same way as COMBO". Since COMBO is the main previous method and also uses this prior, the relative gains in Table 1 could be attributable to the prior rather than to the vision-centric query design. The ablations in Tables 2-4 vary the PPQG components but never remove the Semantic-SAM prior and never compare VCT with and without it. Please add a VCT variant without Semantic-SAM and, if possible, an ACT baseline with and without the prior, to isolate the contribution of the proposed architecture.
  3. [Table 1, MS3 rows] Table 1, MS3 rows: with Swin-B/384, VCT reaches 67.6 MJ / 81.4 MF against AVSBias's 67.2 MJ / 80.8 MF, a 0.4 MJ margin, and every number in the paper is from a single run. Mask2Former-style training is known to vary by more than this amount across seeds, so the claim of beating the previous best on MS3 is not established. Please report mean and standard deviation over at least three seeds (or bootstrap confidence intervals) and state whether the MS3 difference is significant.
  4. [Section 3.3] The audio information fetching block uses the full mixed audio feature At as key and value for every query. The paper's motivation is that vision queries can "independently integrate their corresponding sound information from mixed audio", but this assumes the cross-attention can factorize a mixture of overlapping sources; no experiment demonstrates this. Figure 5 shows query logit diversity, not per-source audio separation. Please provide a diagnostic (e.g., attention-weight inspection on MS3 mixtures or a synthetic mixture experiment) to support the claimed mechanism, or soften the corresponding statement.
minor comments (4)
  1. [Equations (6)-(7)] The softmax and arg max in Equations (6)-(7) are not specified over which dimension of R; state explicitly that the softmax is over the N dimension (per pixel) and the arg max is also over N.
  2. [Section 4.2] "Maskige" appears to be a typo; it should likely be "mask image" or a similar term.
  3. [Section 3.1] The symbol T is used both for the number of video frames and for the time dimension of the audio feature; consider using a distinct symbol (e.g., T_a) for the audio time dimension to avoid ambiguity.
  4. [Table 1] The caption says best and second-best results are bold and underlined for different backbones and image sizes, but the VCT rows are only shown under Swin-B; please clarify the table formatting and ensure the claimed backbones are visible.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported AVSBench scores are external benchmark numbers, and the learned audio prototypes are supervised by ground-truth audio categories rather than by the metrics being predicted.

full rationale

The paper's central claim is empirical: VCT outperforms previous methods on the S4, MS3, and AVSS subsets of AVSBench. The derivation chain does not reduce to its inputs. Vision-derived queries are produced from visual features with audio prototype prompting, and the audio prototypes are trained with a separate multi-label BCE loss (Lpac, Eqs. 4-5) whose supervision comes from dataset annotations of which audio categories are present. The segmentation and classification heads are trained with the standard mask, Dice, and cross-entropy losses against public ground-truth masks and labels. The reported metrics (MJ and MF) are computed on a held-out public benchmark and compared with externally published baselines; no metric value is used as a fitting target or inserted into the model. The only self-citations are to related query-based audio-visual segmentation work and appear in the related-work discussion; they are not load-bearing for the VCT design or for the evaluation. The adoption of Semantic-SAM as segmentation prior, following COMBO, is an external component, and the absence of an ablation that removes it is a legitimate empirical confounding concern about isolating the architecture's contribution, but it is not circularity. No equation or training objective defines VCT's predictions in terms of the claimed results, and no predicted quantity is used to set the reported numbers. The strongest finding is therefore a clean non-finding: the paper's central performance claim rests on external evaluation rather than on a self-referential construction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new physical or causal entities are introduced. The audio prototypes P are learned parameters, not independent entities, and the method uses only standard pretrained backbones and priors.

free parameters (4)
  • Number of vision-derived queries N = 100
    Chosen by hand; determines grouping capacity. No sensitivity study is reported.
  • Decoder repetition count D = 2
    Chosen by hand; no analysis of varying depth is reported.
  • Loss coefficients (lambda_cls, lambda_mask, lambda_pac) = 2, 5, 1
    Set following Mask2Former and COMBO conventions; no tuning analysis is reported.
  • Hidden dimension C_h = 256
    Chosen by hand; affects the capacity of visual embeddings and prototypes.
assumptions (4)
  • standard math Gumbel-Softmax with the straight-through estimator provides a valid differentiable surrogate for hard pixel-to-query assignment.
    Invoked in Eq. (6)-(7) and required for end-to-end training of the grouping step.
  • domain assumption VGGish audio features and ImageNet-pretrained visual features provide representations from which separable audio-visual correspondences can be learned.
    The method relies on these frozen pretrained encoders as described in Section 3.1.
  • domain assumption Ground-truth audio event labels M* in the AVSS subset are accurate and sufficient to supervise prototype learning via L_pac.
    Used in Eq. (4)-(5); if labels are noisy, the prototypes would encode wrong category priors.
  • domain assumption Semantic-SAM provides segmentation priors that transfer to AVS objects and do not leak test-set information.
    Semantic-SAM is trained on external data and used to augment visual features in Section 4.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Revisiting Audio-Visual Segmentation with Vision-Centric Transformer." pith.science (2026). https://pith.science/paper/TZW6GB6Q

@misc{pith2026250623623,
  author       = {Pith},
  title        = {Pith review of: Revisiting Audio-Visual Segmentation with Vision-Centric Transformer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TZW6GB6Q}},
  note         = {Machine review of arXiv:2506.23623}
}
read the original abstract

Audio-Visual Segmentation (AVS) aims to segment sound-producing objects in video frames based on the associated audio signal. Prevailing AVS methods typically adopt an audio-centric Transformer architecture, where object queries are derived from audio features. However, audio-centric Transformers suffer from two limitations: perception ambiguity caused by the mixed nature of audio, and weakened dense prediction ability due to visual detail loss. To address these limitations, we propose a new Vision-Centric Transformer (VCT) framework that leverages vision-derived queries to iteratively fetch corresponding audio and visual information, enabling queries to better distinguish between different sounding objects from mixed audio and accurately delineate their contours. Additionally, we also introduce a Prototype Prompted Query Generation (PPQG) module within our VCT framework to generate vision-derived queries that are both semantically aware and visually rich through audio prototype prompting and pixel context grouping, facilitating audio-visual information aggregation. Extensive experiments demonstrate that our VCT framework achieves new state-of-the-art performances on three subsets of the AVSBench dataset. The code is available at https://github.com/spyflying/VCT_AVS.

Figures

Figures reproduced from arXiv: 2506.23623 by the authors.

Figure 1
Figure 1. Motivation of our method. (a) Previous methods typ [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overall architecture of our proposed VCT. The audio feature and multi-scale visual features are first extracted by the encoders. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Details of Prototype Prompted Query Generation. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison between our full model and the ACT baseline. Existing sounds correspond to masks of the same colors. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Visualization of logit maps from different types of [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

44 extracted references · 36 canonical work pages

  1. [1]

    Youtube-8m: A large- scale video classification benchmark

    Sami Abu-El-Haija, Nisarg Kothari, Joonseok Lee, Paul Natsev, George Toderici, Balakrishnan Varadarajan, and Sudheendra Vijayanarasimhan. Youtube-8m: A large- scale video classification benchmark. arXiv preprint arXiv:1609.08675, 2016. 5

    arXiv 2016

  2. [2]

    Look, listen and learn

    Relja Arandjelovic and Andrew Zisserman. Look, listen and learn. In ICCV, 2017. 2

    work page 2017

  3. [3]

    Objects that sound

    Relja Arandjelovic and Andrew Zisserman. Objects that sound. In ECCV, 2018. 2

    work page 2018

  4. [4]

    End-to- end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to- end object detection with transformers. In ECCV, 2020. 1, 3

    work page 2020

  5. [5]

    Unraveling in- stance associations: A closer look for audio-visual segmen- tation

    Yuanhong Chen, Yuyuan Liu, Hu Wang, Fengbei Liu, Chong Wang, Helen Frazer, and Gustavo Carneiro. Unraveling in- stance associations: A closer look for audio-visual segmen- tation. In CVPR, 2024. 6

    work page 2024

  6. [6]

    Cpm: Class-conditional prompting ma- chine for audio-visual segmentation

    Yuanhong Chen, Chong Wang, Yuyuan Liu, Hu Wang, and Gustavo Carneiro. Cpm: Class-conditional prompting ma- chine for audio-visual segmentation. In ECCV, 2024. 3, 6

    work page 2024

  7. [7]

    Masked-attention mask transformer for universal image segmentation

    Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexan- der Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In CVPR,

  8. [8]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009. 5

    work page 2009

Show all 44 references
  1. [9]

    Avsegformer: Audio-visual segmentation with trans- former

    Shengyi Gao, Zhe Chen, Guo Chen, Wenhai Wang, and Tong Lu. Avsegformer: Audio-visual segmentation with trans- former. In AAAI, 2024. 1, 3, 6

    2024

  2. [10]

    Improving audio-visual segmentation with bidirectional generation

    Dawei Hao, Yuxin Mao, Bowen He, Xiaodong Han, Yuchao Dai, and Yiran Zhong. Improving audio-visual segmentation with bidirectional generation. In AAAI, 2024. 3, 6

    2024

  3. [11]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR,

  4. [12]

    Cnn ar- chitectures for large-scale audio classification

    Shawn Hershey, Sourish Chaudhuri, Daniel PW Ellis, Jort F Gemmeke, Aren Jansen, R Channing Moore, Manoj Plakal, Devin Platt, Rif A Saurous, Bryan Seybold, et al. Cnn ar- chitectures for large-scale audio classification. In ICASSP,

  5. [13]

    Discriminative sounding objects localization via self-supervised audiovisual match- ing

    Di Hu, Rui Qian, Minyue Jiang, Xiao Tan, Shilei Wen, Errui Ding, Weiyao Lin, and Dejing Dou. Discriminative sounding objects localization via self-supervised audiovisual match- ing. NeurIPS, 2020. 1, 2

    2020

  6. [14]

    Mix and local- ize: Localizing sound sources in mixtures

    Xixi Hu, Ziyang Chen, and Andrew Owens. Mix and local- ize: Localizing sound sources in mixtures. In CVPR, 2022. 1, 2

    2022

  7. [15]

    Discovering sound- ing objects by audio queries for audio visual segmentation

    Shaofei Huang, Han Li, Yuqing Wang, Hongji Zhu, Jiao Dai, Jizhong Han, Wenge Rong, and Si Liu. Discovering sound- ing objects by audio queries for audio visual segmentation. In IJCAI, 2023. 3, 6

    2023

  8. [16]

    Unleashing the temporal-spatial reasoning capacity of gpt for training-free audio and language referenced video object segmentation

    Shaofei Huang, Rui Ling, Hongyu Li, Tianrui Hui, Zongheng Tang, Xiaoming Wei, Jizhong Han, and Si Liu. Unleashing the temporal-spatial reasoning capacity of gpt for training-free audio and language referenced video object segmentation. In AAAI, 2025. 3

    2025

  9. [17]

    Categorical reparameterization with gumbel-softmax

    Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144, 2016. 4

    2016 arXiv

  10. [18]

    Learning to visually localize sound sources from mixtures without prior source knowledge

    Dongjin Kim, Sung Jin Um, Sangmin Lee, and Jung Uk Kim. Learning to visually localize sound sources from mixtures without prior source knowledge. In CVPR, 2024. 2

    2024

  11. [19]

    Segment and recognize anything at any granularity

    Feng Li, Hao Zhang, Peize Sun, Xueyan Zou, Shilong Liu, Chunyuan Li, Jianwei Yang, Lei Zhang, and Jianfeng Gao. Segment and recognize anything at any granularity. In ECCV, 2024. 5

    2024

  12. [20]

    Selm: Selective mechanism based audio-visual segmentation

    Jiaxu Li, Songsong Yu, Yifan Wang, Lijun Wang, and Huchuan Lu. Selm: Selective mechanism based audio-visual segmentation. In ACM MM, 2024. 3, 6

    2024

  13. [21]

    Catr: Combinatorial-dependence audio-queried transformer for audio-visual video segmentation

    Kexin Li, Zongxin Yang, Lei Chen, Yi Yang, and Jun Xiao. Catr: Combinatorial-dependence audio-queried transformer for audio-visual video segmentation. In ACM MM, 2023. 1, 6

    2023

  14. [22]

    Qdformer: Towards robust audiovisual segmentation in complex environments with quantization-based semantic decomposition

    Xiang Li, Jinglu Wang, Xiaohao Xu, Xiulian Peng, Rita Singh, Yan Lu, and Bhiksha Raj. Qdformer: Towards robust audiovisual segmentation in complex environments with quantization-based semantic decomposition. In CVPR, 2024

    2024

  15. [23]

    Audio-visual seg- mentation by exploring cross-modal mutual semantics

    Chen Liu, Peike Patrick Li, Xingqun Qi, Hu Zhang, Lincheng Li, Dadong Wang, and Xin Yu. Audio-visual seg- mentation by exploring cross-modal mutual semantics. In ACM MM, 2023. 6

    2023

  16. [24]

    Bavs: bootstrapping audio- visual segmentation by integrating foundation knowledge

    Chen Liu, Peike Li, Hu Zhang, Lincheng Li, Zi Huang, Dadong Wang, and Xin Yu. Bavs: bootstrapping audio- visual segmentation by integrating foundation knowledge. TMM, 2024. 3, 6

    2024

  17. [25]

    Audio-visual segmentation via unlabeled frame exploitation

    Jinxiang Liu, Yikun Liu, Fei Zhang, Chen Ju, Ya Zhang, and Yanfeng Wang. Audio-visual segmentation via unlabeled frame exploitation. In CVPR, 2024. 3, 6

    2024

  18. [26]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, 2021. 3, 5

    2021

  19. [27]

    Step- ping stones: A progressive training strategy for audio-visual semantic segmentation

    Juncheng Ma, Peiwen Sun, Yaoting Wang, and Di Hu. Step- ping stones: A progressive training strategy for audio-visual semantic segmentation. In ECCV, 2024. 5

    2024

  20. [28]

    The concrete distribution: A continuous relaxation of discrete random variables

    Chris J Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous relaxation of discrete random variables. arXiv preprint arXiv:1611.00712 , 2016. 4

    2016 arXiv

  21. [29]

    Multimodal variational auto-encoder based audio-visual segmentation

    Yuxin Mao, Jing Zhang, Mochu Xiang, Yiran Zhong, and Yuchao Dai. Multimodal variational auto-encoder based audio-visual segmentation. In ICCV, 2023. 3, 6

    2023

  22. [30]

    V-net: Fully convolutional neural networks for volumetric medical image segmentation

    Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In 3DV, 2016. 5

    2016

  23. [31]

    Localizing visual sounds the easy way

    Shentong Mo and Pedro Morgado. Localizing visual sounds the easy way. In ECCV, 2022. 2

    2022

  24. [32]

    Imagenet-21k pretraining for the masses

    Tal Ridnik, Emanuel Ben-Baruch, Asaf Noy, and Lihi Zelnik-Manor. Imagenet-21k pretraining for the masses. arXiv preprint arXiv:2104.10972, 2021. 5

    2021 arXiv

  25. [33]

    Learning to localize sound source in visual scenes

    Arda Senocak, Tae-Hyun Oh, Junsik Kim, Ming-Hsuan Yang, and In So Kweon. Learning to localize sound source in visual scenes. In CVPR, 2018. 1, 2

    2018

  26. [34]

    Unveiling and mitigating bias in audio visual segmentation

    Peiwen Sun, Honggang Zhang, and Di Hu. Unveiling and mitigating bias in audio visual segmentation. In ACM MM,

  27. [35]

    Attention is all you need

    A Vaswani. Attention is all you need. NeurIPS, 2017. 1, 3

    2017

  28. [36]

    Pvt v2: Improved baselines with pyramid vision transformer

    Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. Pvt v2: Improved baselines with pyramid vision transformer. CVM, 2022. 5

    2022

  29. [37]

    Prompting segmentation with sound is gen- eralizable audio-visual source localizer

    Yaoting Wang, Weisong Liu, Guangyao Li, Jian Ding, Di Hu, and Xi Li. Prompting segmentation with sound is gen- eralizable audio-visual source localizer. In AAAI, 2024. 1, 3, 6

    2024

  30. [38]

    Can textual semantics mitigate sounding object segmentation preference? In ECCV, 2024

    Yaoting Wang, Peiwen Sun, Yuanchao Li, Honggang Zhang, and Di Hu. Can textual semantics mitigate sounding object segmentation preference? In ECCV, 2024. 3, 5, 6

    2024

  31. [39]

    Language as queries for referring video object segmen- tation

    Jiannan Wu, Yi Jiang, Peize Sun, Zehuan Yuan, and Ping Luo. Language as queries for referring video object segmen- tation. In CVPR, 2022. 3

    2022

  32. [40]

    Groupvit: Semantic segmentation emerges from text supervision

    Jiarui Xu, Shalini De Mello, Sifei Liu, Wonmin Byeon, Thomas Breuel, Jan Kautz, and Xiaolong Wang. Groupvit: Semantic segmentation emerges from text supervision. In CVPR, 2022. 4

    2022

  33. [41]

    Cooperation does matter: Exploring multi-order bilateral relations for audio- visual segmentation

    Qi Yang, Xing Nie, Tong Li, Pengfei Gao, Ying Guo, Cheng Zhen, Pengfei Yan, and Shiming Xiang. Cooperation does matter: Exploring multi-order bilateral relations for audio- visual segmentation. In CVPR, 2024. 1, 3, 5, 6

    2024

  34. [42]

    Dino: Detr with improved denoising anchor boxes for end-to-end object detection

    Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung-Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605, 2022. 3

    2022 arXiv

  35. [43]

    Audio–visual segmentation

    Jinxing Zhou, Jianyuan Wang, Jiayi Zhang, Weixuan Sun, Jing Zhang, Stan Birchfield, Dan Guo, Lingpeng Kong, Meng Wang, and Yiran Zhong. Audio–visual segmentation. In ECCV, 2022. 1, 2, 3, 4, 5, 6

    2022

  36. [44]

    Audio-visual segmentation with semantics

    Jinxing Zhou, Xuyang Shen, Jianyuan Wang, Jiayi Zhang, Weixuan Sun, Jing Zhang, Stan Birchfield, Dan Guo, Ling- peng Kong, Meng Wang, et al. Audio-visual segmentation with semantics. IJCV, 2024. 1, 2, 3, 5, 6

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.