Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

Multimodal Class-aware Semantic Enhancement Network for Audio-Visual Video Parsing

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that explicitly decoupling audio/visual segment features into event-specific and background streams before intra- and cross-modal interaction avoids semantic interference and yields new state-of-the-art audio-visual video…

desk verdict Solid, incremental SOTA on LLP with a clean decoupling idea and a few reporting gaps; the pseudo-label confound is real but not fatal. read the letter →

arxiv 2412.11248 v2 pith:U6RT5R6G submitted 2024-12-15 cs.CV cs.MM

Pengcheng Zhao , Jinxing Zhou , Yang Zhao , Dan Guo , Yanxiang Chen This is my paper · ORCID
classification cs.CVcs.MM
keywords audio-visualvideoparsingsemanticinterferenceclass-awarefeaturedecouplingeventco-occurrencemodelinglocal-globalfusionweaklysupervisedlearningpseudo-labelsbackground
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper takes on audio-visual video parsing—recognizing all events in each audio and visual segment of a video—and argues that the usual practice of feeding whole-segment features into attention layers causes semantic interference: a segment with only speech, for instance, can pick up violin semantics by attending to a segment containing a violin. To avoid this, the authors propose a network that first decouples each segment's mixed representation into one feature per event class plus a background feature, using separate linear projections. It then performs intra- and cross-modal enhancement at the class level, modeling which events tend to co-occur within a timestamp and matching each local segment to a global video representation of the same class. On the LLP benchmark, this MM-CSE network reaches an average F-score of 65.4% with CLAP/CLIP features (2.2 points above the previous state of the art, VALOR) and 61.5% with VGGish/ResNet features (1.0 point higher). The authors attribute the gains to class-aware decoupling and the explicit background stream.

What carries the argument

The load-bearing mechanism is the Class-Aware Feature Decoupling (CAFD) module paired with the Fine-Grained Semantic Enhancement (FGSE) module. CAFD projects each segment's mixed audio/visual feature through K+1 independent linear layers, producing K event-specific features and one background feature, and blends the background into each event stream via a learned weight; a reconstruction loss and an orthogonality loss keep the decoupling faithful and separable. FGSE then operates on these class-wise features: the Segment-wise Event Co-occurrence Modeling (SECM) block computes a softmax similarity matrix among class features at the same timestamp (and between the two modalities) to model which events tend to appear together, supervised by a co-occurrence loss that compares this map to the co-occurrence pattern in the pseudo-labels; the Local-Global Semantic Fusion (LGSF) block averages the class-wise features across time to form a global video representation per class, computes a cosine similarity between each local segment and that global representation, and fuses the matched global semantics into the local feature. L stacked FGSE layers refine the features, and the resulting class-level audio and visual representations are fed to an event parser. Everything is trained under the weak video-level label plus segment-level pseudo-labels.

What would settle it

Train the same MM-CSE architecture using pseudo-labels with a known fraction of random label corruption (e.g., flipping 10%, 20%, and 40% of the segment-level event labels) and measure the resulting F-score; a method truly robust to semantic interference should degrade gracefully, while a method that is merely repeating the teacher's errors would show a large drop. Alternatively, on a random subset of LLP videos, ask human annotators to label a few segments and compare the pseudo-labels to human labels to quantify the noise floor and test whether the model's errors correlate with pseudo-label errors.

Watch

Extended reading notes

Core claim

The paper's central assertion is that the bottleneck in audio-visual video parsing is semantic interference during intra- and cross-modal temporal interaction: when whole segments are mixed together, unrelated event semantics bleed across segments. The proposed solution decouples each segment's audio/visual feature into K event-specific features and one background feature, then runs class-level attention so that an event class aggregates information only from the same or co-occurring classes, both within a modality and across modalities. With the addition of a co-occurrence loss that supervises the learned class-relation map and a local-global fusion that pulls in informative global video semantics, the network sets a new state of the art: 65.4% average F-score on LLP with CLAP/CLIP features and 61.5% with VGGish/ResNet features, exceeding the VALOR baseline by 2.2 and 1.0 points respectively.

Load-bearing premise

The method's segment-level supervision comes from pseudo-labels generated by a separate teacher model (VALOR), and those pseudo-labels also serve as ground truth for the co-occurrence loss; if those pseudo-labels are noisy or biased, the gains may partly reflect the teacher's errors rather than the model's own strength.

Editorial extensions

If this is right

  • If the decoupling claim holds, any audio-visual method that attends over holistic segment features is leaving performance on the table, and class-wise decoupling should benefit other video-understanding tasks where multiple events overlap.
  • The background class contributes a 1.2-point average gain over decoupling with only event classes (65.4% vs 64.2%), indicating that explicitly modeling non-event content is a large part of the improvement.
  • The co-occurrence loss adds 0.6 points on average (64.8% to 65.4%), showing that supervising the inter-class relation map with segment-level pseudo-labels improves multi-event recognition.
  • The same architecture gains 2.2 points with CLAP/CLIP features and 1.0 point with VGGish/ResNet features, so the benefit is not tied to a single feature extractor.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Untested in the paper: training MM-CSE with pseudo-labels from a different teacher or with oracle segment labels on a subset would reveal how much of the 2.2-point gain comes from the decoupling architecture versus inheriting the teacher's labeling tendencies.
  • Untested extension: replacing the fixed K linear layers with text-conditioned projections would allow parsing unseen event classes without retraining.
  • Testable variant: making the background blending weight per-class rather than a single scalar per segment could give finer control on segments that contain both an event and informative context.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes MM-CSE, a network for audio-visual video parsing (AVVP). It introduces a Class-Aware Feature Decoupling (CAFD) module that splits holistic audio/visual features into K event-specific features plus one background feature, and a Fine-Grained Semantic Enhancement (FGSE) module with Segment-wise Event Co-occurrence Modeling (SECM) and Local-Global Semantic Fusion (LGSF) blocks for intra- and cross-modal interactions. The model is trained with reconstruction, orthogonality, and event co-occurrence losses in addition to a basic binary cross-entropy loss. On the LLP dataset, the authors report state-of-the-art results: 65.4% average F-score with CLAP/CLIP features (vs. 63.2% for VALOR*) and 61.5% with VGGish/ResNet features (vs. 60.5% for VALOR). The paper includes ablations of each module and loss, efficiency comparisons, and qualitative visualizations.

Significance. The idea of decoupling semantically mixed features into class-wise event-specific and background features to reduce semantic interference is well-motivated and relevant to the AVVP task. The modular architecture is plausible, and the reported gains over the previous state of the art are non-trivial. The paper also provides an efficiency analysis and qualitative evidence of improved event co-occurrence modeling. However, the central claim is currently not fully pinned down because the training objective heavily relies on VALOR-generated pseudo-labels, including the co-occurrence loss, so the margin over VALOR cannot be cleanly attributed to the proposed architecture without a control experiment. In addition, I found concrete numerical inconsistencies between the main paper and the supplementary material, which need to be corrected. If the authors add the missing control and fix these issues, the paper would be a solid empirical contribution.

major comments (3)
  1. [Sec. 3.3 (Eqs. 7 and 10), Table 4] The central claim of a +2.2% improvement over VALOR* (Table 1) is confounded by the supervision signal. The training objective Lbasic in Eq. 7 uses BCE(pa, ŷ^a) + BCE(pv, ŷ^v), where ŷ^a and ŷ^v are segment-level pseudo-labels generated by the VALOR teacher, and the co-occurrence loss Lec (Eqs. 10–11) builds its target matrix M from these same pseudo-labels. Therefore the reported gains are a joint effect of the proposed architecture and the teacher's labeling quality. Table 4 shows that Lbasic alone (row #1) already reaches 64.3%, and adding Lec raises it to 65.4%; without a control that trains only with the video-level labels Y, or an analysis of pseudo-label accuracy/noise, the margin cannot be attributed to the semantic-interference reduction claimed in Sec. 1. Please add an experiment that trains with only video-level supervision (BCE(P,Y) only) and/or with corrupted or noised pseudo-labels to demonstrate robustness to teacher errors.
  2. [Table 3 vs. Supplementary Table 9] The ablation results for the FGSE blocks are inconsistent between the main paper and the supplementary. In Table 3, "blocks w/o SECM" gives Avg 61.6 and "blocks w/o LGSF" gives 63.2; in Supplementary Table 9 the same two rows report Avg 63.2 and 61.6, respectively. The relative importance of the SECM and LGSF blocks is thus reversed across the two tables. Please correct the numbers and re-state the corresponding conclusion in Sec. 4.3, since the claim that both blocks contribute is not affected but the magnitude and direction of each contribution are.
  3. [Supplementary Sec. 6.2 (Discussion with More Related Works)] The last sentence states that ETAVR achieves an F1 score of 72% on LLP, "compared to 88% for our model." No 88% result appears in Table 1 or anywhere else in the paper; the best reported average is 65.4%. This discrepancy is serious because it misrepresents the comparison and can mislead readers about the state of the art. Please clarify the evaluation protocol and provide the correct numbers, or remove the comparison if it uses a different setting.
minor comments (6)
  1. [Eq. 1] The notation for concatenation [;] is ambiguous: eF^m_bg is T×1×d2 and must be repeated K times before concatenation with eF^m_e, yielding a T×K×2d2 input to h. Please state this explicitly.
  2. [Eq. 4] Please specify that the L2 normalization in ∥·∥ is applied along the feature dimension (d2) and that the inner product is taken over that dimension to produce the T×K×1 relevance map.
  3. [Sec. 4.1 (Implementation details)] The statement "The code will be publicly available" is vague; please provide a link or a clear commitment, and report mean±std over at least three random seeds to support the numerical claims.
  4. [Fig. 2] The framework diagram is very dense; adding a legend or simplifying the notation would improve readability.
  5. [Sec. 4.4 (Fig. 5)] The t-SNE visualization lacks axis labels and a count of embedded points; consider adding a quantitative cluster-quality metric (e.g., silhouette score) to substantiate the claim of intra-class compactness and inter-class separation.
  6. [Sec. 2 (Related Work)] The phrase "the most recent work (Lai, Chen, and Yu-Chiang 2023)" is inaccurate because the same paragraph cites later segment-wise pseudo-labeling works (Zhou et al. 2023, 2024c); please rephrase to avoid misleading the reader about chronology.

Circularity Check

1 steps flagged · score 2.0 of 10

The main SOTA claim is not circular; the only by-construction step is the co-occurrence map visualization, which directly displays the L_ec training target.

  1. fitted input called prediction [Sec. 3.3 (Eqs. 10-11) and Sec. 4.4 (Visualization of learned event co-occurrence map, Fig. 4)]
    "We can also obtain the event co-occurrence matrix M^{m,m'} from the segment-level labels ... as ground truth ... Then, the Lec can be computed by, Lec = MSE(β^{m,m'}, M^{m,m'}). ... We then present the learned event co-occurrence map, derived by averaging the β^{av} ... In Fig. 4, the visualization illustrates that our model effectively captures the co-occurrence among relevant events."

    The learned co-occurrence weight β is not discovered freely: Eq. 11 directly minimizes MSE between β and M, and Eq. 10 constructs M deterministically from the segment-level pseudo-labels. The averaged β displayed in Fig. 4 is therefore a rendering of the training target, so the claim that the visualization shows the model 'effectively captures' co-occurrence is true by construction rather than by independent evidence. Because M is built from VALOR-generated pseudo-labels, the map also reflects the teacher's co-occurrence statistics rather than independently verified event relations. This is a local circular validation, and it does not force the main parsing results, which are evaluated on ground-truth test labels.

full rationale

The core claim—that the MM-CSE architecture improves audio-visual video parsing—is not circular. CAFD and FGSE are novel architectural components whose contributions are tested by ablations on the ground-truth-labeled LLP test set, and no equation defines the reported F-scores in terms of the proposed modules or the pseudo-labels. Training with VALOR segment-level pseudo-labels is a legitimate weak-supervision strategy, not a self-definitional reduction: the student can underperform or outperform the teacher, and the final evaluation is external. The paper's self-citations (Zhou et al.) appear in related work and do not carry the load of the central claim; no uniqueness theorem is imported from the authors' prior work. The only genuine by-construction element is the event co-occurrence map visualization, which merely shows the L_ec target; this is a minor, non-central circularity. The pseudo-label dependency is best viewed as a possible correctness/confound risk rather than circularity, since the paper provides no label-noise or teacher-quality control experiment. Overall central derivation remains self-contained, so the circularity score is low.

Assumptions & free parameters 5 free parameters · 3 assumptions · 1 invented entities

The method relies on several externally provided inputs: pretrained features, VALOR pseudo labels, and the LLP dataset annotations. The main free hyperparameters are the loss weights and network depth. The background class is a modeling construct without independent validation.

free parameters (5)
  • lambda1 = 0.1
    Loss weight for orthogonality loss, selected by ablation (Sec 6.2, Table 6).
  • lambda2 = 0.1
    Loss weight for co-occurrence loss, selected by ablation (Sec 6.2, Table 6).
  • L = 4
    Number of stacked FGSE layers; set empirically without ablation.
  • d2 = 128
    Hidden dimension of class-wise features; design choice.
  • learning rate = 3e-4
    Optimizer setting in Sec 4.1.
assumptions (3)
  • domain assumption Segment-level pseudo labels from VALOR are accurate enough to serve as ground truth for training and for the co-occurrence loss.
    Used in Eq. 10 to define co-occurrence ground truth; no error analysis of pseudo labels.
  • domain assumption Pretrained CLIP, R(2+1)D, and CLAP features contain sufficient event information for decoupling.
    Features are treated as given inputs; the decoupling module is applied on top of them.
  • ad hoc to paper K+1 linear layers can separate mixed event semantics without supervision at segment level.
    The CAFD module assumes this architectural capability; losses guide it but do not guarantee it.
invented entities (1)
  • Background class in feature space
    purpose: Represent non-event and event-related background content in each segment.
    Introduced as an extra (K+1)-th linear head; supported only by internal ablation, no external label or benchmark.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multimodal Class-aware Semantic Enhancement Network for Audio-Visual Video Parsing." pith.science (2026). https://pith.science/paper/U6RT5R6G

@misc{pith2026241211248,
  author       = {Pith},
  title        = {Pith review of: Multimodal Class-aware Semantic Enhancement Network for Audio-Visual Video Parsing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U6RT5R6G}},
  note         = {Machine review of arXiv:2412.11248}
}
read the original abstract

The Audio-Visual Video Parsing task aims to recognize and temporally localize all events occurring in either the audio or visual stream, or both. Capturing accurate event semantics for each audio/visual segment is vital. Prior works directly utilize the extracted holistic audio and visual features for intra- and cross-modal temporal interactions. However, each segment may contain multiple events, resulting in semantically mixed holistic features that can lead to semantic interference during intra- or cross-modal interactions: the event semantics of one segment may incorporate semantics of unrelated events from other segments. To address this issue, our method begins with a Class-Aware Feature Decoupling (CAFD) module, which explicitly decouples the semantically mixed features into distinct class-wise features, including multiple event-specific features and a dedicated background feature. The decoupled class-wise features enable our model to selectively aggregate useful semantics for each segment from clearly matched classes contained in other segments, preventing semantic interference from irrelevant classes. Specifically, we further design a Fine-Grained Semantic Enhancement module for encoding intra- and cross-modal relations. It comprises a Segment-wise Event Co-occurrence Modeling (SECM) block and a Local-Global Semantic Fusion (LGSF) block. The SECM exploits inter-class dependencies of concurrent events within the same timestamp with the aid of a new event co-occurrence loss. The LGSF further enhances the event semantics of each segment by incorporating relevant semantics from more informative global video features. Extensive experiments validate the effectiveness of the proposed modules and loss functions, resulting in a new state-of-the-art parsing performance.

Figures

Figures reproduced from arXiv: 2412.11248 by the authors.

Figure 1
Figure 1. (a) Illustration of the AVVP task. (b) Previous [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Framework Overview. (a) Our MM-CSE network primarily consists of two core modules: the audio-visual Class-Aware [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Qualitative comparisons. Compared to the previ [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Visualization of our decoupled class-wise features. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Qualitative examples of video parsing. Compared to the previous SOTA method VALOR, our method performs better [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Towards Open-Vocabulary Audio-Visual Event Localization

    cs.CV 2024-11 conditional novelty 6.0 of 10

    An ImageBind-based fine-tuned model outperforms a training-free zero-shot baseline on the new OV-AVEBench, which spans 67 event classes with 21 unseen at test time.

  2. Dense Audio-Visual Event Localization under Cross-Modal Consistency and Multi-Temporal Granularity Collaboration

    cs.CV 2024-12 conditional novelty 5.0 of 10

    CCNet combines cross-modal consistency and multi-temporal granularity modules to achieve state-of-the-art dense audio-visual event localization on UnAV-100.

Reference graph

Works this paper leans on

41 extracted references · 33 canonical work pages · cited by 2 Pith papers

  1. [1]

    Cheng, H.; Liu, Z.; Zhou, H.; Qian, C.; Wu, W.; and Wang, L. 2022. Joint-Modal Label Denoising for Weakly-Supervised Audio-Visual Video Parsing. In Proceedings of the European Conference on Computer Vision (ECCV), 431--448

    work page 2022

  2. [2]

    Cheng, Y.; Wang, R.; Pan, Z.; Feng, R.; and Zhang, Y. 2020. Look, listen, and attend: Co-attention network for self-supervised audio-visual representation learning. In Proceedings of the ACM International Conference on Multimedia (ACM MM), 3884--3892

    work page 2020

  3. [3]

    Fan, Y.; Wu, Y.; Lin, Y.; and Du, B. 2023. Revisit Weakly-Supervised Audio-Visual Video Parsing from the Language Perspective. In Advances in Neural Information Processing Systems (NeurIPS), 1--13

    work page 2023

  4. [4]

    Gao, J.; Chen, M.; and Xu, C. 2023. Collecting Cross-Modal Presence-Absence Evidence for Weakly-Supervised Audio-Visual Event Perception. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 18827--18836

    work page 2023

  5. [5]

    F.; Ellis, D

    Gemmeke, J. F.; Ellis, D. P.; Freedman, D.; Jansen, A.; Lawrence, W.; Moore, R. C.; Plakal, M.; and Ritter, M. 2017. Audio set: An ontology and human-labeled dataset for audio events. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 776--780

    work page 2017

  6. [6]

    Geng, T.; Wang, T.; Zhang, Y.; Duan, J.; Guan, W.; and Zheng, F. 2024. UniAV: Unified Audio-Visual Perception for Multi-Task Video Localization. arXiv:2404.03179

    arXiv 2024

  7. [7]

    Guo, R.; Ying, X.; Chen, Y.; Niu, D.; Li, G.; Qu, L.; Qi, Y.; Zhou, J.; Xing, B.; Yue, W.; Shi, J.; Wang, Q.; Zhang, P.; and Liang, B. 2023. Audio-Visual Instance Segmentation. arXiv preprint arXiv:2310.18709

    arXiv 2023

  8. [8]

    He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 770--778

    2016

Show all 41 references
  1. [9]

    P.; Gemmeke, J

    Hershey, S.; Chaudhuri, S.; Ellis, D. P.; Gemmeke, J. F.; Jansen, A.; Moore, R. C.; Plakal, M.; Platt, D.; Saurous, R. A.; Seybold, B.; et al. 2017. CNN architectures for large-scale audio classification. In IEEE International Conference on Acoustics, Speech and Signal Process...

    2017

  2. [10]

    Hou, W.; Li, G.; Tian, Y.; and Hu, D. 2024. Toward Long Form Audio-Visual Video Understanding. ACM Transactions on Multimedia Computing, Communications and Applications, 20(9): 1--26

    2024

  3. [11]

    Jiang, X.; Xu, X.; Chen, Z.; Zhang, J.; Song, J.; Shen, F.; Lu, H.; and Shen, H. T. 2022. DHHN: Dual Hierarchical Hybrid Network for Weakly-Supervised Audio-Visual Video Parsing. In Proceedings of the ACM International Conference on Multimedia (ACM MM), 719--727

    2022

  4. [12]

    Lai, Y.-H.; Chen, Y.-C.; and Yu-Chiang, F. W. 2023. Modality-Independent Teachers Meet Weakly-Supervised Audio-Visual Event Parser. In Advances in Neural Information Processing Systems (NeurIPS), 1--19

    2023

  5. [13]

    Li, Z.; Guo, D.; Zhou, J.; Zhang, J.; and Wang, M. 2024. Object-Aware Adaptive-Positivity Learning for Audio-Visual Question Answering. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 3306--3314

    2024

  6. [14]

    Lin, Y.-B.; Tseng, H.-Y.; Lee, H.-Y.; Lin, Y.-Y.; and Yang, M.-H. 2021. Exploring Cross-Video and Cross-Modality Signals for Weakly-Supervised Audio-Visual Video Parsing. In Advances in Neural Information Processing Systems (NeurIPS), 1--13

    2021

  7. [15]

    Mao, Y.; Shen, X.; Zhang, J.; Qin, Z.; Zhou, J.; Xiang, M.; Zhong, Y.; and Dai, Y. 2024. TAVGBench: Benchmarking text to audible-video generation. In Proceedings of the ACM international conference on multimedia (ACM MM), 6607--6616

    2024

  8. [16]

    Mo, S.; and Tian, Y. 2022. Multi-modal Grouping Network for Weakly-Supervised Audio-Visual Video Parsing. In Advances in Neural Information Processing Systems (NeurIPS), 1--12

    2022

  9. [17]

    Qian, R.; Hu, D.; Dinkel, H.; Wu, M.; Xu, N.; and Lin, W. 2020. Multiple sound sources localization from coarse to fine. In Proceedings of the European Conference on Computer Vision (ECCV), 292--308

    2020

  10. [18]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning (ICML), 8748--8763

    2021

  11. [19]

    Senocak, A.; Kim, J.; Oh, T.-H.; Li, D.; and Kweon, I. S. 2023. Event-specific audio-visual fusion layers: A simple and new perspective on video understanding. In WACV, 2237--2247

    2023

  12. [20]

    Shen, X.; Li, D.; Zhou, J.; Qin, Z.; He, B.; Han, X.; Li, A.; Dai, Y.; Kong, L.; Wang, M.; et al. 2023. Fine-grained audible video description. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 10585--10596

    2023

  13. [21]

    Tian, Y.; Li, D.; and Xu, C. 2020. Unified multisensory perception: Weakly-supervised audio-visual video parsing. In Proceedings of the European Conference on Computer Vision (ECCV), 436--454

    2020

  14. [22]

    Tian, Y.; Shi, J.; Li, B.; Duan, Z.; and Xu, C. 2018. Audio-visual event localization in unconstrained videos. In Proceedings of the European Conference on Computer Vision (ECCV), 247--263

    2018

  15. [23]

    Tran, D.; Wang, H.; Torresani, L.; Ray, J.; LeCun, Y.; and Paluri, M. 2018. A closer look at spatiotemporal convolutions for action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 6450--6459

    2018

  16. [24]

    Van der Maaten, L.; and Hinton, G. 2008. Visualizing data using t-SNE. Journal of machine learning research

    2008

  17. [25]

    N.; Kaiser, .; and Polosukhin, I

    Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), 1--11

    2017

  18. [26]

    Wu, Y.; Chen, K.; Zhang, T.; Hui, Y.; Berg-Kirkpatrick, T.; and Dubnov, S. 2023. Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1--5

    2023

  19. [27]

    Wu, Y.; and Yang, Y. 2021. Exploring heterogeneous clues for weakly-supervised audio-visual video parsing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 1326--1335

    2021

  20. [28]

    Yu, J.; Cheng, Y.; Zhao, R.-W.; Feng, R.; and Zhang, Y. 2022. MM-Pyramid : Multimodal Pyramid Attentional Network for Audio-Visual Event Localization and Video Parsing. In Proceedings of the ACM International Conference on Multimedia (ACM MM), 6241--6249

    2022

  21. [29]

    Zhang, J.; and Li, W. 2023. Multi-Modal and Multi-Scale Temporal Fusion Architecture Search for Audio-Visual Video Parsing. In Proceedings of the ACM International Conference on Multimedia (ACM MM), 3328--3336

    2023

  22. [30]

    Zhao, H.; Gan, C.; Rouditchenko, A.; Vondrick, C.; McDermott, J.; and Torralba, A. 2018. The sound of pixels. In Proceedings of the European Conference on Computer Vision (ECCV), 570--586

    2018

  23. [31]

    Zhou, J.; Guo, D.; Guo, R.; Mao, Y.; Hu, J.; Zhong, Y.; Chang, X.; and Wang, M. 2024 a . Towards Open-Vocabulary Audio-Visual Event Localization. arXiv:2411.11278

    2024 arXiv

  24. [32]

    Zhou, J.; Guo, D.; Mao, Y.; Zhong, Y.; Chang, X.; and Wang, M. 2024 b . Label-anticipated Event Disentanglement for Audio-Visual Video Parsing. In European Conference on Computer Vision (ECCV), 1--22

    2024

  25. [33]

    Zhou, J.; Guo, D.; Mao, Y.; Zhong, Y.; Chang, X.; and Wang, M. 2025. Label-anticipated event disentanglement for audio-visual video parsing. In European Conference on Computer Vision (ECCV), 35--51. Springer

    2025

  26. [34]

    Zhou, J.; Guo, D.; and Wang, M. 2022. Contrastive Positive Sample Propagation along the Audio-Visual Event Line. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 1--18

    2022

  27. [35]

    Zhou, J.; Guo, D.; Zhong, Y.; and Wang, M. 2023. Improving audio-visual video parsing with pseudo visual labels. arXiv:2303.02344

    2023 arXiv

  28. [36]

    Zhou, J.; Guo, D.; Zhong, Y.; and Wang, M. 2024 c . Advancing Weakly-Supervised Audio-Visual Video Parsing via Segment-wise Pseudo Labeling. International Journal of Computer Vision (IJCV), 1--22

    2024

  29. [37]

    Zhou, J.; Shen, X.; Wang, J.; Zhang, J.; Sun, W.; Zhang, J.; Birchfield, S.; Guo, D.; Kong, L.; Wang, M.; and Zhong, Y. 2024 d . Audio-Visual Segmentation with Semantics. International Journal of Computer Vision (IJCV), 1--21

    2024

  30. [38]

    Zhou, J.; Wang, J.; Zhang, J.; Sun, W.; Zhang, J.; Birchfield, S.; Guo, D.; Kong, L.; Wang, M.; and Zhong, Y. 2022. Audio--visual segmentation. In Proceedings of the European Conference on Computer Vision (ECCV), 386--403

    2022

  31. [39]

    Zhou, J.; Zheng, L.; Zhong, Y.; Hao, S.; and Wang, M. 2021. Positive sample propagation along the audio-visual event line. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 8436--8444

    2021

  32. [40]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  33. [41]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.