TeMTG: Text-Enhanced Multi-Hop Temporal Graph Modeling for Audio-Visual Video Parsing

Faegheh Sardari; Fei Li; Peiliang Zhang; Ruohao Guo; Wenwu Wang; Yaru Chen; Zhenbo Li

REVIEW 4 major objections 5 minor 20 references

TeMTG: Text-Enhanced Multi-Hop Temporal Graph Modeling for Audio-Visual Video Parsing

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read TeMTG combines text embeddings generated from CLAP and CLIP pseudo labels with a multi-hop temporal graph to reach state-of-the-art segment-level results on the LLP audio-visual video parsing benchmark.

desk verdict Table 2 contradicts the paper's central claim: removing the multi-hop temporal graph makes the model better, so the reported SOTA gains come from text enhancement alone. read the letter →

arxiv 2505.02096 v1 pith:HRQKZT6N submitted 2025-05-04 cs.MM

Yaru Chen , Peiliang Zhang , Fei Li , Faegheh Sardari , Ruohao Guo , Zhenbo Li , Wenwu Wang This is my paper · ORCID

classification cs.MM

keywords audio-visualvideoparsingweaklysupervisedlearningtext-enhancedfeaturefusionmulti-hoptemporalgraphattentionnetworkCLAPCLIPLLPdataset

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that weakly supervised audio-visual video parsing improves when explicit semantic guidance from text is fused into the features and when temporal relations between segments are modeled explicitly. The proposed framework, TeMTG, turns CLAP and CLIP segment-level pseudo labels into template natural-language sentences, re-embeds those sentences with the same pretrained text encoders, and merges the resulting text embeddings into the audio and visual features. It then builds a bidirectional $K$-hop temporal graph over the segments of each modality and propagates information with multi-head graph attention, so each segment sees both nearby and more distant neighbors. On the LLP benchmark the full system reports state-of-the-art segment-level results — 74.4 audio F1, 72.9 visual F1, and 74.1 Event@AV — exceeding the best prior published numbers by 4.2, 1.6, and 5.3 points. A sympathetic reading is that text semantics and explicit local temporal structure are complementary remedies for the ambiguous event boundaries left by video-level supervision.

What carries the argument

Two mechanisms carry the argument. The first is template-caption re-embedding: segment-level pseudo labels $p^a_t, p^v_t$ from CLAP and CLIP are converted into fixed natural-language sentences, concatenated with conjunctions when several events co-occur, re-embedded through the text branches of the same CLAP/CLIP encoders, and fused with the audio/visual features by concatenation followed by a two-layer MLP with LayerNorm and a final linear projection back to the original dimension. The second is the Multi-Hop Temporal Graph: for each modality, segments are nodes and bidirectional edges connect every node to all nodes within $K$ steps ($K=4$ in the experiments), with a self-loop; the adjacency matrix is $A_{ij}=1$ if $0 \le j-i \le K$, and message passing uses multi-head residual graph attention with LeakyReLU attention weights, followed by global mean pooling and an MLP that re-injects the whole-video representation into every node. The text fusion does the work of making semantically similar segments distinguishable; the graph does the work of propagating context across time so that both short and long events stay temporally coherent. The graph layer is inserted after each of the two feature-aggregation branches of the CoLeaF baseline.

What would settle it

Re-run the ablated configurations on the LLP test split with several seeds: the paper's own Table 2 shows the text-enhancement-only model (w/o MTG) already exceeds the full TeMTG on segment-level audio F1 (76.5 vs 74.4) and Event@AV (75.7 vs 74.1). If that ordering reproduces, the claim that the temporal graph improves joint segment parsing on top of text enhancement fails for those indicators, leaving only the comparison against other published methods as evidence for the overall state-of-the-art claim; if the full model wins on replication, the component claim stands.

Watch

Extended reading notes

Core claim

The central claim is that combining text-enhanced feature fusion with multi-hop temporal graph modeling yields the best published results on the LLP dataset for weakly supervised audio-visual video parsing. On the paper's own terms, the text enhancement component supplies what weak video-level labels cannot: per-segment semantic guidance. Segment-level pseudo labels produced by CLAP and CLIP are rewritten as captions such as 'This is the sound of x audio event' or 'There is no sound in the segment', re-embedded by the same pretrained text encoders, and fused with audio and visual features through a two-layer MLP. The multi-hop temporal graph then links each segment to its neighbors up to $K$ steps away in both directions and applies residual multi-head graph attention, letting the model reason about short-term continuity and long-range event dependencies. The reported segment-level gains over the previous best system are +4.2 audio F1, +1.6 visual F1, and +5.3 Event@AV; the paper also reports that joint audio-visual event parsing (62.0 segment-level AV F1) remains below one competitor's 64.4, which it attributes to residual noise in the CLAP/CLIP pseudo labels at event boundaries.

Load-bearing premise

The load-bearing premise is that re-embedding CLAP/CLIP pseudo labels transformed into fixed template sentences yields text vectors that genuinely describe what happens in each segment; if the pseudo labels are noisy, the text guidance can distort the very features it is meant to disambiguate, and the paper's own reported weakness on audio-visual event boundaries suggests such noise is already present.

Editorial extensions

If this is right

On the LLP benchmark, the combined model reports 74.4 segment-level audio F1, 72.9 visual F1, and 74.1 segment-level Event@AV, the best published numbers on those indicators and gains of 4.2, 1.6, and 5.3 points over the previous best system.
The ablation shows text enhancement alone is the largest single contributor, lifting audio parsing by more than ten points over the same-feature baseline, which implies that re-embedded caption semantics can substitute, in part, for the segment-level supervision the task lacks.
The $K$-hop temporal graph, when used without text enhancement, still improves single-modality parsing over the baseline (audio 64.2 to 64.8, visual 67.4 to 68.9 at segment level), indicating that explicit local temporal dependencies are learnable from weak video-level labels.
Because the full model's joint Event@AV is the best reported, the framework implies that text semantics plus temporal propagation yields comparatively accurate audio-visual event boundaries, which matters for applications such as surveillance and content-based indexing.
The drop from text-enhancement-only to the full model (for example 76.5 to 74.4 audio F1) shows the two modules interact; the paper attributes this to a smoothing effect of temporal aggregation that reduces feature distinctiveness and names it as a limitation to explore.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper's own ablation suggests the temporal graph may cost accuracy once text enhancement is present (the w/o MTG configuration beats the full model on most segment-level metrics); my inference is that the state-of-the-art claim is carried by the text component plus the comparison to published systems, and a natural redesign would gate or weight the graph's propagation so its smoothing does not
A cheap experiment the paper does not report would sweep the hop count $K$ from 1 upward: if performance saturates at small $K$, the long-range modeling justification is not doing the work, while a peak near $K=4$ would confirm that temporal range genuinely matters and show how far the smoothing extends.
The template-caption trick should transfer to other weakly supervised temporal tasks with asynchronous modalities, such as sound event detection with video or audio-visual localization; replacing the fixed templates with free-form captions from a language model, or adding a confidence gate before re-embedding, would test whether pseudo-label noise rather than fusion architecture is the bottleneck.
The graphs are built separately per modality, so a further extension the paper leaves implicit is an inter-modal temporal edge set that models asynchrony directly (events heard but not seen, or vice versa); the current design handles that only implicitly through the later attention-based aggregation.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The punchline is that the paper's advertised contribution doesn't survive its own ablation. The multi-hop temporal graph (MTG) is supposed to be a central piece, but Table 2 shows that removing it strictly improves or ties every single metric at both segment and event level. On event-level audio, the gap is 4.8 points (66.6 vs 61.9). Section 3.2 credits MTG for the +5.3 Event@AV gain over NREP, yet the text-enhancement-only variant beats NREP by 6.9. So the gain comes from text enhancement, not the graph.

What is actually good here: the text-enhancement recipe works. Converting CLAP/CLIP pseudo-labels into natural-language descriptions, re-embedding them with the same encoders, and fusing via a small MLP gives large improvements over CoLeaF and lifts several LLP metrics beyond previous SOTA. That is a legitimate, if incremental, extension of the pseudo-label line in VALOR++ and LSLD+, and the authors report their ablation fully — the table is right there, so the contradiction is visible rather than hidden. I give credit for that.

The soft spots are the obvious ones. The named contribution is not supported by the evidence the paper itself provides. The best configuration in Table 2 is 'w/o MTG', which is not the model proposed. There are no error bars, no code, and only partial training details (K=4, but no pseudo-label threshold, MLP hidden dimension, seeds, or GAT heads). For a 5-page conference paper these omissions are common, but they matter more here because the positive result already looks fragile without them.

Novelty is incremental — each block is known — but that alone wouldn't bother me. The issue is that the story the authors tell about their own numbers is off. They acknowledge in Section 3.3 that 'some indicators are slightly lower' with the full model, but 4.8 points is not slight, and it directly undercuts the abstract's claim that combining text enhancement and MTG is what achieves SOTA.

Who gets value from this paper: anyone working on weakly supervised AVVP who wants a fast, effective text-enhancement trick. The MTG part should be repositioned as optional or dropped.

If I were the editor, I would send it to review rather than desk-reject, because the empirical finding about text enhancement is real and the ablation is informative. But I'd expect a major revision that either demotes MTG or finds a setting where it actually helps.

Referee Report

4 major / 5 minor

Summary. The paper proposes TeMTG, a weakly supervised audio-visual video parsing (AVVP) framework that combines text-enhanced feature fusion with a multi-hop temporal graph (MTG). The text-enhancement component uses CLAP/CLIP to generate segment-level pseudo labels, converts them into natural-language descriptions, and fuses the resulting text embeddings with audio and visual features via modality-specific MLPs. The multi-hop temporal graph connects each segment to neighbors within K hops and applies multi-head graph attention to model temporal dependencies. Experiments on the LLP dataset report state-of-the-art results on several segment-level and event-level metrics, with the largest gains on audio F1 and Event@AV. An ablation study compares the full model against variants without text enhancement and without the MTG.

Significance. If the reported results hold, the text-enhancement mechanism is practically significant: on the standard LLP benchmark it yields large improvements over recent methods, such as a 5.3-point gain in segment-level Event@AV over NREP (74.1 vs. 68.8), and the evaluation is conducted against external baselines without circularity. However, the paper's central contribution is the multi-hop temporal graph, and the paper's own ablation contradicts the claim that this component is beneficial. The full TeMTG model is outperformed or tied by the text-enhancement-only variant on every segment-level metric and on most event-level metrics, including a 4.8-point drop in event-level audio F1 and a 3.8-point drop in event-level Event@AV. The attribute of the SOTA gains to MTG in Section 3.2 is therefore not supported by the controlled comparison, and the paper does not present the best model it found. The core idea of text enhancement is promising and well motivated, but the paper as written does not establish the value of the proposed multi-hop temporal graph, so the contribution claim needs substantial revision.

major comments (4)

[Section 3.3, Table 2] The ablation shows that removing the multi-hop temporal graph (the w/o MTG variant) strictly improves or ties the full TeMTG on every segment-level metric and on all event-level metrics except visual F1, with the largest degradation being 4.8 points on event-level audio F1 (61.9 vs. 66.6) and 3.8 points on event-level Event@AV (62.2 vs. 66.0). This contradicts the abstract's claim that the combination of text enhancement and multi-hop temporal graph modeling is the source of SOTA performance, and it undermines the attribution in Section 3.2 of the +5.3 Event@AV gain over NREP to the multi-hop temporal graph, since the w/o MTG variant exceeds NREP by an even larger margin (75.7 vs. 68.8). The authors must either revise the contribution claim, present a configuration or analysis in which MTG helps, or reposition the contribution around text enhancement alone.
[Section 3.3] The sentence "Some TeMTG indicators are slightly lower than using only the TE mechanism" understates the observed degradation: compared to w/o MTG, the full model drops by 4.8 points on event-level audio F1, by 3.8 points on event-level Event@AV, and by 1.6 points on segment-level Event@AV, and it never improves on any reported metric. Because this sentence is the only discussion of the central negative result, the paper does not currently reconcile the proposed architecture with its own controlled comparison, which is an internal inconsistency that must be resolved before the SOTA claim can be accepted.
[Section 2.3, Equation (6)] The definition of the adjacency matrix is internally inconsistent: the text says "1, if 0 ≤ j−i ≤ K; 0, otherwise," but the equation assigns 0 in both cases, making the graph edgeless and the subsequent GAT propagation vacuous. This appears to be a typographical error, but as written it prevents the multi-hop temporal graph from being implemented as described, so the equation must be corrected to assign 1 to the within-K entries.
[Section 3.2, Table 1] The SOTA claim is phrased too strongly: TeMTG is below NREP on segment-level AV parsing (62.0 vs. 64.4), below NREP on event-level audio (61.9 vs. 62.8), and below both NREP and LSLD+ on event-level AV (53.2 vs. 57.6). The paper should state precisely which indicators are improved, quantify the trade-offs, and avoid the unqualified phrase "achieves SOTA performance" in the introduction and conclusion.

minor comments (5)

[Section 2.2, Equation (4)] Equation (4) for visual fusion uses z^a_t in the MLP expression, but it should use z^v_t to be consistent with the definition of the concatenated visual feature.
[Section 2.2, Equation (3)] The symbol "bs" in R^{bs×T×d} is not defined; it should be "b" for consistency with the audio case and the later notation.
[References, [12]] The venue name "NuerIPS" is misspelled; it should be "NeurIPS."
[Table 1 caption] The caption says "second results highlighted in text," but the table does not visibly indicate second-best entries; either underline them or adjust the caption.
[Section 3.1] The paper fixes the number of hops K=4 without any sensitivity analysis; given that the ablation shows the MTG component is harmful or neutral, a sweep over K would help clarify whether the negative effect is particular to this setting or fundamental.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: TeMTG's empirical SOTA claim is benchmarked externally and no fitted parameter is renamed as a prediction.

full rationale

TeMTG's central claim is an empirical SOTA result on the publicly benchmarked LLP dataset, with no theoretical derivation that reduces to its own inputs. The method uses segment-level pseudo labels generated by pretrained CLAP/CLIP, converts them into natural-language prompts, and re-embeds those prompts with the same pretrained text encoders; this is a design choice using external models, not a fitted quantity derived from the LLP test labels. The multi-hop temporal graph adjacency is defined purely by segment index distance (Eqs. 5 and 6), independent of the target labels, and the final evaluation compares against external published methods on the fixed LLP benchmark. The paper's self-citations, such as adopting CoLeaF as a baseline and citing CM-PIE for prior event-category connection work, are background references and are not load-bearing for the claimed result. The ablation study does show that removing the multi-hop temporal graph improves or ties every metric, which is an internal consistency weakness in attributing gains to MTG, but this is an empirical confound rather than a circularity: the ablation is a controlled comparison, not an input to the method. No equation is equivalent to its own output by construction, and no fitted parameter is renamed as a prediction, so no circular step is present.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or conceptual entities. Its load-bearing choices are all design or hyperparameter decisions: K, pseudo-label thresholds, MLP dimensions, and GAT configuration, several of which are unreported. The main scientific risk is the assumption that text descriptions built from noisy pseudo labels improve segment-level semantics, and the assumption that K-hop temporal aggregation helps more than it smooths.

free parameters (4)

K (number of temporal hops) = 4
Set by hand to 'ensure a balanced temporal dependency modeling across modalities'; no sensitivity analysis is reported.
Pseudo-label binarization threshold = not reported
CLAP/CLIP outputs are converted to binary segment-level pseudo labels, but the threshold or rule for binarization is not stated.
MLP hidden dimension m = not reported
The text fusion MLP in Equations (2) and (4) depends on a hidden dimension m that is never specified.
GAT heads and layer count = not reported
Multi-head residual GAT is used, but the number of heads H and the number of stacked layers are not reported.

assumptions (4)

domain assumption CLAP and CLIP embeddings encode audio and visual semantics relevant to the 25 LLP event categories.
The method relies on pretrained models whose features and text embeddings are treated as reliable semantic guides for events such as speech, cat, and music.
domain assumption Video-level weak labels plus pretrained-model pseudo labels are sufficient to train useful segment-level text prompts.
Text descriptions are generated from CLAP/CLIP pseudo labels, not from ground-truth segment labels; the paper assumes these pseudo labels are accurate enough for the fusion to help.
domain assumption Graph attention over K-hop temporal edges improves temporal reasoning without excessive smoothing.
This is the premise behind the multi-hop temporal graph, but the ablation in Table 2 shows the full model underperforms the text-enhancement-only variant, suggesting the smoothing effect may outweigh the benefit.
domain assumption The LLP evaluation protocol, including the mIoU >= 0.5 threshold and F1 metrics, is accepted as valid ground truth.
All comparisons use the standard LLP splits and metrics from the baseline paper, which the authors do not independently verify.

how reviews work

0 comments

Cite this review

Pith. "Pith review of TeMTG: Text-Enhanced Multi-Hop Temporal Graph Modeling for Audio-Visual Video Parsing." pith.science (2026). https://pith.science/paper/HRQKZT6N

@misc{pith2026250502096,
  author       = {Pith},
  title        = {Pith review of: TeMTG: Text-Enhanced Multi-Hop Temporal Graph Modeling for Audio-Visual Video Parsing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HRQKZT6N}},
  note         = {Machine review of arXiv:2505.02096}
}

read the original abstract

Audio-Visual Video Parsing (AVVP) task aims to parse the event categories and occurrence times from audio and visual modalities in a given video. Existing methods usually focus on implicitly modeling audio and visual features through weak labels, without mining semantic relationships for different modalities and explicit modeling of event temporal dependencies. This makes it difficult for the model to accurately parse event information for each segment under weak supervision, especially when high similarity between segmental modal features leads to ambiguous event boundaries. Hence, we propose a multimodal optimization framework, TeMTG, that combines text enhancement and multi-hop temporal graph modeling. Specifically, we leverage pre-trained multimodal models to generate modality-specific text embeddings, and fuse them with audio-visual features to enhance the semantic representation of these features. In addition, we introduce a multi-hop temporal graph neural network, which explicitly models the local temporal relationships between segments, capturing the temporal continuity of both short-term and long-range events. Experimental results demonstrate that our proposed method achieves state-of-the-art (SOTA) performance in multiple key indicators in the LLP dataset.

Figures

Figures reproduced from arXiv: 2505.02096 by the authors.

**Figure 2.** TeMTG architecture: Audio and visual features are fused with text embeddings, aggregated, and then processed by [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 19 canonical work pages

[1]

Unified multisensory perception: Weakly-supervised audio-visual video parsing

Yapeng Tian, Dingzeyu Li, and Chenliang Xu. Unified multisensory perception: Weakly-supervised audio-visual video parsing. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16, pages 436–454. Springer, 2020

work page 2020
[2]

Anchor-aware Deep Metric Learning for Audio-visual Retrieval

Donghuo Zeng, Yanan Wang, Kazushi Ikeda, and Yi Yu. Anchor-aware Deep Metric Learning for Audio-visual Retrieval. InProceedings of the 2024 International Conference on Multimedia Retrieval , pages 211–219, 2024

work page 2024
[3]

Boosting Audio Visual Question Answer- ing via Key Semantic-Aware Cues

Guangyao Li, Henghui Du, and Di Hu. Boosting Audio Visual Question Answer- ing via Key Semantic-Aware Cues. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 5997–6005, 2024

work page 2024
[4]

Open-Vocabulary Audio-Visual Semantic Segmentation

Ruohao Guo, Liao Qu, Dantong Niu, Yanyu Qi, Wenzhen Yue, Ji Shi, Bowei Xing, and Xianghua Ying. Open-Vocabulary Audio-Visual Semantic Segmentation. In Proceedings of the 32nd ACM International Conference on Multimedia , pages 7533–7541, 2024

work page 2024
[5]

Drcnet: Dynamic image restoration contrastive network

Fei Li, Lingfeng Shen, Yang Mi, and Zhenbo Li. Drcnet: Dynamic image restoration contrastive network. In European Conference on Computer Vision , pages 514–532. Springer, 2022

work page 2022
[6]

Collecting cross-modal presence-absence evidence for weakly-supervised audio-visual event percep- tion

Junyu Gao, Mengyuan Chen, and Changsheng Xu. Collecting cross-modal presence-absence evidence for weakly-supervised audio-visual event percep- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18827–18836, 2023

work page 2023
[7]

ColeaF: A Contrastive-Collaborative Learning Framework for Weakly Supervised Audio- Visual Video Parsing

Faegheh Sardari, Armin Mustafa, Philip JB Jackson, and Adrian Hilton. ColeaF: A Contrastive-Collaborative Learning Framework for Weakly Supervised Audio- Visual Video Parsing. In European Conference on Computer Vision , pages 1–17. Springer, 2024

work page 2024
[8]

Modality-independent teachers meet weakly-supervised audio-visual event parser

Yung-Hsuan Lai, Yen-Chun Chen, and Frank Wang. Modality-independent teachers meet weakly-supervised audio-visual event parser. Advances in Neural Information Processing systems, 36:73633–73651, 2023

work page 2023

Show all 20 references

[9]

Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation

Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg-Kirkpatrick, and Shlomo Dubnov. Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Sign...

2023
[10]

Learning transferable visual models from natural language supervision

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , p...

2021
[11]

Label-anticipated event disentanglement for audio-visual video parsing

Jinxing Zhou, Dan Guo, Yuxin Mao, Yiran Zhong, Xiaojun Chang, and Meng Wang. Label-anticipated event disentanglement for audio-visual video parsing. In European Conference on Computer Vision , pages 35–51. Springer, 2024

2024
[12]

Revisit weakly-supervised audio- visual video parsing from the language perspective

Yingying Fan, Yu Wu, Bo Du, and Yutian Lin. Revisit weakly-supervised audio- visual video parsing from the language perspective. Advances in Neural Informa- tion Processing Systems, 36, 2024

2024
[13]

Multi-modal grouping network for weakly- supervised audio-visual video parsing

Shentong Mo and Yapeng Tian. Multi-modal grouping network for weakly- supervised audio-visual video parsing. Advances in Neural Information Processing Systems, 35:34722–34733, 2022

2022
[14]

CM-PIE: Cross-modal perception for interactive-enhanced audio- visual video parsing

Yaru Chen, Ruohao Guo, Xubo Liu, Peipei Wu, Guangyao Li, Zhenbo Li, and Wenwu Wang. CM-PIE: Cross-modal perception for interactive-enhanced audio- visual video parsing. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , page...

2024
[15]

Advancing Weakly- Supervised Audio-Visual Video Parsing via Segment-Wise Pseudo Labeling

Jinxing Zhou, Dan Guo, Yiran Zhong, and Meng Wang. Advancing Weakly- Supervised Audio-Visual Video Parsing via Segment-Wise Pseudo Labeling. In- ternational Journal of Computer Vision , pages 1–22, 2024

2024
[16]

Resisting Noise in Pseudo Labels: Audible Video Event Parsing With Evidential Learning

Xun Jiang, Xing Xu, Liqing Zhu, Zhe Sun, Andrzej Cichocki, and Heng Tao Shen. Resisting Noise in Pseudo Labels: Audible Video Event Parsing With Evidential Learning. IEEE Transactions on Neural Networks and Learning Systems , 2024

2024
[17]

LINK: Adap- tive Modality Interaction for Audio-Visual Video Parsing

Langyu Wang, Bingke Zhu, Yingying Chen, and Jinqiao Wang. LINK: Adap- tive Modality Interaction for Audio-Visual Video Parsing. arXiv preprint arXiv:2412.20872, 2024

2024 arXiv
[18]

Multilayer perceptron (MLP)

Hind Taud and Jean-Franccois Mas. Multilayer perceptron (MLP). In Geomatic approaches for modeling land change scenarios , pages 451–455. Springer, 2017

2017
[19]

Graph Attention Networks

Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph Attention Networks. In International Conference on Learning Representations, 2018

2018
[20]

Exploring heterogeneous clues for weakly-supervised audio- visual video parsing

Yu Wu and Yi Yang. Exploring heterogeneous clues for weakly-supervised audio- visual video parsing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1326–1335, 2021

2021

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

Unified multisensory perception: Weakly-supervised audio-visual video parsing

Yapeng Tian, Dingzeyu Li, and Chenliang Xu. Unified multisensory perception: Weakly-supervised audio-visual video parsing. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16, pages 436–454. Springer, 2020

work page 2020

[2] [2]

Anchor-aware Deep Metric Learning for Audio-visual Retrieval

Donghuo Zeng, Yanan Wang, Kazushi Ikeda, and Yi Yu. Anchor-aware Deep Metric Learning for Audio-visual Retrieval. InProceedings of the 2024 International Conference on Multimedia Retrieval , pages 211–219, 2024

work page 2024

[3] [3]

Boosting Audio Visual Question Answer- ing via Key Semantic-Aware Cues

Guangyao Li, Henghui Du, and Di Hu. Boosting Audio Visual Question Answer- ing via Key Semantic-Aware Cues. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 5997–6005, 2024

work page 2024

[4] [4]

Open-Vocabulary Audio-Visual Semantic Segmentation

Ruohao Guo, Liao Qu, Dantong Niu, Yanyu Qi, Wenzhen Yue, Ji Shi, Bowei Xing, and Xianghua Ying. Open-Vocabulary Audio-Visual Semantic Segmentation. In Proceedings of the 32nd ACM International Conference on Multimedia , pages 7533–7541, 2024

work page 2024

[5] [5]

Drcnet: Dynamic image restoration contrastive network

Fei Li, Lingfeng Shen, Yang Mi, and Zhenbo Li. Drcnet: Dynamic image restoration contrastive network. In European Conference on Computer Vision , pages 514–532. Springer, 2022

work page 2022

[6] [6]

Collecting cross-modal presence-absence evidence for weakly-supervised audio-visual event percep- tion

Junyu Gao, Mengyuan Chen, and Changsheng Xu. Collecting cross-modal presence-absence evidence for weakly-supervised audio-visual event percep- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18827–18836, 2023

work page 2023

[7] [7]

ColeaF: A Contrastive-Collaborative Learning Framework for Weakly Supervised Audio- Visual Video Parsing

Faegheh Sardari, Armin Mustafa, Philip JB Jackson, and Adrian Hilton. ColeaF: A Contrastive-Collaborative Learning Framework for Weakly Supervised Audio- Visual Video Parsing. In European Conference on Computer Vision , pages 1–17. Springer, 2024

work page 2024

[8] [8]

Modality-independent teachers meet weakly-supervised audio-visual event parser

Yung-Hsuan Lai, Yen-Chun Chen, and Frank Wang. Modality-independent teachers meet weakly-supervised audio-visual event parser. Advances in Neural Information Processing systems, 36:73633–73651, 2023

work page 2023