Pith. sign in

REVIEW 4 major objections 5 minor 64 references

DEL: Dense Event Localization for Multi-modal Audio-Visual Understanding

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DEL, a masked-attention audio-visual framework, reports state-of-the-art average mAP on UnAV-100, THUMOS14, ActivityNet 1.3, and EPIC-Kitchens-100.

desk verdict A useful integration of known audio-visual fusion techniques with consistent benchmark gains, but the headline SOTA claim is undercut by test-set-tuned hyperparameters and missing variance; worth refereeing if the authors fix the protocol. read the letter →

arxiv 2506.23196 v1 pith:GDNUGN44 submitted 2025-06-29 cs.CV

Mona Ahmadian , Amir Shirian , Frank Guerin , Andrew Gilbert This is my paper
classification cs.CV
keywords temporalactionlocalizationaudio-visualeventdensemultimodalfusioncontrastivelearningfeaturepyramidnetworkuntrimmedvideoadaptiveattention
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes DEL, a dense audio-visual event localization framework for untrimmed videos. It aims to show that jointly modeling audio and visual streams, rather than fusing them late, lets a single-stage model localize and classify events at fine temporal resolution. On four benchmarks the paper reports higher average mAP than previous methods, with gains of 3.3 points on UnAV-100, 2.6 on THUMOS14, 1.2 on ActivityNet 1.3, and 1.7 on verbs plus 1.4 on nouns for EPIC-Kitchens-100. The gains are attributed to masked adaptive attention for cross-modal alignment, score-based contrastive learning with dynamically selected hard negatives, and a path aggregation network that fuses multiple temporal scales. If correct, the paper shows that audio is not a side channel for temporal action localization but a load-bearing cue that should be aligned with vision throughout the network.

What carries the argument

The central mechanism is a masked adaptive attention matrix that governs audio-visual alignment: entries within a modality are set to 1, while cross-modality entries are active only for corresponding temporal segments, so audio and video features attend to each other when they refer to the same moment. Around this, the framework adds score-based contrastive learning, where a binary event-score predictor and a category predictor select positive and hard-negative pairs automatically during training, and a path aggregation network with max-sigmoid modality-guided adapters and adaptive pooling that propagates fine-grained and high-level cues across six temporal scales. Together these modules aim to keep temporal structure intact while letting each modality sharpen the other.

What would settle it

On UnAV-100, check whether any pair of ground-truth events whose centers fall within the same temporal segment ever both appear in DEL's output; if no such concurrent pair is ever localized, the central claim about handling overlapping events in a shared moment is not supported.

Watch

Extended reading notes

Core claim

DEL claims that dense localization of audio-visual events in untrimmed videos improves when the audio and visual streams are aligned early and fused at multiple temporal resolutions, and it demonstrates this with a transformer-based single-stage detector. The model tokenizes video with I3D or SlowFast and audio with VGGish, applies masked adaptive attention so each modality attends globally within itself but only to the aligned temporal segment of the other modality, trains with inter- and intra-sample contrastive losses whose positive and hard-negative pairs are selected by a learned score and category predictor, and refines features through a six-level path aggregation network with modality-guided adapters. On UnAV-100, THUMOS14, ActivityNet 1.3, and EPIC-Kitchens-100, using the same pre-extracted features as the baselines, DEL reports higher average mAP in every setting, with the largest margins at high temporal IoU thresholds. The paper reads the consistency of these gains as evidence that adaptive cross-modal alignment plus multi-scale fusion is the right recipe for dense, overlapping events.

Load-bearing premise

The load-bearing premise is that every time segment can be represented by a single action class: predictions are made by picking the most likely class per step, so the model cannot output two overlapping actions at the same moment even though overlapping events are a stated motivation.

Editorial extensions

If this is right

  • Audio-visual temporal action localization should align modalities at the token level rather than waiting for late fusion, since DEL's gains come from early masked attention and multi-scale fusion.
  • The largest improvements appear at high tIoU thresholds, such as 60.5 versus 47.4 at 0.7 on THUMOS14, so the practical advantage is precise boundary refinement, not just coarse event detection.
  • Score-based contrastive pair selection shows that hard-negative mining can be learned from token-level predictions instead of relying on manual sampling heuristics.
  • Replacing I3D/Video features with DINOv2 and VGGish audio with MERT improves results further, indicating DEL is complementary to stronger feature extractors.
  • The gap between audio-only and visual-only variants, 40.6 and 37.9 versus 51.1 average mAP on UnAV-100, implies that fusion contributes substantially beyond either single modality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A test not in the paper: because predictions take the argmax over classes at every time step, DEL cannot emit two action labels for the same moment; measuring whether any concurrent ground-truth pair is ever jointly recovered would directly test the paper's overlapping-events motivation.
  • The score-based contrastive selection scheme could transfer to other weakly supervised or dense prediction tasks, but that transfer is an open question rather than a claim of this paper.
  • The DINOv2/MERT experiment hints that feature quality may drive part of the gains, so an ablation holding fusion fixed while varying features would clarify how much of DEL's improvement comes from the fusion machinery itself.
  • Because verb gains on EPIC-Kitchens-100 exceed noun gains, audio-visual alignment may help action labels more than object labels, which could guide where future multimodal fusion efforts are spent.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces DEL, a transformer-based framework for dense audio-visual temporal action localization. The architecture combines an adaptive masked-attention mechanism for cross-modal alignment, score-based inter- and intra-sample contrastive learning, and a path aggregation network for multi-scale feature fusion. The authors report state-of-the-art average mAP on THUMOS14, ActivityNet-1.3, EPIC-Kitchens-100, and UnAV-100, with claimed gains of +2.6%, +1.2%, +1.7%/+1.4%, and +3.3% over previous methods. The main empirical claims rest on comparisons with the same feature extractors and on ablations of components, pyramid depth, feature encoders, and input modalities.

Significance. If the empirical results are reproducible and the protocol is clean, DEL would be a useful contribution to multimodal temporal action localization: it addresses a genuinely under-studied setting, compares against strong baselines with matched features, and provides ablations of each proposed module. The qualitative examples also illustrate a plausible benefit of audio-visual fusion for disambiguating visually similar events. However, the current manuscript contains load-bearing protocol problems: the pyramid depth is selected on a test set, the UnAV-100 average-mAP column is arithmetically inconsistent with the displayed thresholds, and the single-label-per-segment output formulation conflicts with the paper's emphasis on overlapping events. These issues must be resolved before the state-of-the-art claims can be accepted.

major comments (4)
  1. [Sec. 4.2, Table 6 vs. Table 4] The pyramid depth L is selected by ablating on the UnAV-100 test set. Table 4 is explicitly labeled "UnAV-100 test set," and Table 6 reports the same split: the L=6 row in Table 6 is identical to the DEL row in Table 4 (Avg 51.1). Thus the headline +3.3% gain on UnAV-100, and the fixed L=6 subsequently applied to THUMOS14, ActivityNet-1.3, and EPIC-Kitchens-100, all inherit a hyperparameter choice made by peeking at test labels. The non-monotonic behavior (L=4: 47.2, L=6: 51.1, L=7: 49.0) is consistent with noise or overfitting. The authors should choose L on a held-out validation split and either re-report all benchmarks with that L or provide a sensitivity analysis showing that the conclusions do not depend on the choice. They should also report variance across the five runs they state they averaged.
  2. [Tables 4-6] The Avg column in the UnAV-100 tables is not the mean of the displayed thresholds. For the DEL row in Table 4, the values 53.4, 48.1, 42.6, 35.6, 26.9 have arithmetic mean 41.3, not the reported 51.1; the same discrepancy appears for every row in Tables 4, 5, and 6. The supplementary text (Sec. 6.2) says the method computes mAP at tIoU thresholds from 0.5 to 0.9 but reports average mAP over 0.1 to 0.9, which is impossible without also computing thresholds 0.1-0.4. Because the abstract's +3.3% UnAV-100 gain is based on this Avg, the headline result is not verifiable from the reported numbers. The authors must either list all thresholds used to compute the average or redefine Avg consistently with the columns shown.
  3. [Sec. 3.1, Eq. (4)] The output representation assigns exactly one class per temporal segment through argmax over classes. This cannot jointly represent two events of different classes that occur in the same segment, yet the abstract and introduction repeatedly motivate the method with overlapping, concurrent events, and the supplementary notes that about 25% of UnAV-100 videos contain concurrent events. The paper never states or tests the single-label-per-segment assumption, and the limitations section does not acknowledge it. The authors should clarify how overlapping events are handled under this formulation, or provide an explicit analysis of the model's behavior on concurrent-event segments; otherwise the claim about dense overlap handling is unsupported.
  4. [Sec. 4.1, Table 1] The text says DEL "consistently improves" across IoU thresholds, but on THUMOS14 DEL is below TriDet at tIoU 0.3 (81.0 vs. 83.6) and 0.4 (78.0 vs. 80.1) and only surpasses it from 0.5 upward. The statement should be qualified as a relative gain concentrated at higher IoU thresholds, which is itself an interesting result but should be described accurately. This comment is secondary to the protocol issues above.
minor comments (5)
  1. [Sec. 3.1] There is a typo in the sentence defining ground-truth annotations: "represents the the event class" should be "represents the event class."
  2. [Sec. 3.3, Eq. (7)] The expectation notation in Eq. (7) is difficult to parse; the subscripted argument of E is ambiguous and should be rewritten with a clear definition of the distribution being averaged over.
  3. [Sec. 6.2 (supplementary)] The sentence saying mAP is computed at 0.5-0.9 but that the average is over 0.1-0.9 is logically contradictory; if the UnAV-100 benchmark convention reports an average over 0.1-0.9, the main paper should either show all nine thresholds or cite the convention explicitly.
  4. [Sec. 4.1] The sentence "Across all benchmarks, DEL consistently improves as the IoU threshold increases" is misleading because mAP values naturally decrease as the threshold increases; the intended meaning appears to be that the relative gain over baselines grows at higher thresholds.
  5. [Sec. 4.3, Fig. 4] The qualitative figure is informative but the event labels are small and partly truncated in some rows; the authors should ensure all text is legible or enlarge the figure.

Circularity Check

1 steps flagged · score 6.0 of 10

The UnAV-100 headline gain is selected, not predicted: pyramid depth L is chosen by ablation on the same UnAV-100 test set that is later reported as the +3.3% SOTA, and the fixed L=6 is then inherited by all other benchmarks.

  1. fitted input called prediction [Sec. 4.2 (Table 6) selecting L, reported in Sec. 4.1 (Table 4) on the UnAV-100 test set]
    "Finally, on UnA V-100, Tab. 4, a dataset featuring complex multi-event scenarios and significant audio-visual overlap, DEL achieves a state-of-the-art average mAP of 51.1%, outperforming previous methods for 3.3%. ... Table 6: Ablation study on the design of the feature pyramid. L shows the number of layers for both audio and video. ... 6 ... 53.4 48.1 42.6 35.6 26.9 51.1. [Tab. 4 caption:] Performance on the UnA V-100 test set."

    The pyramid depth L is selected by maximizing Avg mAP on the UnAV-100 split that is explicitly labeled the test set in Table 4. Table 6 shows the non-monotonic sweep and crowns L=6 with 51.1 Avg mAP; that exact number is then presented in Section 4.1 as the state-of-the-art +3.3% improvement on UnAV-100. The reported UnAV-100 result is therefore the maximum of a test-set search, i.e., a fitted choice renamed as an independent prediction, not a held-out evaluation of a fixed configuration. The same test-selected L=6 is subsequently used for THUMOS14, ActivityNet 1.3, and EPIC-Kitchens-100, so those headline gains inherit a hyperparameter chosen on another dataset's test labels. No validation split or error bars accompany the ablation, so the +3.3% margin is not independently verified.

full rationale

The paper's central claim is an empirical performance comparison on public benchmarks, which is usually externally grounded; the architecture components (adaptive attention, score-based contrastive loss, path aggregation) are not derived by fitting constants to the reported metrics. The only self-citation, MRAV-FF [11], appears as a baseline and in the limitations paragraph and is not load-bearing. However, the pyramid-depth ablation is performed on the UnAV-100 test set (Table 4 labels the split "test set"), and the best L=6 is exactly the configuration whose 51.1 Avg mAP is reported as the +3.3% SOTA. This turns the headline UnAV-100 result into a selected maximum rather than an independent evaluation, and the fixed L=6 propagates to the other benchmarks. I therefore flag one fitted-input-called-prediction step; the remaining evaluation is not circular.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The framework introduces no new physical or mathematical entities. Its assumptions are standard domain assumptions about feature quality and label completeness, plus one structural design choice: the single-label-per-segment output, which is in tension with the paper's dense-overlap motivation. The main free parameters are the loss weights and the number of pyramid levels, neither of which is given a principled derivation.

free parameters (3)
  • loss weights λ1-λ5 = unspecified
    Eq. (14) defines the total loss as a weighted sum of five terms; the paper states values are set to balance terms but never reports the actual numbers. These weights are tuned and directly affect the final performance.
  • number of pyramid levels L = 6
    Selected via ablation on UnAV-100 (Table 6); performance ranges from 45.5 (L=1) to 51.1 (L=6) and drops at L=7. This single hyperparameter is a major determinant of the reported gains.
  • contrastive temperature τ = learnable
    τ in Eq. (8) is described as a learnable temperature for the contrastive loss, so it is optimized during training rather than fixed by theory.
assumptions (4)
  • domain assumption Audio and visual features are segment-wise aligned after preprocessing, so cross-modal attention can be restricted to the same temporal segment.
    The adaptive attention mask in Sec. 3.2 sets cross-modal entries to 1 only for the same temporal segment; this assumes the VGGish and I3D/SlowFast feature streams are synchronized at the segment level.
  • domain assumption Pre-extracted features from frozen I3D, VGGish, and SlowFast backbones are sufficient for fair comparison with prior methods.
    The method relies on fixed features, as acknowledged in Sec. 7.3; the SOTA claims assume all compared methods use equivalent feature quality.
  • domain assumption Ground-truth event annotations on the four benchmarks are complete and correct, including for concurrent events.
    Training and evaluation depend on these annotations; any missing or noisy labels affect the contrastive score function (Eq. 9) and the mAP metrics.
  • ad hoc to paper A single predicted label per temporal segment is sufficient to represent the event structure.
    Eq. (4) uses argmax over classes for each segment, so the model cannot output multiple simultaneous events at the same time. The claim of handling overlapping events relies on this representational limitation not being triggered in practice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DEL: Dense Event Localization for Multi-modal Audio-Visual Understanding." pith.science (2026). https://pith.science/paper/GDNUGN44

@misc{pith2026250623196,
  author       = {Pith},
  title        = {Pith review of: DEL: Dense Event Localization for Multi-modal Audio-Visual Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GDNUGN44}},
  note         = {Machine review of arXiv:2506.23196}
}
read the original abstract

Real-world videos often contain overlapping events and complex temporal dependencies, making multimodal interaction modeling particularly challenging. We introduce DEL, a framework for dense semantic action localization, aiming to accurately detect and classify multiple actions at fine-grained temporal resolutions in long untrimmed videos. DEL consists of two key modules: the alignment of audio and visual features that leverage masked self-attention to enhance intra-mode consistency and a multimodal interaction refinement module that models cross-modal dependencies across multiple scales, enabling high-level semantics and fine-grained details. Our method achieves state-of-the-art performance on multiple real-world Temporal Action Localization (TAL) datasets, UnAV-100, THUMOS14, ActivityNet 1.3, and EPIC-Kitchens-100, surpassing previous approaches with notable average mAP gains of +3.3%, +2.6%, +1.2%, +1.7% (verb), and +1.4% (noun), respectively.

Figures

Figures reproduced from arXiv: 2506.23196 by the authors.

Figure 1
Figure 1. Real-world videos contain overlapping events of vary [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our proposed DEL framework. Our model integrates (1) an adaptive attention mechanism for aligning audio and visual features, (2) inter- and intra-sample contrastive learning to enhance event discrimination, and (3) a multi-scale path aggregation network for feature fusion. ∥ represents the concatenation operation. tion affects their ability to handle multi-scale variations and modality-specific character… view at source ↗
Figure 3
Figure 3. Score-based contrastive pair selection for identifying [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Qualitative results of our DEL framework for audio [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Illustration of the Path Aggregation Network for multi-scale feature fusion. The network employs a top-down and bottom [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Detailed diagram of the max sigmoid module. This [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

64 extracted references · 50 canonical work pages

  1. [1]

    Maas: Multi-modal assignation for active speaker detection

    Juan Le ´on Alc´azar, Fabian Caba, Ali K Thabet, and Bernard Ghanem. Maas: Multi-modal assignation for active speaker detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 265–274, 2021. 2

    work page 2021

  2. [2]

    Hear me out: Fusional ap- proaches for audio augmented temporal action localization

    Anurag Bagchi, Jazib Mahmood, Dolton Fernandes, and Ravi Kiran Sarvadevabhatla. Hear me out: Fusional ap- proaches for audio augmented temporal action localization. arXiv preprint arXiv:2106.14118, 2021. 2

    arXiv 2021

  3. [3]

    Activitynet: A large-scale video benchmark for human activity understanding

    Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem, and Juan Carlos Niebles. Activitynet: A large-scale video benchmark for human activity understanding. In Proceed- ings of the ieee conference on computer vision and pattern recognition, pages 961–970, 2015. 1, 2, 6

    work page 2015

  4. [4]

    Quo vadis, action recognition? a new model and the kinetics dataset

    Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. In pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6299–6308, 2017. 6, 1

    work page 2017

  5. [5]

    Augmented transformer with adaptive graph for tem- poral action proposal generation

    Shuning Chang, Pichao Wang, Fan Wang, Hao Li, and Zheng Shou. Augmented transformer with adaptive graph for tem- poral action proposal generation. In Proceedings of the 3rd International Workshop on Human-Centric Multimedia Analysis, pages 41–50, 2022. 2

    work page 2022

  6. [6]

    Re- thinking the faster r-cnn architecture for temporal action localization

    Yu-Wei Chao, Sudheendra Vijayanarasimhan, Bryan Sey- bold, David A Ross, Jia Deng, and Rahul Sukthankar. Re- thinking the faster r-cnn architecture for temporal action localization. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1130–1139,

    work page

  7. [7]

    Tallformer: Temporal ac- tion localization with a long-memory transformer

    Feng Cheng and Gedas Bertasius. Tallformer: Temporal ac- tion localization with a long-memory transformer. In Eu- ropean Conference on Computer Vision , pages 503–521. Springer, 2022. 2

    work page 2022

  8. [8]

    Yolo-world: Real-time open-vocabulary object detection

    Tianheng Cheng, Lin Song, Yixiao Ge, Wenyu Liu, Xing- gang Wang, and Ying Shan. Yolo-world: Real-time open-vocabulary object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16901–16911, 2024. 2, 5

    2024

Show all 64 references
  1. [9]

    Rescaling egocentric vision: Collection, pipeline and chal- lenges for epic-kitchens-100

    Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Antonino Furnari, Evangelos Kazakos, Jian Ma, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, et al. Rescaling egocentric vision: Collection, pipeline and chal- lenges for epic-kitchens-100. International Journa...

    2022

  2. [10]

    Slowfast networks for video recognition

    Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6202–6211, 2019. 6, 1

    2019

  3. [11]

    Multi- resolution audio-visual feature fusion for temporal action lo- calization

    Edward Fish, Jon Weinbren, and Andrew Gilbert. Multi- resolution audio-visual feature fusion for temporal action lo- calization. arXiv preprint arXiv:2310.03456, 2023. 5, 6, 3

    2023 arXiv

  4. [12]

    Audio set: An ontology and human- labeled dataset for audio events

    Jort F Gemmeke, Daniel PW Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R Channing Moore, Manoj Plakal, and Marvin Ritter. Audio set: An ontology and human- labeled dataset for audio events. In 2017 IEEE interna- tional conference on acoustics, speech and signal processin...

    2017

  5. [13]

    Dense-localizing audio-visual events in untrimmed videos: A large-scale benchmark and baseline

    Tiantian Geng, Teng Wang, Jinming Duan, Runmin Cong, and Feng Zheng. Dense-localizing audio-visual events in untrimmed videos: A large-scale benchmark and baseline. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22942–22951, 2023. 1...

    2023

  6. [14]

    Momentum contrast for unsupervised visual rep- resentation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual rep- resentation learning. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 9729–9738, 2020. 4

    2020

  7. [15]

    Cnn archi- tectures for large-scale audio classification

    Shawn Hershey, Sourish Chaudhuri, Daniel PW Ellis, Jort F Gemmeke, Aren Jansen, R Channing Moore, Manoj Plakal, Devin Platt, Rif A Saurous, Bryan Seybold, et al. Cnn archi- tectures for large-scale audio classification. In 2017 ieee in- ternational conference on acoustics, spe...

    2017

  8. [16]

    Mix and local- ize: Localizing sound sources in mixtures

    Xixi Hu, Ziyang Chen, and Andrew Owens. Mix and local- ize: Localizing sound sources in mixtures. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10483–10492, 2022. 2

    2022

  9. [17]

    in the wild

    Haroon Idrees, Amir R Zamir, Yu-Gang Jiang, Alex Gorban, Ivan Laptev, Rahul Sukthankar, and Mubarak Shah. The thumos challenge on action recognition for videos “in the wild”. Computer Vision and Image Understanding , 155:1– 23, 2017. 1, 2, 6

    2017

  10. [18]

    Causal inference meets deep learning: A compre- hensive survey

    Licheng Jiao, Yuhan Wang, Xu Liu, Lingling Li, Fang Liu, Wenping Ma, Yuwei Guo, Puhua Chen, Shuyuan Yang, and Biao Hou. Causal inference meets deep learning: A compre- hensive survey. Research, 7:0467, 2024. 1

    2024

  11. [19]

    Epic-fusion: Audio-visual temporal bind- ing for egocentric action recognition

    Evangelos Kazakos, Arsha Nagrani, Andrew Zisserman, and Dima Damen. Epic-fusion: Audio-visual temporal bind- ing for egocentric action recognition. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5492–5501, 2019. 2

    2019

  12. [20]

    Mert: Acoustic music un- derstanding model with large-scale self-supervised training

    Yizhi Li, Ruibin Yuan, Ge Zhang, Yinghao Ma, Xingran Chen, Hanzhi Yin, Chenghao Xiao, Chenghua Lin, Anton Ragni, Emmanouil Benetos, et al. Mert: Acoustic music un- derstanding model with large-scale self-supervised training. arXiv preprint arXiv:2306.00107, 2023. 7

    2023 arXiv

  13. [21]

    Learning salient boundary feature for anchor- free temporal action localization

    Chuming Lin, Chengming Xu, Donghao Luo, Yabiao Wang, Ying Tai, Chengjie Wang, Jilin Li, Feiyue Huang, and Yan- wei Fu. Learning salient boundary feature for anchor- free temporal action localization. In Proceedings of the IEEE/CVF conference on computer vision and pattern reco...

    2021

  14. [22]

    Bsn: Boundary sensitive network for temporal action proposal generation

    Tianwei Lin, Xu Zhao, Haisheng Su, Chongjing Wang, and Ming Yang. Bsn: Boundary sensitive network for temporal action proposal generation. In Proceedings of the European conference on computer vision (ECCV) , pages 3–19, 2018. 2, 1

    2018

  15. [23]

    Bmn: Boundary-matching network for temporal action pro- posal generation

    Tianwei Lin, Xiao Liu, Xin Li, Errui Ding, and Shilei Wen. Bmn: Boundary-matching network for temporal action pro- posal generation. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 3889–3898,

  16. [24]

    Progressive boundary refine- ment network for temporal action detection

    Qinying Liu and Zilei Wang. Progressive boundary refine- ment network for temporal action detection. In Proceed- ings of the AAAI conference on artificial intelligence , pages 11612–11619, 2020. 5, 6

    2020

  17. [25]

    Dense modality interaction network for audio-visual event localization

    Shuo Liu, Weize Quan, Chaoqun Wang, Yuan Liu, Bin Liu, and Dong-Ming Yan. Dense modality interaction network for audio-visual event localization. IEEE Transactions on Multimedia, 25:2734–2748, 2022. 2

    2022

  18. [26]

    Multi-shot temporal event localization: a benchmark

    Xiaolong Liu, Yao Hu, Song Bai, Fei Ding, Xiang Bai, and Philip HS Torr. Multi-shot temporal event localization: a benchmark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12596– 12606, 2021. 5, 6

    2021

  19. [27]

    End-to-end temporal ac- tion detection with transformer.IEEE Transactions on Image Processing, 31:5427–5441, 2022

    Xiaolong Liu, Qimeng Wang, Yao Hu, Xu Tang, Shiwei Zhang, Song Bai, and Xiang Bai. End-to-end temporal ac- tion detection with transformer.IEEE Transactions on Image Processing, 31:5427–5441, 2022. 5, 6

    2022

  20. [28]

    Gaussian temporal awareness networks for action localization

    Fuchen Long, Ting Yao, Zhaofan Qiu, Xinmei Tian, Jiebo Luo, and Tao Mei. Gaussian temporal awareness networks for action localization. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 344–353, 2019. 2

    2019

  21. [29]

    Proposal-free temporal action detection via global segmen- tation mask learning

    Sauradip Nag, Xiatian Zhu, Yi-Zhe Song, and Tao Xiang. Proposal-free temporal action detection via global segmen- tation mask learning. In European Conference on Computer Vision, pages 645–662. Springer, 2022. 2

    2022

  22. [30]

    Attention bottlenecks for multimodal fusion

    Arsha Nagrani, Shan Yang, Anurag Arnab, Aren Jansen, Cordelia Schmid, and Chen Sun. Attention bottlenecks for multimodal fusion. Advances in neural information process- ing systems, 34:14200–14213, 2021. 1

    2021

  23. [31]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 7

    2023 arXiv

  24. [32]

    Audio-visual scene analysis with self-supervised multisensory features

    Andrew Owens and Alexei A Efros. Audio-visual scene analysis with self-supervised multisensory features. In Pro- ceedings of the European conference on computer vision (ECCV), pages 631–648, 2018. 1

    2018

  25. [33]

    A review of deep learning techniques in audio event recognition (aer) applications

    Arjun Prashanth, SL Jayalakshmi, and R Vedhapriyavad- hana. A review of deep learning techniques in audio event recognition (aer) applications. Multimedia Tools and Appli- cations, 83(3):8129–8143, 2024. 1

    2024

  26. [34]

    Owl (observe, watch, listen): Audiovisual temporal context for localizing actions in ego- centric videos

    Merey Ramazanova, Victor Escorcia, Fabian Caba, Chen Zhao, and Bernard Ghanem. Owl (observe, watch, listen): Audiovisual temporal context for localizing actions in ego- centric videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages...

    2023

  27. [35]

    You only look once: Unified, real-time object de- tection

    Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object de- tection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 779–788, 2016. 2

    2016

  28. [36]

    Action sensitivity learning for temporal action localization

    Jiayi Shao, Xiaohan Wang, Ruijie Quan, Junjun Zheng, Jiang Yang, and Yi Yang. Action sensitivity learning for temporal action localization. InProceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 13457–13469,

  29. [37]

    Temporal action localization with enhanced instant discriminability

    Dingfeng Shi, Qiong Cao, Yujie Zhong, Shan An, Jian Cheng, Haogang Zhu, and Dacheng Tao. Temporal action localization with enhanced instant discriminability. arXiv preprint arXiv:2309.05590, 2023. 2

    2023 arXiv

  30. [38]

    Tridet: Temporal action detection with relative boundary modeling

    Dingfeng Shi, Yujie Zhong, Qiong Cao, Lin Ma, Jia Li, and Dacheng Tao. Tridet: Temporal action detection with relative boundary modeling. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 18857–18866, 2023. 5, 6

    2023

  31. [39]

    Re- laxed transformer decoders for direct action proposal gener- ation

    Jing Tan, Jiaqi Tang, Limin Wang, and Gangshan Wu. Re- laxed transformer decoders for direct action proposal gener- ation. In Proceedings of the IEEE/CVF international confer- ence on computer vision, pages 13526–13535, 2021. 2

    2021

  32. [40]

    Audio-visual event localization in unconstrained videos

    Yapeng Tian, Jing Shi, Bochen Li, Zhiyao Duan, and Chen- liang Xu. Audio-visual event localization in unconstrained videos. In Proceedings of the European conference on com- puter vision (ECCV), pages 247–263, 2018. 1, 2

    2018

  33. [41]

    Deep learning-based action detection in untrimmed videos: A survey.IEEE Transactions on Pattern Analysis and Machine Intelligence , 45(4):4302– 4320, 2022

    Elahe Vahdani and Yingli Tian. Deep learning-based action detection in untrimmed videos: A survey.IEEE Transactions on Pattern Analysis and Machine Intelligence , 45(4):4302– 4320, 2022. 1

    2022

  34. [42]

    You only hear once: a yolo-like algorithm for audio segmentation and sound event detection

    Satvik Venkatesh, David Moffat, and Eduardo Reck Mi- randa. You only hear once: a yolo-like algorithm for audio segmentation and sound event detection. Applied Sciences, 12(7):3293, 2022. 1

    2022

  35. [43]

    Temporal action proposal generation with transformers

    Lining Wang, Haosen Yang, Wenhao Wu, Hongxun Yao, and Hujie Huang. Temporal action proposal generation with transformers. arXiv preprint arXiv:2105.12043, 2021. 2

    2021 arXiv

  36. [44]

    Rcl: Recurrent continuous localization for temporal action detec- tion

    Qiang Wang, Yanhao Zhang, Yun Zheng, and Pan Pan. Rcl: Recurrent continuous localization for temporal action detec- tion. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 13566–13575,

  37. [45]

    Internvideo2: Scaling foundation models for mul- timodal video understanding

    Yi Wang, Kunchang Li, Xinhao Li, Jiashuo Yu, Yinan He, Guo Chen, Baoqi Pei, Rongkun Zheng, Zun Wang, Yansong Shi, et al. Internvideo2: Scaling foundation models for mul- timodal video understanding. In European Conference on Computer Vision, pages 396–416. Springer, 2024. 2

    2024

  38. [46]

    An efficient spatio-temporal pyramid transformer for action detection

    Yuetian Weng, Zizheng Pan, Mingfei Han, Xiaojun Chang, and Bohan Zhuang. An efficient spatio-temporal pyramid transformer for action detection. In European Conference on Computer Vision, pages 358–375. Springer, 2022. 2

    2022

  39. [47]

    Dual attention matching for audio-visual event localization

    Yu Wu, Linchao Zhu, Yan Yan, and Yi Yang. Dual attention matching for audio-visual event localization. In Proceedings of the IEEE/CVF international conference on computer vi- sion, pages 6292–6300, 2019. 2

    2019

  40. [48]

    Dual relation network for temporal action localization

    Kun Xia, Le Wang, Sanping Zhou, Gang Hua, and Wei Tang. Dual relation network for temporal action localization. Pat- tern Recognition, 129:108725, 2022. 2

    2022

  41. [49]

    Learning to refactor action and co-occurrence fea- tures for temporal action localization

    Kun Xia, Le Wang, Sanping Zhou, Nanning Zheng, and Wei Tang. Learning to refactor action and co-occurrence fea- tures for temporal action localization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13884–13893, 2022. 2

    2022

  42. [50]

    Audiovisual slowfast networks for video recognition

    Fanyi Xiao, Yong Jae Lee, Kristen Grauman, Jitendra Malik, and Christoph Feichtenhofer. Audiovisual slowfast networks for video recognition. arXiv preprint arXiv:2001.08740 ,

    2001 arXiv

  43. [51]

    G-tad: Sub-graph localization for tempo- ral action detection

    Mengmeng Xu, Chen Zhao, David S Rojas, Ali Thabet, and Bernard Ghanem. G-tad: Sub-graph localization for tempo- ral action detection. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 10156–10165, 2020. 2, 6

    2020

  44. [52]

    Audio-visual event localization by learning spatial and semantic co-attention

    Cheng Xue, Xionghu Zhong, Minjie Cai, Hao Chen, and Wenwu Wang. Audio-visual event localization by learning spatial and semantic co-attention. IEEE Transactions on Multimedia, 25:418–429, 2021. 2

    2021

  45. [53]

    Temporal pyramid network for action recognition

    Ceyuan Yang, Yinghao Xu, Jianping Shi, Bo Dai, and Bolei Zhou. Temporal pyramid network for action recognition. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 591–600, 2020. 2

    2020

  46. [54]

    Revisiting anchor mechanisms for temporal ac- tion localization

    Le Yang, Houwen Peng, Dingwen Zhang, Jianlong Fu, and Junwei Han. Revisiting anchor mechanisms for temporal ac- tion localization. IEEE Transactions on Image Processing , 29:8535–8548, 2020. 2, 5, 6

    2020

  47. [55]

    Mpn: Multimodal parallel network for audio-visual event localization

    Jiashuo Yu, Ying Cheng, and Rui Feng. Mpn: Multimodal parallel network for audio-visual event localization. In 2021 IEEE International Conference on Multimedia and Expo (ICME), pages 1–6. IEEE, 2021. 2

    2021

  48. [56]

    Mm-pyramid: Multimodal pyramid attentional network for audio-visual event localization and video pars- ing

    Jiashuo Yu, Ying Cheng, Rui-Wei Zhao, Rui Feng, and Yue- jie Zhang. Mm-pyramid: Multimodal pyramid attentional network for audio-visual event localization and video pars- ing. In Proceedings of the 30th ACM international confer- ence on multimedia, pages 6241–6249, 2022. 5

    2022

  49. [57]

    Graph con- volutional networks for temporal action localization

    Runhao Zeng, Wenbing Huang, Mingkui Tan, Yu Rong, Peilin Zhao, Junzhou Huang, and Chuang Gan. Graph con- volutional networks for temporal action localization. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 7094–7103, 2019. 2

    2019

  50. [58]

    Actionformer: Lo- calizing moments of actions with transformers

    Chen-Lin Zhang, Jianxin Wu, and Yin Li. Actionformer: Lo- calizing moments of actions with transformers. In European Conference on Computer Vision , pages 492–510. Springer,

  51. [59]

    Video self- stitching graph network for temporal action localization

    Chen Zhao, Ali K Thabet, and Bernard Ghanem. Video self- stitching graph network for temporal action localization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 13658–13667, 2021. 2, 6

    2021

  52. [60]

    Bottom-up temporal action localization with mutual regularization

    Peisen Zhao, Lingxi Xie, Chen Ju, Ya Zhang, Yanfeng Wang, and Qi Tian. Bottom-up temporal action localization with mutual regularization. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VIII 16 , pages 539–555. Springer,

    2020

  53. [61]

    Enriching local and global contexts for temporal action localization

    Zixin Zhu, Wei Tang, Le Wang, Nanning Zheng, and Gang Hua. Enriching local and global contexts for temporal action localization. In Proceedings of the IEEE/CVF international conference on computer vision , pages 13516–13525, 2021. 5, 6 DEL: Dense Event Localization for Multi-m...

    2021

  54. [62]

    Our code provides further information

    Experimental Details We present the implementation details, including the net- work architecture, training process, and inference strategy. Our code provides further information. 6.1. Evaluation Metric: Mean A verage Precision (mAP) In temporal action localization, mean Averag...

  55. [63]

    Performance was measured us- ing mAP@[0.5:0.05:0.95], along with the average mAP

    Following previous studies [22, 23, 58], features were downsampled to fixed lengths of 160 using linear inter- polation for I3D features. Performance was measured us- ing mAP@[0.5:0.05:0.95], along with the average mAP. The model was trained for 15 epochs with a 5-epoch lin- e...

  56. [64]

    This approach ensures greater feature consis- tency across different temporal resolutions, ultimately im- proving the regression of an event’s location

    Path Aggregation Network for Multi-Scale Feature Fusion We propose a path aggregation network to ensure informa- tion is aggregated across multiple temporal resolutions, pre- serving short-term event cues and long-term contextual de- pendencies. This approach ensures greater f...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.