Pith. sign in

REVIEW 6 major objections 5 minor 46 references

Attention-Driven Multimodal Alignment for Long-term Action Quality Assessment

T0 review · 6 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that forcing RGB, optical flow, and audio to attend to the same temporal segments is the key to long-term action quality assessment.

desk verdict Plausible idea, but the central loss is underspecified and the ablation numbers don't match the headline tables; as written, 'significantly outperforms' is not supported. read the letter →

arxiv 2507.21945 v1 pith:EOOTOXWM submitted 2025-07-29 cs.CV

Xin Wang , Peng-Jie Li , Yuan-Yuan Shen This is my paper
classification cs.CV
keywords actionqualityassessmentmultimodallearningattentionconsistencytemporalalignmentrhythmicgymnasticsfigureskatingtransformerdecoderscoreinterpretability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that long-term action quality assessment in artistic sports improves when the model is explicitly forced to align what each modality attends to in time, rather than fusing video, motion, and audio features only at a global level. It proposes LMAC-Net, which runs three independent transformer-decoder branches over RGB, optical flow, and audio segments and penalizes differences between their attention centers so the modalities focus on the same key moments. A two-level scoring module first scores each temporal query and then learns weights to fuse them into a final score. On the rhythmic gymnastics (RG) and figure skating (Fis-V) benchmarks, the method reports average Spearman correlations of 0.840 and 0.850, higher than the unimodal and multimodal baselines it compares against.

What carries the argument

The load-bearing object is the attention center: for each of the K learnable queries in a modality branch, the weighted average temporal position of its attention weights across the T segments. LMAC-Net computes one attention center per query per modality and defines a multimodal consistency loss that minimizes the squared distance between the centers of different modalities at each time step, thereby pushing RGB, optical flow, and audio to concentrate on the same key moments. The same centers feed a ranking loss that enforces temporal order across queries and a sparsity loss that concentrates each query's attention, and the final fused features are scored by a two-level module that regresses a score per query and then adaptively weights the query scores.

What would settle it

Take a set of RG or Fis-V training videos and retrain LMAC-Net after shifting the audio track by a fixed offset of one or two 32-frame segments relative to the video. If the attention-consistency loss is genuinely exploiting music-to-motion correspondence, average Spearman correlation should drop relative to the aligned version; if it stays flat, the reported gains come from the feature-level losses or the two-level scoring rather than from cross-modal temporal alignment.

Watch

Extended reading notes

Core claim

The central claim is that temporal alignment of attention across modalities is the missing ingredient for multimodal AQA: when RGB, optical flow, and audio queries are made to center their attention on the same segments of a long performance, the fused representation scores better than either unimodal temporal transformers or multimodal methods that align globally. The paper argues this through an attention-center consistency loss that minimizes squared distances between modality attention centers, supported by ranking and sparsity losses that keep each query focused and ordered, and demonstrates it with average Spearman correlations of 0.840 on RG and 0.850 on Fis-V, including gains from adding audio to RGB plus flow of 0.047 and 0.096 on the two datasets.

Load-bearing premise

The method assumes that cutting RGB, optical flow, and audio into the same non-overlapping 32-frame segments yields temporally matching units, so that a musical accent and the corresponding visual motion are placed in the same segment; if the audio is offset from the video frame grid or a musical phrase straddles a boundary, the consistency loss will pull modality attention centers toward the wrong points and can hurt scoring.

Editorial extensions

If this is right

  • If the central claim holds, multimodal video scoring should be built around temporal alignment of attention, not just feature contrast or global fusion.
  • Audio, though nearly useless alone (0.315 average on RG), becomes a measurable supplement: adding it to RGB plus optical flow raises average Spearman correlation from 0.793 to 0.840 on RG and from 0.754 to 0.850 on Fis-V.
  • Scoring each temporal query first and then weighting query scores beats one-stage global regression on these benchmarks, giving interpretable per-segment scores along with better accuracy.
  • The alignment mechanism adds little compute: 8.95M parameters, 0.419G FLOPs, and 4 ms inference, roughly in line with unimodal methods and well below the PAMFN baseline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to apply the same attention-center consistency loss to other long-form multimodal tasks where sound and motion are coupled, such as dance evaluation, surgical procedure assessment, or music-conditioned movement analysis; the predicted pattern is that gains track how tightly the audio and visual streams are beat-aligned.
  • The consistency loss is defined on attention positions rather than feature embeddings, so it may be more robust to mismatched feature spaces across modalities, but it inherits sensitivity to the segment grid; testing finer or music-beat-aligned segmentation would reveal whether 32-frame chunks are the right granularity for the alignment signal.
  • An offset-augmentation experiment would sharpen the claim: if training with random small audio-video shifts degrades or slows alignment, that would confirm the loss is genuinely exploiting temporal correspondence rather than the score supervision indirectly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 5 minor

Summary. The paper proposes LMAC-Net, a multimodal action quality assessment (AQA) model for long-term artistic sports that uses RGB, optical flow, and audio. The method employs modality-specific transformer decoders with learnable queries, an attention-center consistency loss meant to align queries across modalities, and a two-level score evaluation module that first regresses per-query scores and then fuses them into a global score. Experiments on the RG and Fis-V datasets report average Spearman correlations of 0.840 and 0.850, which the paper claims significantly outperform prior unimodal and multimodal baselines. Ablation studies examine the contribution of the query encoder, the various feature-level losses, fusion strategies, modality combinations, score-fusion strategies, and alignment strategies; qualitative figures show training-time changes in attention-center positions and similarities.

Significance. The problem is relevant: long-term AQA in rhythmic gymnastics and figure skating requires joint modeling of visual motion and music, and existing multimodal AQA methods mostly rely on global fusion. The proposed direction—per-modality learnable queries plus an explicit consistency constraint on attention centers—is a plausible and potentially interpretable way to induce cross-modal temporal alignment. The paper provides a broad set of ablations (fusion strategies, modality contributions, two-level scoring, alignment losses) and also reports parameter and FLOP comparisons, which are useful. If the method and its results hold after the technical issues below are fixed, it would be a reasonable step beyond PAMFN and related work. However, the central loss term is not reproducibly defined, the attention equation is internally inconsistent, and the headline performance gain is presented without uncertainty quantification. No code or pretrained models are provided, which further limits reproducibility. The core idea is promising, but the evidence as presented does not yet support the paper's strongest claims.

major comments (6)
  1. [Section 3.4, Eq. (17)] The multimodal consistency loss is not well-defined. Eq. (14) defines \bar{\alpha}^m_k as a scalar attention center for each query k, obtained by summing t·α_{k,t} over all time steps. Eq. (17) instead sums over t and compares quantities \bar{\alpha}^{m_i}_t and \bar{\alpha}^{m_j}_t, which are never defined anywhere in the paper. A reader cannot implement this loss as written. The natural fix is to sum over queries k (and pairs of modalities) of (\bar{\alpha}^{m_i}_k − \bar{\alpha}^{m_j}_k)^2, or to introduce a separate per-time-step center definition. This is load-bearing, since the claimed contribution is precisely the alignment mechanism.
  2. [Section 3.2, Eqs. (5)–(7)] The key projection k^{(m,i)}_t is computed in Eq. (5) but never used: Eq. (6) computes attention weights from \tilde{q}^{(m,i)}_k and v^{(m,i)}_t, and Eq. (7) updates features with those weights and v. Either Eq. (6) should use k^{(m,i)}_t (with an inner product, as usual) or the key projection should be removed. As written, the described architecture is internally inconsistent and not reproducible.
  3. [Section 4.4, Tables 2 and 3] The claim that LMAC-Net “significantly outperforms existing methods” is not backed by uncertainty quantification. The reported average Spearman gains over PAMFN are 0.021 on RG and 0.028 on Fis-V, computed on test sets of 200 and 100 videos, with no confidence intervals, significance tests, or multiple-seed runs reported. The word “significantly” should be removed or supported by appropriate statistical testing, especially because on the Clubs category PAMFN actually scores higher (0.825 vs. 0.806).
  4. [Section 4.5, Tables 5, 8 vs. Tables 2, 3] The ablation table labeled “Ours” (Table 5) reports RG 0.797 and Fis-V 0.808, while the headline tables report 0.840 and 0.850. The numbers in Table 8’s one-stage linear row (RG 0.797, Fis-V 0.808) and its two-level “Weight” row (0.840, 0.850) suggest the difference is the two-level score evaluation module, but the text in Section 4.5 never states this. Please state explicitly which configuration produced the headline results, so that the ablation and main tables can be reconciled.
  5. [Section 4.6, Figures 5–8] The qualitative alignment evidence is circular. These figures show that attention-center distances decrease and cosine similarities increase during training on training samples; the consistency loss directly minimizes those very quantities, so convergence of the plots is a restatement of the optimization objective, not evidence that the model has learned meaningful cross-modal alignment. Either evaluate alignment on held-out samples against a ground-truth alignment signal (e.g., annotated beat/action correspondence) or temper the claims in Section 4.6.
  6. [Section 4.4, baseline comparison] The text says that comparison results for other multimodal methods were “directly cited from prior work [9]” while also asserting that all methods were evaluated under identical experimental settings; these statements are contradictory. If the Joint-VA, MSAF, UMT, and PAMFN numbers are copied from [9], the comparison inherits [9]’s protocol rather than the present setting; if they were re-implemented, state so and provide the re-implementation details. This matters because the PAMFN baseline defines the main comparison.
minor comments (5)
  1. [Section 3.4, Eq. (14)] The constraint “α^m_{k,t} = 1” should be “Σ_t α^m_{k,t} = 1”; the current phrasing is a typo that obscures the normalization.
  2. [Section 3.2, Eq. (7)] Eq. (7) uses a^{(m,i)}_{k,j} while Eq. (6) uses α^{(m,i)}_{k,t}; please unify the notation for attention weights.
  3. [Section 3.4 and Section 2] There are several typos: the heading “Feature-Level Loss d(Lfeature)” contains a stray “d”; “subfileds” should be “subfields” in Section 2; “Section 4.3 describe” should be “describes.”
  4. [Section 4.6, Score Interpretability] The text says the query scores and weights are “as shown in Figure 9,” but Figure 9 is the cross-attention weight heatmap; the scores and weights appear in Figure 10. Please fix the cross-reference.
  5. [References] References [38]/[44], [39]/[46], and [40]/[47] are duplicated entries for the same papers; consolidate them.

Circularity Check

1 steps flagged · score 2.0 of 10

One minor circularity in the qualitative alignment evidence; the benchmark claim is independently grounded.

  1. self definitional [Section 4.6 (Qualitative Analysis), Eq. (17), Figures 5-8]
    "Lconsistency = Σ_{t=1}^T Σ_{1≤i<j≤M} (ᾱ^{m_i}_t − ᾱ^{m_j}_t)^2 (Eq. 17) ... In contrast, after introducing our multimodal consistency method (Figures 5(c)-(d)), the attention centers of different modalities gradually become highly consistent. ... The area charts depict the trends and magnitudes of the absolute distances between attention centers of different modalities over training epochs. Subfigure (a) shows results without the proposed attention-based modality alignment method, while subfigure (b) shows results with the method applied."

    Eq. (17) defines the training objective as the squared distance between attention centers of different modality pairs. Figures 5-8 then present, as evidence of the mechanism's benefit, that these same attention-center distances shrink when the loss is enabled. This reduction is the objective function being minimized during training, so observing it after optimization is true by construction. It does not independently establish that such convergence causes better score prediction. However, the central performance claim rests on held-out Spearman correlations against external baselines on RG and Fis-V and on loss ablations, so this circularity is confined to the qualitative 'alignment effect' evidence and does not make the main result circular.

full rationale

The paper's load-bearing claim, that LMAC-Net significantly outperforms existing methods for long-term AQA, is tested against external benchmarks (RG and Fis-V) with held-out splits, and the consistency loss is ablated in Table 5. No load-bearing self-citation chain appears in the reference list; the attention-center mechanism is explicitly attributed to prior non-self work [42]. The only identified circular step is the qualitative analysis in Section 4.6, which uses the quantity directly minimized by Eq. (17) as evidence that the proposed alignment is effective. That is a self-definitional confirmation of the loss's effect, but it does not by itself support the score-based claim. Separate issues, such as the discrepancy between the Table 5 'Ours' values (0.797/0.808) and the Tables 2-3 headline values (0.840/0.850), and the index mismatch in Eq. (17), are correctness and reporting concerns rather than circularity.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on several hand-chosen hyperparameters (loss weights, ranking margin, per-modality weights, temperature, query count) and on domain assumptions about temporal synchronization and the meaning of attention centers. No new physical or conceptual entities are introduced.

free parameters (6)
  • Loss weights lambda1, lambda2 = not disclosed
    Control the tradeoff between score MSE and feature-level losses in Eq (11); no values or tuning procedure reported.
  • Ranking margin d = not disclosed
    Boundary margin in the multimodal ranking loss (Eq 15), chosen by hand.
  • Per-modality loss weights lambda^m_rank, lambda^m_sparsity = not disclosed
    Balance ranking and sparsity losses across RGB, flow, and audio (Eqs 15-16).
  • Attention temperature tau = initialized to 0.07, learnable
    Scales the query-value dot product in Eq (6).
  • Score fusion weight vector w = learned, dimension K=5
    Softmax-normalized weights in the two-level score module (Eqs 9-10).
  • Number of queries K = 5
    Chosen for both datasets; no sensitivity analysis reported (Section 4.3).
assumptions (4)
  • domain assumption Attention centers, defined as weighted mean positions of attention weights, are a sufficient summary of each query's temporal focus.
    The ranking, sparsity, and consistency losses (Eqs 14-17) all operate on these centers; if the attention distribution is multi-peaked, the mean position can misrepresent the focus.
  • domain assumption RGB, optical flow, and audio streams are temporally synchronized by dividing them into identical non-overlapping segments.
    Section 4.3 segments all modalities into equal 32-frame chunks, assuming musical accents and corresponding motions occur in the same segment.
  • ad hoc to paper Queries attend to temporally ordered and roughly distinct action segments.
    The ranking loss (Eq 15) enforces a strict temporal order on attention centers, an imposed prior not derived from the data.
  • standard math Standard transformer cross-attention, softmax, and MSE regression are valid building blocks.
    Adopted without proof from prior AQA work; used throughout the encoder and score modules.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Attention-Driven Multimodal Alignment for Long-term Action Quality Assessment." pith.science (2026). https://pith.science/paper/EOOTOXWM

@misc{pith2026250721945,
  author       = {Pith},
  title        = {Pith review of: Attention-Driven Multimodal Alignment for Long-term Action Quality Assessment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EOOTOXWM}},
  note         = {Machine review of arXiv:2507.21945}
}
read the original abstract

Long-term action quality assessment (AQA) focuses on evaluating the quality of human activities in videos lasting up to several minutes. This task plays an important role in the automated evaluation of artistic sports such as rhythmic gymnastics and figure skating, where both accurate motion execution and temporal synchronization with background music are essential for performance assessment. However, existing methods predominantly fall into two categories: unimodal approaches that rely solely on visual features, which are inadequate for modeling multimodal cues like music; and multimodal approaches that typically employ simple feature-level contrastive fusion, overlooking deep cross-modal collaboration and temporal dynamics. As a result, they struggle to capture complex interactions between modalities and fail to accurately track critical performance changes throughout extended sequences. To address these challenges, we propose the Long-term Multimodal Attention Consistency Network (LMAC-Net). LMAC-Net introduces a multimodal attention consistency mechanism to explicitly align multimodal features, enabling stable integration of visual and audio information and enhancing feature representations. Specifically, we introduce a multimodal local query encoder module to capture temporal semantics and cross-modal relations, and use a two-level score evaluation for interpretable results. In addition, attention-based and regression-based losses are applied to jointly optimize multimodal alignment and score fusion. Experiments conducted on the RG and Fis-V datasets demonstrate that LMAC-Net significantly outperforms existing methods, validating the effectiveness of our proposed approach.

Figures

Figures reproduced from arXiv: 2507.21945 by the authors.

Figure 1
Figure 1. The core idea of the proposed LMAC-Net. In the multimodal local query [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The proposed framework of LMAC-Net. RGB, optical flow, and audio input are [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Illustration of our multimodal local query encoder module. The module contains [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Illustration of our two-level score evaluation module. [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: Changes in attention centers of modality-specific queries across three randomly [PITH_FULL_IMAGE:figures/full_fig_p031_5.png]
Figure 6
Figure 6. Figure 6: Heatmaps comparing the three-modality alignment performance on the Fis [PITH_FULL_IMAGE:figures/full_fig_p033_6.png]
Figure 7
Figure 7. Figure 7: The area charts depict the trends and magnitudes of the absolute distances [PITH_FULL_IMAGE:figures/full_fig_p034_7.png]
Figure 8
Figure 8. Figure 8: The alignment accuracy between modalities measured by the attention center [PITH_FULL_IMAGE:figures/full_fig_p035_8.png]
Figure 9
Figure 9. Figure 9: Visualization of the cross-attention weights in the multimodal local query en [PITH_FULL_IMAGE:figures/full_fig_p036_9.png]
Figure 10
Figure 10. Figure 10: Visualization of the cross-attention weights in the final decoder layer of each [PITH_FULL_IMAGE:figures/full_fig_p037_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

46 extracted references · 41 canonical work pages

  1. [9]

    Multimodal action quality assess- ment.IEEE Transactions on Image Processing, 2024

    Ling-An Zeng and Wei-Shi Zheng. Multimodal action quality assess- ment.IEEE Transactions on Image Processing, 2024

    work page 2024

  2. [1]

    Finediving: A fine-grained dataset for procedure-aware action quality assessment

    Jinglin Xu, Yongming Rao, Xumin Yu, Guangyi Chen, Jie Zhou, and Jiwen Lu. Finediving: A fine-grained dataset for procedure-aware action quality assessment. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2949–2958, 2022

    work page 2022

  3. [2]

    Fine-grained spatio-temporal parsing network for action quality assess- ment.IEEE Transactions on Image Processing, 32:6386–6400, 2023

    Kumie Gedamu, Yanli Ji, Yang Yang, Jie Shao, and Heng Tao Shen. Fine-grained spatio-temporal parsing network for action quality assess- ment.IEEE Transactions on Image Processing, 32:6386–6400, 2023

    work page 2023

  4. [3]

    Learning to score figure skating sport videos.IEEE transactions on circuits and systems for video technology, 30(12):4578– 4590, 2019

    Chengming Xu, Yanwei Fu, Bing Zhang, Zitian Chen, Yu-Gang Jiang, and Xiangyang Xue. Learning to score figure skating sport videos.IEEE transactions on circuits and systems for video technology, 30(12):4578– 4590, 2019

    work page 2019

  5. [4]

    Hybrid dynamic-static context-aware attention network for action assessment in long videos

    Ling-AnZeng, Fa-TingHong, Wei-ShiZheng, Qi-ZhiYu, WeiZeng, Yao- Wei Wang, and Jian-Huang Lai. Hybrid dynamic-static context-aware attention network for action assessment in long videos. InProceedings of the 28th ACM international conference on multimedia, pages 2526–2534, 2020

    work page 2020

  6. [5]

    End-to-end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. InEuropean conference on computer vision, pages 213–229. Springer, 2020

    2020

  7. [6]

    Likert scoring with grade decoupling for long-term action assessment

    Angchi Xu, Ling-An Zeng, and Wei-Shi Zheng. Likert scoring with grade decoupling for long-term action assessment. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3232–3241, 2022

    work page 2022

  8. [7]

    Localization-assisted uncertainty score disentanglement net- work for action quality assessment

    Yanli Ji, Lingfeng Ye, Huili Huang, Lijing Mao, Yang Zhou, and Lin- gling Gao. Localization-assisted uncertainty score disentanglement net- work for action quality assessment. InProceedings of the 31st ACM International Conference on Multimedia, pages 8590–8597, 2023

    work page 2023

Show all 46 references
  1. [8]

    Skating-mixer: Long-term sport audio- visual modeling with mlps

    Jingfei Xia, Mingchen Zhuge, Tiantian Geng, Shun Fan, Yuantai Wei, Zhenyu He, and Feng Zheng. Skating-mixer: Long-term sport audio- visual modeling with mlps. InProceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 2901–2909, 2023. 39

    2023

  2. [10]

    Audio-visual scene analysis with self-supervised multisensory features

    Andrew Owens and Alexei A Efros. Audio-visual scene analysis with self-supervised multisensory features. InProceedings of the European conference on computer vision (ECCV), pages 631–648, 2018

    2018

  3. [11]

    Dual attention matching for audio-visual event localization

    Yu Wu, Linchao Zhu, Yan Yan, and Yi Yang. Dual attention matching for audio-visual event localization. InProceedings of the IEEE/CVF international conference on computer vision, pages 6292–6300, 2019

    2019

  4. [14]

    Egocentric deep multi-channel audio-visual active speaker localization

    Hao Jiang, Calvin Murdock, and Vamsi Krishna Ithapu. Egocentric deep multi-channel audio-visual active speaker localization. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10544–10552, 2022

    2022

  5. [15]

    Assessing the quality of actions

    Hamed Pirsiavash, Carl Vondrick, and Antonio Torralba. Assessing the quality of actions. InComputer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VI 13, pages 556–571. Springer, 2014

    2014

  6. [16]

    Learning to score olympic events

    Paritosh Parmar and Brendan Tran Morris. Learning to score olympic events. InProceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 20–28, 2017

    2017

  7. [17]

    Scoringnet: Learning key fragmentforactionqualityassessmentwithrankinglossinskilledsports

    Yongjun Li, Xiujuan Chai, and Xilin Chen. Scoringnet: Learning key fragmentforactionqualityassessmentwithrankinglossinskilledsports. InAsian Conference on Computer Vision, pages 149–164. Springer, 2018

    2018

  8. [18]

    S3d: Stacking segmental p3d for action quality assessment

    Xiang Xiang, Ye Tian, Austin Reiter, Gregory D Hager, and Trac D Tran. S3d: Stacking segmental p3d for action quality assessment. In 40 2018 25th IEEE international conference on image processing (ICIP), pages 928–932. IEEE, 2018

    2018

  9. [19]

    What and how well you performed? a multitask learning approach to action quality assessment

    Paritosh Parmar and Brendan Tran Morris. What and how well you performed? a multitask learning approach to action quality assessment. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 304–313, 2019

    2019

  10. [20]

    Action quality assess- ment using siamese network-based deep metric learning.IEEE Trans- actions on Circuits and Systems for Video Technology, 31(6):2260–2273, 2020

    Hiteshi Jain, Gaurav Harit, and Avinash Sharma. Action quality assess- ment using siamese network-based deep metric learning.IEEE Trans- actions on Circuits and Systems for Video Technology, 31(6):2260–2273, 2020

    2020

  11. [21]

    Group-aware contrastive regression for action quality assessment

    Xumin Yu, Yongming Rao, Wenliang Zhao, Jiwen Lu, and Jie Zhou. Group-aware contrastive regression for action quality assessment. In Proceedings of the IEEE/CVF international conference on computer vi- sion, pages 7919–7928, 2021

    2021

  12. [22]

    Tsa-net: Tube self-attention network for action quality assess- ment

    Shunli Wang, Dingkang Yang, Peng Zhai, Chixiao Chen, and Lihua Zhang. Tsa-net: Tube self-attention network for action quality assess- ment. InProceedings of the 29th ACM international conference on mul- timedia, pages 4902–4910, 2021

    2021

  13. [23]

    The pros and cons: Rank-aware temporal attention for skill determination in long videos

    Hazel Doughty, Walterio Mayol-Cuevas, and Dima Damen. The pros and cons: Rank-aware temporal attention for skill determination in long videos. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7862–7871, 2019

    2019

  14. [24]

    Logo: A long-form video dataset for group action quality assessment

    Shiyi Zhang, Wenxun Dai, Sujia Wang, Xiangwei Shen, Jiwen Lu, Jie Zhou, and Yansong Tang. Logo: A long-form video dataset for group action quality assessment. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2405–2414, 2023

    2023

  15. [25]

    Audiovisual slowfast networks for video recog- nition.arXiv preprint arXiv:2001.08740, 2020

    Fanyi Xiao, Yong Jae Lee, Kristen Grauman, Jitendra Malik, and Christoph Feichtenhofer. Audiovisual slowfast networks for video recog- nition.arXiv preprint arXiv:2001.08740, 2020

    2001 arXiv

  16. [26]

    Listentolook: Actionrecognitionbypreviewingaudio

    Ruohan Gao, Tae-Hyun Oh, Kristen Grauman, and Lorenzo Torresani. Listentolook: Actionrecognitionbypreviewingaudio. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10457–10467, 2020. 41

    2020

  17. [27]

    Cross- attentional audio-visual fusion for weakly-supervised action localization

    Jun-Tae Lee, Mihir Jain, Hyoungwoo Park, and Sungrack Yun. Cross- attentional audio-visual fusion for weakly-supervised action localization. InInternational conference on learning representations, 2020

    2020

  18. [28]

    Cross-modal background suppression for audio- visual event localization

    Yan Xia and Zhou Zhao. Cross-modal background suppression for audio- visual event localization. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19989–19998, 2022

    2022

  19. [29]

    Mmw-aqa: Multimodal in-the-wild dataset for action quality assess- ment.IEEE Access, 2024

    TakasukeNagai, ShoichiroTakeda, SatoshiSuzuki, andHitoshiSeshimo. Mmw-aqa: Multimodal in-the-wild dataset for action quality assess- ment.IEEE Access, 2024

    2024

  20. [30]

    Vision-language action knowledge learning for semantic-aware action quality assessment

    Huangbiao Xu, Xiao Ke, Yuezhou Li, Rui Xu, Huanqi Wu, Xiaofeng Lin, and Wenzhong Guo. Vision-language action knowledge learning for semantic-aware action quality assessment. InEuropean Conference on Computer Vision, pages 423–440. Springer, 2024

    2024

  21. [31]

    Learning semantics- guided representations for scoring figure skating.IEEE Transactions on Multimedia, 26:4987–4997, 2023

    Zexing Du, Di He, Xue Wang, and Qing Wang. Learning semantics- guided representations for scoring figure skating.IEEE Transactions on Multimedia, 26:4987–4997, 2023

    2023

  22. [32]

    Temporal and cross-modal attention for audio-visual zero-shot learning

    Otniel-BogdanMercea, ThomasHummel, ASophiaKoepke, andZeynep Akata. Temporal and cross-modal attention for audio-visual zero-shot learning. InEuropean Conference on Computer Vision, pages 488–505. Springer, 2022

    2022

  23. [33]

    Cross-attention is not always needed: Dynamic cross-attention for audio-visual dimensional emotion recognition

    R Gnana Praveen and Jahangir Alam. Cross-attention is not always needed: Dynamic cross-attention for audio-visual dimensional emotion recognition. In2024 IEEE International Conference on Multimedia and Expo (ICME), pages 1–6. IEEE, 2024

    2024

  24. [34]

    Alignvsr: Audio-visual cross-modal alignment for visual speech recogni- tion.arXiv preprint arXiv:2410.16438, 2024

    Zehua Liu, Xiaolou Li, Chen Chen, Li Guo, Lantian Li, and Dong Wang. Alignvsr: Audio-visual cross-modal alignment for visual speech recogni- tion.arXiv preprint arXiv:2410.16438, 2024

    2024 arXiv

  25. [35]

    Cross-modal global interaction and local alignment for audio-visual speech recognition.arXiv preprint arXiv:2305.09212, 2023

    Yuchen Hu, Ruizhe Li, Chen Chen, Heqing Zou, Qiushi Zhu, and Eng Siong Chng. Cross-modal global interaction and local alignment for audio-visual speech recognition.arXiv preprint arXiv:2305.09212, 2023. 42

    2023 arXiv

  26. [36]

    Temporal alignment networks for long-term video

    Tengda Han, Weidi Xie, and Andrew Zisserman. Temporal alignment networks for long-term video. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 2906–2916, 2022

    2022

  27. [37]

    Temporal and cross-modal attention for audio-visual zero-shot learning.Advances in neural information processing systems, 35:38032–38045, 2022

    Yuchong Sun, Hongwei Xue, Ruihua Song, Bei Liu, Huan Yang, and Jianlong Fu. Temporal and cross-modal attention for audio-visual zero-shot learning.Advances in neural information processing systems, 35:38032–38045, 2022

    2022

  28. [38]

    Video and accelerometer-based motion analysis for automated surgical skills assessment.International journal of computer assisted radiology and surgery, 13:443–455, 2018

    Aneeq Zia, Yachna Sharma, Vinay Bettadapura, Eric L Sarin, and Irfan Essa. Video and accelerometer-based motion analysis for automated surgical skills assessment.International journal of computer assisted radiology and surgery, 13:443–455, 2018

    2018

  29. [41]

    Audio set: An ontology and human-labeled dataset for audio events

    JortFGemmeke, DanielPWEllis, DylanFreedman, ArenJansen, Wade Lawrence, R Channing Moore, Manoj Plakal, and Marvin Ritter. Audio set: An ontology and human-labeled dataset for audio events. In2017 IEEE international conference on acoustics, speech and signal processing (ICASSP)...

    2017

  30. [42]

    Action quality assessment with temporal parsing transformer

    Yang Bai, Desen Zhou, Songyang Zhang, Jian Wang, Errui Ding, Yu Guan, Yang Long, and Jingdong Wang. Action quality assessment with temporal parsing transformer. InEuropean conference on computer vision, pages 422–438. Springer, 2022

    2022

  31. [43]

    Learning spatiotemporal features with 3d convolu- tional networks

    Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. Learning spatiotemporal features with 3d convolu- tional networks. InProceedings of the IEEE international conference on computer vision, pages 4489–4497, 2015

    2015

  32. [44]

    Video and accelerometer-based motion analysis for automated 43 surgical skills assessment.International journal of computer assisted radiology and surgery, 13:443–455, 2018

    Aneeq Zia, Yachna Sharma, Vinay Bettadapura, Eric L Sarin, and Irfan Essa. Video and accelerometer-based motion analysis for automated 43 surgical skills assessment.International journal of computer assisted radiology and surgery, 13:443–455, 2018

    2018

  33. [45]

    Deep resid- uallearningforimagerecognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep resid- uallearningforimagerecognition. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  34. [46]

    Quo vadis, action recognition? a new model and the kinetics dataset

    Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. Inproceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition, pages 6299–6308, 2017

    2017

  35. [47]

    Ast: Audio spectrogram transformer.arXiv preprint arXiv:2104.01778, 2021

    Yuan Gong, Yu-An Chung, and James Glass. Ast: Audio spectrogram transformer.arXiv preprint arXiv:2104.01778, 2021

    2021 arXiv

  36. [48]

    Joint visual and audio learning for video highlight detection

    Taivanbat Badamdorj, Mrigank Rochan, Yang Wang, and Li Cheng. Joint visual and audio learning for video highlight detection. InProceed- ings of the IEEE/CVF International Conference on Computer Vision, pages 8127–8137, 2021

    2021

  37. [49]

    Msaf: Multimodal split attention fusion.arXiv preprint arXiv:2012.07175, 2020

    Lang Su, Chuqing Hu, Guofa Li, and Dongpu Cao. Msaf: Multimodal split attention fusion.arXiv preprint arXiv:2012.07175, 2020

    2012 arXiv

  38. [50]

    Umt: Unified multi-modal transformers for joint video moment retrieval and highlight detection

    Ye Liu, Siyuan Li, Yang Wu, Chang-Wen Chen, Ying Shan, and Xiaohu Qie. Umt: Unified multi-modal transformers for joint video moment retrieval and highlight detection. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 3042–3051, 2022. 44

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.