Pith. sign in

REVIEW 5 major objections 5 minor 3 cited by

Explainable Action Form Assessment by Exploiting Multimodal Chain-of-Thoughts Reasoning

T0 review · 5 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper defines action form assessment: judging whether a movement is standard, and explaining why with corrective reasoning.

desk verdict The dataset is a real contribution; the model comparison is not, because EFA gets the standard-technical-steps text that baselines don't see. read the letter →

arxiv 2512.15153 v2 pith:XX2CXONO submitted 2025-12-17 cs.CV

Mengshi Qi , Yeteng Wu , Wulian Yun , Xianlin Zhang , Huadong Ma This is my paper
classification cs.CV
keywords ActionQualityAssessmentFormChain-of-ThoughtReasoningVideoUnderstandingExplainableAIMultimodalFusionCaptioningFitnessAnalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces a new video-understanding task: judging whether a human action is performed in standard form, and doing so with an explanation, not just a score. To support it, the authors build CoT-AFA, a dataset of 3,392 fitness and martial-arts videos annotated with standard/non-standard labels, viewpoints, and chain-of-thought corrective texts. They propose EFA, a framework that aligns video features against the action's standard technical steps through dual cross-attention streams and a dynamic gating fusion, then outputs action class, quality label, and the explanatory text. On their dataset EFA reports the best results across all three tasks, including a 16.0-point CIDEr gain in explanation generation. If the approach holds, it could turn generic coaching tips into objective, explainable assessments that connect an error to its cause and a correction.

What carries the argument

The central object is the Standard Technical Steps: an ordered set of five procedural instructions plus a general instruction per action category, generated by an LLM and stored as text. EFA fuses video tokens from a Video Swin Transformer with BERT embeddings of these steps using two bidirectional cross-attention branches—Step-Aware Fusion for fine-grained grounding and Global-Aware Fusion for overall context—followed by a dynamic gating layer that weights the two streams. The gated representation feeds three heads: an MLP action classifier, an MLP quality classifier, and a Transformer text generator that auto-regressively produces the chain-of-thought explanation.

What would settle it

Run EFA without any text input against the best text-generation baseline, and separately give that baseline the standard technical steps as auxiliary input; if EFA's lead vanishes under matched-input conditions, the claimed architectural advantage is not established.

Watch

Extended reading notes

Core claim

The paper claims that a model which explicitly aligns visual evidence with predefined standard technical steps can simultaneously classify an action, judge whether its form is standard, and generate a chain-of-thought explanation that moves from identifying a non-standard step to analyzing its consequence to proposing a concrete solution. The authors define the Human Action Form Assessment (AFA) task, introduce the CoT-AFA dataset with 141 action categories and multi-level annotations, and present the Explainable Fitness Assessor (EFA) framework. EFA uses two parallel bidirectional-attention branches—one for global action context, one for step-level procedural detail—and a dynamic gating lay

Load-bearing premise

The evaluation assumes that comparing EFA—which receives the standard technical steps as input—against baselines that do not receive this input is a fair test of the architecture; the large w/o-text drop shows the text input itself drives much of the gain.

Editorial extensions

If this is right

  • If the AFA task gains traction, coaching, rehabilitation, and training systems can move from generic tips to structured feedback that names the error, explains its consequence, and prescribes a fix.
  • The CoT-AFA dataset provides a public benchmark with standard/non-standard labels and chain-of-thought texts, enabling direct comparison of future explainable action assessment models.
  • EFA's improvement in classification and quality accuracy shows that feeding procedural text alongside video can sharpen visual understanding, not just caption generation.
  • The generated explanations use expert-level terminology (e.g., 'lumbar hyperextension'), suggesting the framework can learn precise technical language from ordered instructions.
  • The authors state the method will be applied to robotic applications, implying the same step-guided assessment could be used for robot action verification and correction.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported gains may substantially reflect the extra text input: the w/o-text ablation drops CIDEr from 33.3 to 29.9, so a matched-input comparison—giving baselines the same standard technical steps—is needed to isolate the architectural contribution.
  • The chain-of-thought explanation paradigm could transfer to other domains with objective procedural standards, such as surgical technique, industrial assembly, or rehabilitation exercises, where step-by-step corrective feedback matters.
  • Because non-standard videos were partly self-shot by the authors, real-world deployment would need testing on naturally occurring errors to confirm the model generalizes beyond the dataset's distribution.
  • At inference, the model requires the action's class-matched steps; it cannot diagnose errors for unseen action categories, so an open-set or retrieval-based variant is a natural next test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper defines a new task, Human Action Form Assessment (AFA), and introduces a dataset CoT-AFA containing 3,392 fitness and martial-arts videos annotated with action class, standard/non-standard quality labels, and multi-sentence Chain-of-Thought explanations. The proposed Explainable Fitness Assessor (EFA) uses a Video Swin visual encoder and a BERT text encoder, fuses visual features with the action's Standard Technical Steps through global-aware and step-aware bidirectional attention branches, and combines the branches with a dynamic gating layer. The model jointly predicts the action class, the quality label, and a CoT explanation. Experiments on CoT-AFA report improvements over video-captioning and action-recognition baselines in all three tasks. The main technical novelty is the use of pre-defined Standard Technical Steps as structured textual guidance for form assessment and explanation generation.

Significance. The dataset is a potentially valuable resource: it addresses a real gap by combining objective standard/non-standard labels with causal CoT explanations, and the annotation pipeline includes human expert review. If the empirical claims were supported, the step-aware gating architecture would be a useful baseline for AFA. However, the current evidence is not sufficient. The main comparisons are confounded by an input-modality mismatch (EFA receives Standard Technical Steps text while the baselines are video-only), the reported headline gains in the abstract do not match the numbers in Table V, and no uncertainty estimates or significance tests are provided. The paper's own w/o-text ablation in Table VI shows that most of the apparent gain disappears when the text input is removed. The dataset and task framing are contributions, but the architecture SOTA claim is not established as presented.

major comments (5)
  1. [§V-B, Tables IV–VI; Algorithm 1] EFA is trained and evaluated with the per-class Standard Technical Steps T as an additional input (Algorithm 1, line 4: T_i = Map(y_i)), while the captioning baselines in Table IV and the recognition baselines in Table V are video-only. The w/o-text ablation in Table VI removes this input, and most of the apparent gain disappears: CIDEr drops from 33.3 to 29.9, BLEU from 49.2 to 48.1, METEOR from 23.8 to 22.9, ROUGE-L from 37.7 to 35.6, Top-1 from 77.2 to 75.3, and quality Acc from 81.8 to 81.2. At this video-only setting, EFA is not clearly better than LAVENDER (ROUGE-L 36.0, METEOR 22.8), RICA2 (quality 81.4), or Video Swin (Top-1 75.2). The claimed +16.0% CIDEr, +2.7% Top-1, and +2.1% quality gains are therefore largely explained by the privileged text input rather than by the proposed architecture. The comparison must be controlled by giving the identical Standard Technical Steps to
  2. [Abstract; §V-B, Table V] The headline deltas do not match Table V. EFA Top-1 is 0.772 and the strongest action-recognition baseline (Video Swin) is 0.752, a 2.0% relative difference, not 2.7%. For quality, the best baseline in Table V is RICA2 at 0.814; EFA is 0.818, a 0.5% relative difference, not 2.1%. The +2.1% figure matches a comparison against Video Swin (0.801) on quality, but Table V identifies RICA2 as stronger, and the text says 'surpassing the best baseline by 2.1%.' The reported improvements need to be recomputed and stated consistently with the tables.
  3. [§V-A, §V-B] All results are reported on a single 70/15/15 split of a self-built dataset, with no standard deviations, confidence intervals, or significance tests. Many of the headline differences are small (quality 0.818 vs 0.814; METEOR 23.8 vs 23.5; Top-1 77.2 vs 75.2) and could easily lie within run-to-run variation. I ask for multi-seed runs with variance reporting and, where possible, a significance test. External validation on at least one existing AQA/action dataset would also materially strengthen the claim that the method, rather than the dataset, is responsible for the results.
  4. [§III-D, Figure 4; §IV-C] The CoT ground-truth explanations are produced by a VLM from the same Standard Technical Steps (plus video and quality label) that EFA receives as input. Consequently, the explanation-generation gains may partly reflect copying or rephrasing the provided steps rather than reasoning from video. The 'Shuffled text' ablation shows order matters, but it does not rule out copying, since the same vocabulary and step content are present. An evaluation with held-out action categories (where the correct steps are withheld at inference) or with semantically irrelevant/corrupted text would better separate video-grounded explanation from text copying.
  5. [Eq. (11); Algorithm 1; §V-C, Table VII] The loss equation and the algorithm disagree: Eq. (11) is L = λL_c + L_q + L_t, while Algorithm 1 (line 11) is L_total = L_c + λL_q + L_t. Section V-C describes λ as the weight on L_c. Moreover, Table VII shows λ=15 yields CIDEr 34.7, the best captioning CIDEr in the table, contradicting the claim that λ=3 is 'best or jointly best across all evaluation metrics.' These inconsistencies need correction because the hyperparameter is part of the method's definition.
minor comments (5)
  1. [Fig. 1 caption] 'respcetively' is a typo for 'respectively'.
  2. [Table IV and §V-B] 'LA VENDER' appears inconsistently; the correct name is 'LAVENDER'.
  3. [§II Related Work] The text says 'Zhang et al. [16], [49] introduced a prompt-guided multimodal framework' but [16] is Panchal et al. on live fitness coaching. If the narrative evaluation framework is only [49], the citation needs correction.
  4. [Table II] The average number of reasoning steps is reported as 0.91 per sample and actionable suggestions as 0.75. This implies many explanations contain no explicit causal step or suggestion; please clarify how these samples satisfy the claimed Chain-of-Thought structure.
  5. [§V-C, Table VI] For the 'Add' fusion variant, CIDEr is 33.3, equal to the full model. The claim that both 'Concatenate' and 'Add' 'result in lower scores' is therefore not strictly true for CIDEr; please qualify that the drop is in the other metrics.

Circularity Check

1 steps flagged · score 6.0 of 10

Explanation-generation claims are partly tautological: EFA is fed the same Standard Technical Steps that were used to generate the CoT reference texts, while video-only baselines never receive them.

  1. self definitional [Sec. III-D / Fig. 4; Sec. IV Algorithm 1; Sec. V-C Table VI]
    "Next, these generated steps are consolidated with the target action video and its pre-annotated quality label, which are then collectively input into a VLM (i.e., VideoChat). Ultimately, the VLM generates a detailed CoT text explanation. ... Retrieve standard technical steps and the general instruction T_i = Map(y_i) ... Generate CoT text explanation \hat t_i = TextGenerator(F_f). ... The complete absence of text (w/o text) severely degrades the model’s descriptive capability, causing the CIDEr score to plummet to 29.9."

    The CoT reference text t_i that EFA is trained to predict was itself generated from the Standard Technical Steps T_i (Sec. III-D and Fig. 4). EFA is then given the same T_i as an explicit input during training and inference (Algorithm 1, lines 4 and 10) and optimizes L_t(\hat t_i, t_i). Thus the explanation-generation 'prediction' is, by construction, a function of the very text used to create the reference: the model can copy or paraphrase T_i rather than perform independent video-to-text reasoning. The +16.0% CIDEr improvement over video-only baselines in Table IV is therefore not a controlled measure of the architecture; Table VI shows removing T_i drops CIDEr from 33.3 to 29.9, close to the strongest baseline. This is the definitional form of circularity: the target explanation is gene

full rationale

The dataset construction itself is partly independent: standard/non-standard labels are pre-annotated and human-reviewed, and the CoT texts pass VLM-based consistency checks followed by expert manual editing. The architectural components (dual-branch fusion, gating) are ordinary supervised modules and are not derived from the target. The circularity is concentrated in the explanation-generation evaluation. The CoT reference t_i is produced by a VLM from the Standard Technical Steps T_i (Sec. III-D/Fig. 4), and EFA is explicitly trained and run with the same T_i as an input (Algorithm 1: 'Retrieve standard technical steps ... T_i = Map(y_i)'; 'Generate CoT text explanation \hat t_i = TextGenerator(F_f)'). The model is therefore rewarded for copying or paraphrasing the very text that was used to create the reference, while the video-only baselines in Table IV never receive T_i. The paper's own Table VI w/o-text variant (CIDEr 33.3→29.9; BLEU 49.2→48.1; Top-1 77.2→75.3; quality Acc 81.8→81.2) shows that most of the headline gap is supplied by T_i, not by the proposed fusion architecture. This is an evaluation-design circularity/confound rather than a derivation from first principles. The class and quality labels are independently supervised, so those sub-tasks retain independent content, but the headline explanation-generation claim (+16.0% CIDEr) is partially tautological. I therefore set the circularity score at 6, not higher, because the model still has to ground T_i in the video to produce video-specific explanations, and the classification/quality tasks are not definitionally forced.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central claims rest on the quality of the newly collected dataset and on the assumption that the standard-technical-steps input is valid. The only explicit hand-tuned free parameter is the loss weight λ. No new physical or conceptual entities are introduced.

free parameters (1)
  • loss weight λ = 3
    Chosen by hand to balance classification vs. other losses; Table VII shows λ=3 gives the best trade-off, but the value is tuned on the validation set.
assumptions (3)
  • domain assumption The pre-trained Video Swin and BERT embeddings are suitable for workout videos without domain adaptation.
    Used as backbones in Section IV-A; no experiments test whether Kinetics-600 or BERT pretraining transfers well to fitness/martial-arts videos.
  • domain assumption Human expert review ensures that quality labels and CoT explanations are reliable ground truth.
    Section III-D describes a two-stage review process, but the paper provides no inter-annotator agreement or quantitative validation of the labels.
  • domain assumption Standard technical steps generated by Gemini 2.0 are correct and helpful guidance.
    Section III-D uses these steps as input to the model; there is no external validation that the steps are accurate or complete.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Explainable Action Form Assessment by Exploiting Multimodal Chain-of-Thoughts Reasoning." pith.science (2026). https://pith.science/paper/XX2CXONO

@misc{pith2026251215153,
  author       = {Pith},
  title        = {Pith review of: Explainable Action Form Assessment by Exploiting Multimodal Chain-of-Thoughts Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XX2CXONO}},
  note         = {Machine review of arXiv:2512.15153}
}
read the original abstract

Evaluating whether human action is standard or not and providing reasonable feedback to improve action standardization is very crucial but challenging in real-world scenarios. However, current video understanding methods are mainly concerned with what and where the action is, which is unable to meet the requirements. Meanwhile, most of the existing datasets lack the labels indicating the degree of action standardization, and the action quality assessment datasets lack explainability and detailed feedback. Therefore, we define a new Human Action Form Assessment (AFA) task, and introduce a new diverse dataset CoT-AFA, which contains a large scale of fitness and martial arts videos with multi-level annotations for comprehensive video analysis. We enrich the CoT-AFA dataset with a novel Chain-of-Thought explanation paradigm. Instead of offering isolated feedback, our explanations provide a complete reasoning process--from identifying an action step to analyzing its outcome and proposing a concrete solution. Furthermore, we propose a framework named Explainable Fitness Assessor, which can not only judge an action but also explain why and provide a solution. This framework employs two parallel processing streams and a dynamic gating mechanism to fuse visual and semantic information, thereby boosting its analytical capabilities. The experimental results demonstrate that our method has achieved improvements in explanation generation (e.g., +16.0% in CIDEr), action classification (+2.7% in accuracy) and quality assessment (+2.1% in accuracy), revealing great potential of CoT-AFA for future studies. Our dataset and source code is available at https://github.com/MICLAB-BUPT/EFA.

Figures

Figures reproduced from arXiv: 2512.15153 by the authors.

Figure 1
Figure 1. Illustrations of standard and non-standard action examples in our [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Three-level lexicon annotation structure of apparatus (left) and manual (right). The first colored layer outside the center of the circle represents martial [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Illustration of multi-element annotations of CoT-AFA dataset, which shows two categories of actions from different views, including front view, side [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The workflow for textual explanation generation process of CoT-AFA, showing the generation of Standard Technical Steps for each action using LLM [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: The architecture of our proposed Explainable Fitness Assessor (EFA). EFA receives video frames and text as input. Visual and text features are [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Illustration of the proposed Step-Aware Fusion Module, Global-Aware [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Examples of EFA’s text generation for the non-standard ”Seated Cable Row” action (top) and the standard ”Barbell Bent-Over Row” action (bottom). [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A DVDrive Approach for doScenes Instructed Driving Challenge

    cs.CV 2026-06 unverdicted novelty 3.0 of 10

    The submission adapts OmniDrive with a DVPE-style divided-view perception module to enhance instruction-conditioned ego trajectory prediction on nuScenes scenes for the doScenes challenge.

  2. Leveraging Metric Depth for Relative Depth Prediction

    cs.CV 2026-06 unverdicted novelty 2.0 of 10

    Competition solution applies zero-shot pretrained models for metric depth to achieve relative depth prediction in football scenes with limited data, scoring 2.68e-3.

  3. A VideoMAE-v2 Approach to Zero-Shot Traffic Accident Anticipation

    cs.CV 2026-06 unverdicted novelty 2.0 of 10

    VideoMAE-v2 backbone with per-frame head achieves 2nd place in 2026 CVPR zero-shot traffic accident anticipation competition by training solely on public binary-labeled data.

Reference graph

Works this paper leans on

77 extracted references · 5 linked inside Pith · cited by 3 Pith papers

  1. [1]

    Two-stream convolutional networks for action recognition in videos,

    K. Simonyan and A. Zisserman, “Two-stream convolutional networks for action recognition in videos,” inProceedings of the 27th International Conference on Neural Information Processing Systems - Volume 1, ser. NIPS’14. Cambridge, MA, USA: MIT Press, 2014, p. 568–576

    2014

  2. [2]

    Temporal segment networks for action recognition in videos,

    L. Wang, Y . Xiong, Z. Wang, Y . Qiao, D. Lin, X. Tang, and L. Van Gool, “Temporal segment networks for action recognition in videos,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 11, pp. 2740–2755, 2019

    2019

  3. [3]

    Long-term recurrent convolutional networks for visual recognition and description,

    J. Donahue, L. A. Hendricks, M. Rohrbach, S. Venugopalan, S. Guadar- rama, K. Saenko, and T. Darrell, “Long-term recurrent convolutional networks for visual recognition and description,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 4, pp. 677–691, 2017

    2017

  4. [4]

    Tsm: Temporal shift module for efficient video understanding,

    J. Lin, C. Gan, and S. Han, “Tsm: Temporal shift module for efficient video understanding,” in2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 7082–7092

    2019

  5. [5]

    Quo vadis, action recognition? a new model and the kinetics dataset,

    J. Carreira and A. Zisserman, “Quo vadis, action recognition? a new model and the kinetics dataset,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017

    2017

  6. [6]

    Slowfast networks for video recognition,

    C. Feichtenhofer, H. Fan, J. Malik, and K. He, “Slowfast networks for video recognition,” in2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 6201–6210

    2019

  7. [7]

    Temporal action detection with structured segment networks,

    Y . Zhao, Y . Xiong, L. Wang, Z. Wu, X. Tang, and D. Lin, “Temporal action detection with structured segment networks,” in2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2933– 2942

    2017

  8. [8]

    Weakly supervised temporal action localization via representative snippet knowledge propagation,

    L. Huang, L. Wang, and H. Li, “Weakly supervised temporal action localization via representative snippet knowledge propagation,” inPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 3272–3281

    2022

Show all 77 references
  1. [9]

    Asm- loc: Action-aware segment modeling for weakly-supervised temporal action localization,

    B. He, X. Yang, L. Kang, Z. Cheng, X. Zhou, and A. Shrivastava, “Asm- loc: Action-aware segment modeling for weakly-supervised temporal action localization,”Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), 2022

    2022

  2. [10]

    The kinetics human action video dataset,

    W. Kay, J. Carreira, K. Simonyan, B. Zhang, C. Hillier, S. Vijaya- narasimhan, F. Viola, T. Green, T. Back, P. Natsev, M. Suleyman, and A. Zisserman, “The kinetics human action video dataset,” 05 2017

    2017

  3. [11]

    Ucf101: A dataset of 101 human actions classes from videos in the wild,

    K. Soomro, A. R. Zamir, and M. Shah, “Ucf101: A dataset of 101 human actions classes from videos in the wild,”ArXiv, vol. abs/1212.0402, 2012

    2012 arXiv

  4. [12]

    Activitynet: A large-scale video benchmark for human activity understanding,

    F. C. Heilbron, V . Escorcia, B. Ghanem, and J. C. Niebles, “Activitynet: A large-scale video benchmark for human activity understanding,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 961–970

    2015

  5. [13]

    Thumos challenge: Action recognition with a large number of classes,

    Y .-G. Jiang, J. Liu, A. R. Zamir, G. Toderici, I. Laptev, M. Shah, and R. Sukthankar, “Thumos challenge: Action recognition with a large number of classes,” 2014, https://www.crcv.ucf.edu/THUMOS14/

    2014

  6. [14]

    Assessing the quality of actions,

    H. Pirsiavash, C. V ondrick, and A. Torralba, “Assessing the quality of actions,” inComputer Vision – ECCV 2014, D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, Eds. Cham: Springer International Publishing, 2014, pp. 556–571

    2014

  7. [15]

    Action quality assessment across multiple actions,

    P. Parmar and B. T. Morris, “Action quality assessment across multiple actions,”2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1468–1476, 2018. 12

    2019

  8. [16]

    What to say and when to say it: Live fitness coaching as a testbed for situated interac- tion,

    S. Panchal, A. Bhattacharyya, G. Berger, A. Mercier, C. B ¨ohm, F. Diet- richkeit, R. Pourreza, X. Li, P. Madan, M. Leeet al., “What to say and when to say it: Live fitness coaching as a testbed for situated interac- tion,”Advances in Neural Information Processing Systems, vol...

    2024

  9. [17]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhouet al., “Chain-of-thought prompting elicits reasoning in large language models,”Advances in neural information processing systems, vol. 35, pp. 24 824–24 837, 2022

    2022

  10. [18]

    Temporal relational reasoning in videos,

    B. Zhou, A. Andonian, A. Oliva, and A. Torralba, “Temporal relational reasoning in videos,”European Conference on Computer Vision, 2018

    2018

  11. [19]

    Spatial temporal graph convolutional networks for skeleton-based action recognition,

    S. Yan, Y . Xiong, and D. Lin, “Spatial temporal graph convolutional networks for skeleton-based action recognition,” inProceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and E...

    2018

  12. [20]

    P-CNN: Pose-based CNN Features for Action Recognition,

    G. Ch’eron, I. Laptev, and C. Schmid, “P-CNN: Pose-based CNN Features for Action Recognition,” inICCV, 2015

    2015

  13. [21]

    Beyond label semantics: Language-guided action anatomy for few-shot action recognition,

    Z. Qian, X. Yao, Y . Huang, C. Zhang, J. Ying, and H. Sun, “Beyond label semantics: Language-guided action anatomy for few-shot action recognition,”arXiv preprint arXiv:2507.16287, 2025

    2025 arXiv

  14. [22]

    Semantic-guided cross- modal prompt learning for skeleton-based zero-shot action recognition,

    A. Zhu, J. Zhu, J. Bailey, M. Gong, and Q. Ke, “Semantic-guided cross- modal prompt learning for skeleton-based zero-shot action recognition,” inProceedings of the Computer Vision and Pattern Recognition Confer- ence, 2025, pp. 13 876–13 885

    2025

  15. [23]

    Attentive relational networks for mapping images to scene graphs,

    M. Qi, W. Li, Z. Yang, Y . Wang, and J. Luo, “Attentive relational networks for mapping images to scene graphs,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 3957–3966

    2019

  16. [24]

    Stc-gan: Spatio-temporally coupled generative adversarial networks for predictive scene parsing,

    M. Qi, Y . Wang, A. Li, and J. Luo, “Stc-gan: Spatio-temporally coupled generative adversarial networks for predictive scene parsing,”IEEE Transactions on Image Processing, vol. 29, pp. 5420–5430, 2020

    2020

  17. [25]

    T2sg: Traffic topology scene graph for topology reasoning in autonomous driving,

    C. Lv, M. Qi, L. Liu, and H. Ma, “T2sg: Traffic topology scene graph for topology reasoning in autonomous driving,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 17 197– 17 206

    2025

  18. [26]

    Unsupervised self-driving attention prediction via uncertainty mining and knowledge embedding,

    P. Zhu, M. Qi, X. Li, W. Li, and H. Ma, “Unsupervised self-driving attention prediction via uncertainty mining and knowledge embedding,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 8558–8568

    2023

  19. [27]

    Bsn: Boundary sensitive network for temporal action proposal generation,

    T. Lin, X. Zhao, H. Su, C. Wang, and M. Yang, “Bsn: Boundary sensitive network for temporal action proposal generation,” inProceedings of the European Conference on Computer Vision (ECCV), September 2018

    2018

  20. [28]

    Weakly supervised temporal action localization via dual-prior collab- orative learning guided by multimodal large language models,

    Q. Zhang, J. Fang, R. Yuan, X. Tang, Y . Qi, K. Zhang, and C. Yuan, “Weakly supervised temporal action localization via dual-prior collab- orative learning guided by multimodal large language models,” inPro- ceedings of the Computer Vision and Pattern Recognition Conference, 2...

    2025

  21. [29]

    Disentangled counter- factual learning for physical audiovisual commonsense reasoning,

    C. Lv, S. Zhang, Y . Tian, M. Qi, and H. Ma, “Disentangled counter- factual learning for physical audiovisual commonsense reasoning,”Ad- vances in Neural Information Processing Systems, vol. 36, pp. 12 476– 12 488, 2023

    2023

  22. [30]

    stagnet: An attentive semantic rnn for group activity recognition,

    M. Qi, J. Qin, A. Li, Y . Wang, J. Luo, and L. Van Gool, “stagnet: An attentive semantic rnn for group activity recognition,” inProceedings of the European conference on computer vision (ECCV), 2018, pp. 101– 117

    2018

  23. [31]

    Imitative non-autoregressive modeling for trajectory forecasting and imputation,

    M. Qi, J. Qin, Y . Wu, and Y . Yang, “Imitative non-autoregressive modeling for trajectory forecasting and imputation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 12 736–12 745

    2020

  24. [32]

    Ke-gan: Knowledge embedded generative adversarial networks for semi-supervised scene parsing,

    M. Qi, Y . Wang, J. Qin, and A. Li, “Ke-gan: Knowledge embedded generative adversarial networks for semi-supervised scene parsing,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 5237–5246

    2019

  25. [33]

    Robust disentangled counterfactual learning for physical audiovisual commonsense reasoning,

    M. Qi, C. Lv, and H. Ma, “Robust disentangled counterfactual learning for physical audiovisual commonsense reasoning,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    2025

  26. [34]

    Pitn: Physics- informed temporal networks for cuffless blood pressure estimation,

    R. Wang, M. Qi, Y . Shao, A. Zhou, and H. Ma, “Pitn: Physics- informed temporal networks for cuffless blood pressure estimation,” IEEE Transactions on Mobile Computing, pp. 1–14, 2025

    2025

  27. [35]

    Video swin transformer,

    Z. Liu, J. Ning, Y . Cao, Y . Wei, Z. Zhang, S. Lin, and H. Hu, “Video swin transformer,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 3202–3211

    2022

  28. [36]

    Is space-time attention all you need for video understanding?

    G. Bertasius, H. Wang, and L. Torresani, “Is space-time attention all you need for video understanding?” inProceedings of the International Conference on Machine Learning (ICML), July 2021

    2021

  29. [37]

    Mvitv2: Improved multiscale vision transformers for classification and detection,

    Y . Li, C.-Y . Wu, H. Fan, K. Mangalam, B. Xiong, J. Malik, and C. Feichtenhofer, “Mvitv2: Improved multiscale vision transformers for classification and detection,” inCVPR, 2022

    2022

  30. [38]

    Automated video assessment of human performance,

    A. S. Gordon, “Automated video assessment of human performance,” 1997

    1997

  31. [39]

    Learning to score olympic events,

    P. Parmar and B. T. Morris, “Learning to score olympic events,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2017, pp. 76–84

    2017

  32. [40]

    What and how well you performed? a multitask learning approach to action quality assessment,

    Parmar, Paritosh and Morris, Brendan Tran, “What and how well you performed? a multitask learning approach to action quality assessment,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019

    2019

  33. [41]

    Who’s better? who’s best? pairwise deep ranking for skill determination,

    H. Doughty, D. Damen, and W. Mayol-Cuevas, “Who’s better? who’s best? pairwise deep ranking for skill determination,” in2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 6057–6066

    2018

  34. [42]

    Pairwise contrastive learning network for action quality assessment,

    M. Li, H.-B. Zhang, Q. Lei, Z. Fan, J. Liu, and J.-X. Du, “Pairwise contrastive learning network for action quality assessment,” inComputer Vision – ECCV 2022, S. Avidan, G. Brostow, M. Ciss ´e, G. M. Farinella, and T. Hassner, Eds. Cham: Springer Nature Switzerland, 2022, pp. 457–473

    2022

  35. [43]

    Action quality assessment via hierarchical pose-guided multi-stage contrastive regression,

    M. Qi, H. Ye, J. Peng, and H. Ma, “Action quality assessment via hierarchical pose-guided multi-stage contrastive regression,”IEEE Transactions on Image Processing, vol. 34, pp. 6461–6474, 2025

    2025

  36. [44]

    Semi-supervised teacher-reference- student architecture for action quality assessment,

    W. Yun, M. Qi, F. Peng, and H. Ma, “Semi-supervised teacher-reference- student architecture for action quality assessment,” inEuropean Confer- ence on Computer Vision. Springer, 2024, pp. 161–178

    2024

  37. [45]

    Learning long-range action repre- sentation by two-stream mamba pyramid network for figure skating assessment,

    F. Wang, Q. Wang, and P. Zhao, “Learning long-range action repre- sentation by two-stream mamba pyramid network for figure skating assessment,”arXiv preprint arXiv:2508.16291, 2025

    2025 arXiv

  38. [46]

    Multi-stage contrastive regression for action quality assessment,

    Q. An, M. Qi, and H. Ma, “Multi-stage contrastive regression for action quality assessment,” inICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 4110–4114

    2024

  39. [47]

    Safedriverag: Towards safe autonomous driving with knowledge graph-based retrieval-augmented generation,

    H. Ye, M. Qi, Z. Liu, L. Liu, and H. Ma, “Safedriverag: Towards safe autonomous driving with knowledge graph-based retrieval-augmented generation,” inProceedings of the 33rd ACM International Conference on Multimedia, 2025, pp. 11 170–11 178

    2025

  40. [48]

    Support vector regression machines,

    H. Drucker, C. J. C. Burges, L. Kaufman, A. Smola, and V . Vapnik, “Support vector regression machines,” inProceedings of the 9th In- ternational Conference on Neural Information Processing Systems, ser. NIPS’96. Cambridge, MA, USA: MIT Press, 1996, p. 155–161

    1996

  41. [49]

    Narrative action evaluation with prompt-guided multimodal interac- tion,

    S. Zhang, S. Bai, G. Chen, L. Chen, J. Lu, J. Wang, and Y . Tang, “Narrative action evaluation with prompt-guided multimodal interac- tion,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 18 430–18 439

    2024

  42. [50]

    Fine- grained video captioning for sports narrative,

    H. Yu, S. Cheng, B. Ni, M. Wang, J. Zhang, and X. Yang, “Fine- grained video captioning for sports narrative,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 6006–6015

    2018

  43. [51]

    Sports video captioning via attentive motion representation and group relationship modeling,

    M. Qi, Y . Wang, A. Li, and J. Luo, “Sports video captioning via attentive motion representation and group relationship modeling,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 30, no. 8, pp. 2617–2633, 2019

    2019

  44. [52]

    Automatic punjabi caption generation for sports images,

    M. Kaur, G. Josan, and J. Kaur, “Automatic punjabi caption generation for sports images,”INFOCOMP Journal of Computer Science, vol. 20, no. 1, 2021

    2021

  45. [53]

    Semantics-aware spatial- temporal binaries for cross-modal video retrieval,

    M. Qi, J. Qin, Y . Yang, Y . Wang, and J. Luo, “Semantics-aware spatial- temporal binaries for cross-modal video retrieval,”IEEE Transactions on Image Processing, vol. 30, pp. 2989–3004, 2021

    2021

  46. [54]

    Sgformer: Semantic graph transformer for point cloud-based 3d scene graph generation,

    C. Lv, M. Qi, X. Li, Z. Yang, and H. Ma, “Sgformer: Semantic graph transformer for point cloud-based 3d scene graph generation,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 5, 2024, pp. 4035–4043

    2024

  47. [55]

    Deep learning for video captioning: A review

    S. Chen, T. Yao, and Y .-G. Jiang, “Deep learning for video captioning: A review.” inIJCAI, 2019, p. 2

    2019

  48. [56]

    Swinbert: End-to-end transformers with sparse attention for video captioning,

    K. Lin, L. Li, C.-C. Lin, F. Ahmed, Z. Gan, Z. Liu, Y . Lu, and L. Wang, “Swinbert: End-to-end transformers with sparse attention for video captioning,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 17 949–17 958

    2022

  49. [57]

    End-to- end dense video captioning with parallel decoding,

    T. Wang, R. Zhang, Z. Lu, F. Zheng, R. Cheng, and P. Luo, “End-to- end dense video captioning with parallel decoding,” inProceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 6847–6857

    2021

  50. [58]

    Vid2seq: Large-scale pretraining of a visual language model for dense video captioning,

    A. Yang, A. Nagrani, P. H. Seo, A. Miech, J. Pont-Tuset, I. Laptev, J. Sivic, and C. Schmid, “Vid2seq: Large-scale pretraining of a visual language model for dense video captioning,” inProceedings of the 13 IEEE/CVF conference on computer vision and pattern recognition, 2023, ...

    2023

  51. [59]

    Spatio-temporal dynamics and semantic attribute enriched visual encoding for video captioning,

    N. Aafaq, N. Akhtar, W. Liu, S. Z. Gilani, and A. Mian, “Spatio-temporal dynamics and semantic attribute enriched visual encoding for video captioning,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 12 487–12 496

    2019

  52. [60]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,”Advances in neural information processing systems, vol. 30, 2017

    2017

  53. [61]

    Event- equalized dense video captioning,

    K. Wu, P. Li, J. Fu, Y . Li, Y . Wu, Y . Liu, J. Wang, and S. Zhou, “Event- equalized dense video captioning,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 8417–8427

    2025

  54. [62]

    Player-centric multimodal prompt generation for large lan- guage model based identity-aware basketball video captioning,

    Z. Xi, H. Sun, Y . Wu, J. Yan, H. Zhang, L. Wu, L. Wang, and C. Chen, “Player-centric multimodal prompt generation for large lan- guage model based identity-aware basketball video captioning,”arXiv preprint arXiv:2507.20163, 2025

    2025 arXiv

  55. [63]

    Finegym: A hierarchical video dataset for fine-grained action understanding,

    D. Shao, Y . Zhao, B. Dai, and D. Lin, “Finegym: A hierarchical video dataset for fine-grained action understanding,” inIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020

    2020

  56. [64]

    Hmdb: A large video database for human motion recognition,

    H. Kuehne, H. Jhuang, E. Garrote, T. Poggio, and T. Serre, “Hmdb: A large video database for human motion recognition,” in2011 Interna- tional Conference on Computer Vision, 2011, pp. 2556–2563

    2011

  57. [65]

    Finediving: A fine-grained dataset for procedure-aware action quality assessment,

    J. Xu, Y . Rao, X. Yu, G. Chen, J. Zhou, and J. Lu, “Finediving: A fine-grained dataset for procedure-aware action quality assessment,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 2949–2958

    2022

  58. [66]

    Win-fail action recognition,

    P. Parmar and B. Morris, “Win-fail action recognition,” inProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2022, pp. 161–171

    2022

  59. [67]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” inPro- ceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technolog...

    2019

  60. [68]

    Gated multimodal units for information fusion,

    J. Arevalo, T. Solorio, M. Montes-y G ´omez, and F. A. Gonz ´alez, “Gated multimodal units for information fusion,”arXiv preprint arXiv:1702.01992, 2017

    2017 arXiv

  61. [69]

    Bleu: a method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” inACL, 2002, pp. 311– 318

    2002

  62. [70]

    Meteor: An automatic metric for mt evaluation with improved correlation with human judgments,

    S. Banerjee and A. Lavie, “Meteor: An automatic metric for mt evaluation with improved correlation with human judgments,” inACL workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, 2005, pp. 65–72

    2005

  63. [71]

    Cider: Consensus- based image description evaluation,

    R. Vedantam, C. Lawrence Zitnick, and D. Parikh, “Cider: Consensus- based image description evaluation,” inCVPR, 2015, pp. 4566–4575

    2015

  64. [72]

    ORANGE: a method for evaluating automatic evaluation metrics for machine translation,

    C.-Y . Lin and F. J. Och, “ORANGE: a method for evaluating automatic evaluation metrics for machine translation,” inCOLING 2004: Proceedings of the 20th International Conference on Computational Linguistics. Geneva, Switzerland: COLING, aug 23–aug 27 2004, pp. 501–507. [Online...

    2004

  65. [73]

    Lavender: Unifying video-language understanding as masked language modeling,

    L. Li, Z. Gan, K. Lin, C.-C. Lin, Z. Liu, C. Liu, and L. Wang, “Lavender: Unifying video-language understanding as masked language modeling,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 23 119–23 129

    2023

  66. [74]

    Ricaˆ 2: Rubric- informed, calibrated assessment of actions,

    A. Majeedi, V . R. Gajjala, S. S. S. N. GNVV , and Y . Li, “Ricaˆ 2: Rubric- informed, calibrated assessment of actions,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 143–161

    2024

  67. [75]

    Learning spatiotemporal features with 3d convolutional networks,

    D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, “Learning spatiotemporal features with 3d convolutional networks,” inProceedings of the IEEE International Conference on Computer Vision (ICCV), December 2015

    2015

  68. [76]

    Tsm: Temporal shift module for efficient video understanding,

    J. Lin, C. Gan, and S. Han, “Tsm: Temporal shift module for efficient video understanding,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019

    2019

  69. [77]

    Is space-time attention all you need for video understanding?

    G. Bertasius, H. Wang, and L. Torresani, “Is space-time attention all you need for video understanding?” inIcml, vol. 2, no. 3, 2021, p. 4

    2021

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.