Pith. sign in

REVIEW 3 major objections 5 minor 81 references

Bridging the Skeleton-Text Modality Gap: Diffusion-Powered Modality Alignment for Zero-shot Skeleton-based Action Recognition

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read For zero-shot skeleton action recognition, text-guided denoising beats direct alignment of skeleton and text spaces.

desk verdict New diffusion-based alignment recipe for zero-shot skeleton action recognition with big reported gains; the diffusion-specific attribution and ttest selection need hardening before I'd trust the headline margins. read the letter →

arxiv 2411.10745 v4 pith:HI4YBZLQ submitted 2024-11-16 cs.CV

Jeonghyeok Do , Munchurl Kim This is my paper · ORCID
classification cs.CV
keywords zero-shotactionrecognitionskeleton-baseddiffusionmodelsmodalityalignmenttext-conditioneddenoisingtripletlossskeleton-textmatchingtransformer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the right way to bridge skeleton and text spaces for zero-shot action recognition is not to align them directly but to let text guide the denoising of skeleton features inside a diffusion process. The proposed framework, TDSM, trains a diffusion transformer to predict, from a noisy skeleton feature, the noise that was added, with a text prompt acting as the conditioning signal. At inference, an unseen skeleton sample is corrupted with a fixed Gaussian noise, and the label whose text condition produces the smallest residual $\|\epsilon_{\mathrm{test}} - \hat{\epsilon}_k\|^2$ is chosen. The claims are backed by reported accuracy gains of 2.36 to 13.05 percentage points over recent methods on NTU-60, NTU-120, PKU-MMD, and Kinetics-200/400. If the paper is right, diffusion models can serve as a general cross-modal alignment tool for zero-shot recognition even in small, domain-specific data regimes.

What carries the argument

The central object is TDSM (Triplet Diffusion for Skeleton-Text Matching), in which a diffusion transformer conditioned on both global and local text features predicts the noise added to a skeleton feature. The load-bearing identity is the one-step residual score $\|\epsilon_{\mathrm{test}} - \hat{\epsilon}_k\|^2$: the denoising residual is treated as a similarity score between a skeleton and a text label. The text-conditioned reverse diffusion performs the modality alignment, while the triplet diffusion loss enforces discrimination by comparing residuals of positive and negative text conditions. The framework deliberately uses the diffusion model's conditioning machinery rather than its generative path, so inference is a single forward pass at a fixed timestep.

What would settle it

A concrete check is to take the trained model, draw many random noise samples for a held-out unseen class, and verify that the correct label's residual is systematically the smallest among all candidate labels; the prediction rule fails if it is not. A second check is to set the inference timestep on a validation split instead of the test accuracy curves and measure the resulting accuracy drop.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a diffusion model used purely as an alignment mechanism, rather than as a generator, can map skeleton features and text prompts into a unified latent space where unseen action labels rank correctly. The model is trained on seen classes only, with a diffusion loss $\mathcal{L}_{\mathrm{diff}} = \|\epsilon - \hat{\epsilon}_p\|^2$ on the ground-truth label and a triplet diffusion loss $\mathcal{L}_{\mathrm{TD}} = \max(\|\epsilon - \hat{\epsilon}_p\|^2 - \|\epsilon - \hat{\epsilon}_n\|^2 + \tau, 0)$ that pulls correct skeleton-text pairs together and pushes wrong pairs apart. At inference, a fixed noise $\epsilon_{\mathrm{test}}$ and a fixed timestep $t_{\mathrm{test}} = 25$ are used, and the predicted label is $\arg\min_k \|\epsilon_{\mathrm{test}} - \hat{\epsilon}_k\|^2$. The paper attributes the gain to implicit alignment: the reverse diffusion process fuses the modalities under text guidance, avoiding the direct point-wise alignment that prior VAE-based and contrastive methods rely on.

Load-bearing premise

The load-bearing premise is that, for a fixed Gaussian noise and a fixed timestep chosen from test accuracy curves, the one-step denoising residual ranks unseen action labels correctly even though the diffusion model was trained only on seen classes.

Editorial extensions

If this is right

  • Unseen action labels can be ranked by a one-step diffusion residual, so zero-shot inference needs no iterative sampling and no generated features.
  • The same framework transfers across standard, extreme, and body-part-based splits on NTU-60, NTU-120, PKU-MMD, and Kinetics, with reported gains of 2.36 to 13.05 percentage points over recent methods.
  • The triplet diffusion loss carries the discriminative part of the gain: combining it with the diffusion loss turns roughly 53-58 percent top-1 accuracy into 56-65 percent on the harder NTU-120 splits.
  • A single text prompt per action suffices; on Kinetics-200 and Kinetics-400, one-prompt TDSM outperforms a seven-prompt baseline.
  • Random Gaussian noise during training acts as a regularizer, with fixed noise cutting accuracy by roughly ten points.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not claim this, but the same residual-ranking recipe should transfer to other skeleton-text tasks such as retrieval or few-shot classification, since the alignment mechanism is not tied to the action-label format.
  • The paper mentions but does not develop the refinement that predicting the clean skeleton latent instead of the noise reduces inference fluctuation by a factor of five; that is an extension toward a more stable variant.
  • A testable implication beyond the reported experiments is that because the fixed inference timestep was chosen from test accuracy curves, selecting it on a validation split could change the reported margins; the paper does not measure that.
  • A further semantic probe, not run in the paper, would be to perturb a text prompt toward a wrong action and check that the diffusion residual increases monotonically, which would confirm the unified space is genuinely semantic.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The manuscript proposes TDSM, a diffusion-based method for zero-shot skeleton-based action recognition (ZSAR). Skeleton features are extracted with a frozen GCN encoder, and text is encoded with CLIP into global and local features. In training, a DiT-based denoiser is conditioned on positive and negative text prompts and is trained with a combination of a diffusion loss and a triplet diffusion loss. At inference, each unseen skeleton is noised once with a fixed Gaussian noise vector at a fixed timestep, the denoiser predicts the noise for each candidate label, and the label is selected by minimizing the residual distance ||epsilon_test - epsilon_hat_k||^2. The paper reports consistent gains over several recent methods on NTU-60, NTU-120, PKU-MMD, and Kinetics under SynSE, PURLS, and SMIE protocols, with ablations over loss terms, text feature types, total timesteps, and noise randomization.

Significance. The paper introduces a new formulation for ZSAR by converting cross-modal alignment into a text-conditioned denoising task, and the empirical results are strong if the protocol concerns are resolved. It also has positive methodological hygiene: the random seed is fixed, results are averaged over ten noise realizations, and the same skeleton encoders and text prompts as prior work are used for comparisons. The Kinetics results and the U-Net versus DiT comparison are useful additions. The significance is currently conditional: without a direct-alignment control, the claimed advantage of the diffusion mechanism over standard alignment is not established, and the test-set selection of the inference timestep weakens the reported margins.

major comments (3)
  1. [§5.4, Tables 3–6] The paper's central claim is that conditioning on text during the reverse diffusion process, rather than directly aligning skeleton and text latent spaces, is what produces the gains. The ablations vary only components inside the DiT denoiser, such as the loss terms, text features, total timesteps, and noise randomization. There is no control that replaces the diffusion denoiser with a direct alignment head, for example a linear projection or MLP trained on the same frozen skeleton encoder, the same CLIP text features, the same triplet-style loss, and the same hyperparameters. Without such a control, the large margins over prior methods cannot be attributed to the diffusion mechanism, and the central 'diffusion-powered alignment' claim remains unsupported.
  2. [§5.2, Fig. 4] The inference timestep t_test = 25 is selected from test-accuracy trends shown in Fig. 4, and the same figure shows that accuracy varies by several points across t_test and across noise realizations. Because t_test is a free hyperparameter tuned on the test set, the headline gains in Tables 1 and 2 may be inflated. The authors should either choose t_test on a held-out validation split or report all benchmark numbers for a range of t_test values and show that the margins over the second-best methods are stable across that range. The supplementary statement in §B.4 that t_test = T/2 is empirically best is not a substitute for validation-based selection.
  3. [§4.4, Eq. (15)] The inference rule assumes that ||epsilon_test - epsilon_hat_k||^2 is a reliable compatibility score for unseen labels. Training enforces residual margins only for random seen negatives, and no argument or diagnostic establishes that the correct unseen label systematically yields the smallest residual. The paper should provide a ranking diagnostic on unseen classes, such as the mean rank of the correct label or the separation between correct and incorrect residuals, and ideally compare Eq. (15) with alternative scoring functions, for example predicting the clean skeleton feature rather than the noise. The supplementary's own noise-sensitivity analysis in §A.1 makes this validation particularly important.
minor comments (5)
  1. [Tables 1–6] The tables report only mean accuracies without standard deviations; given that Fig. 4 shows up to about ±2.5% noise-induced variation, the 10-run averages should be accompanied by error bars or confidence intervals.
  2. [Fig. 1] The word 'Non' appears in the negative-pair caption of the overview figure; this appears to be a typo for 'No' or 'Wrong', and it should be corrected.
  3. [Table 5] The claim that T = 50 is the best total timestep is not uniformly supported by the table, since T = 100 gives a higher accuracy on the NTU-60 48/12 split (56.27 versus 56.03); the authors should reconcile this observation with their conclusion.
  4. [Supp. §E] The main text says hyperparameters were set 'through empirical validation', while the supplementary says they were tuned on the NTU-60 SynSE benchmark; the main text should state the validation protocol explicitly so that the t_test tuning concern is not compounded by ambiguity about other hyperparameters.
  5. [Supp. Table 9] The comparison with BSZSL uses a method that also consumes RGB input; this should be clearly described as an additional reference point rather than a direct comparison, since the modality difference is material.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation; the diffusion-residual scoring rule is a direct test-time application of the training objective, and the only self-citation is non-load-bearing.

full rationale

The claimed prediction rule, Eq. (15), ranks unseen labels by the denoising residual \|eps_test - eps_hat_k\|^2, which is the same quantity minimized by the training loss Ldiff in Eq. (9) and separated by the triplet term in Eq. (10). This is an inductive inference criterion, not a tautology: the model is trained only on seen classes, and whether the learned residual ranking transfers to unseen labels is an empirical question that the benchmark tables address. No equation reduces the reported gain to a fitted parameter by construction. The choice ttest = 25 is selected from test-accuracy curves (Sec. 5.2, Fig. 4), which is a test-set selection concern that can inflate reported numbers, but it is not a fitted parameter renamed as a prediction and does not make the central claim true by definition. The only same-author reference, [12], appears in a general list of skeleton transformer works and is not used to justify the diffusion-alignment mechanism; the architectural premise is attributed to external DiT and CLIP works. The absence of a direct-alignment control baseline weakens causal attribution of gains to the diffusion mechanism, but that is an experimental-validity gap, not circular reasoning. No self-definitional, fitted-input-as-prediction, or self-citation-load-bearing step could be identified from the paper's equations and text.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The method is an empirical pipeline whose performance depends on frozen pretrained encoders, learned diffusion transformer weights, and the hyperparameters listed above. The most consequential choices are ttest=25, selected from test curves, and the assumption that noise-prediction residual is a valid matching score. No new physical or conceptual entities are introduced.

free parameters (6)
  • inference timestep ttest = 25
    Selected from test-set accuracy trends in Fig. 4; directly controls all reported results and is a post-hoc choice.
  • total diffusion timesteps T = 50
    Chosen by ablation in Table 5; affects the training noise distribution.
  • TD loss weight lambda = 1.0
    Set by 'empirical validation' in Sec. 5.2.
  • triplet margin tau = 1.0
    Set by 'empirical validation' in Sec. 5.2.
  • feature dimension C, blocks B, heads = C=768, B=12, heads=12
    Architecture choices from Supplementary E; not derived from data.
  • skeleton token count Mx = 1
    Design choice to avoid higher-resolution skeleton features, stated in Supplementary E.
assumptions (6)
  • domain assumption Seen and unseen actions share skeletal movement patterns that permit zero-shot generalization.
    Stated in Sec. 1 as the basis for ZSAR.
  • domain assumption A CLIP text encoder provides semantically meaningful global and local features for action labels.
    Used in Sec. 4.2, Eq. (5), without validation on the skeleton domain.
  • domain assumption A skeleton encoder trained with cross-entropy on seen classes and then frozen yields features that transfer to unseen actions.
    Sec. 4.2 describes this training protocol, which is central to the evaluation.
  • domain assumption One-step reverse diffusion at ttest=T/2 with fixed Gaussian noise is distributionally consistent with training.
    Supplementary B.4 asserts no mismatch but provides no formal proof.
  • ad hoc to paper The residual ||eps_test - eps_hat_k||^2 is a valid compatibility score for unseen labels.
    Underpins inference Eq. (15); assumed rather than derived.
  • standard math Diffusion forward process reparameterization xt = sqrt(alpha_t)x0 + sqrt(1-alpha_t)eps.
    Standard DDPM result used in Eq. (6).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bridging the Skeleton-Text Modality Gap: Diffusion-Powered Modality Alignment for Zero-shot Skeleton-based Action Recognition." pith.science (2026). https://pith.science/paper/HI4YBZLQ

@misc{pith2026241110745,
  author       = {Pith},
  title        = {Pith review of: Bridging the Skeleton-Text Modality Gap: Diffusion-Powered Modality Alignment for Zero-shot Skeleton-based Action Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HI4YBZLQ}},
  note         = {Machine review of arXiv:2411.10745}
}
read the original abstract

In zero-shot skeleton-based action recognition (ZSAR), aligning skeleton features with the text features of action labels is essential for accurately predicting unseen actions. ZSAR faces a fundamental challenge in bridging the modality gap between the two-kind features, which severely limits generalization to unseen actions. Previous methods focus on direct alignment between skeleton and text latent spaces, but the modality gaps between these spaces hinder robust generalization learning. Motivated by the success of diffusion models in multi-modal alignment (e.g., text-to-image, text-to-video), we firstly present a diffusion-based skeleton-text alignment framework for ZSAR. Our approach, Triplet Diffusion for Skeleton-Text Matching (TDSM), focuses on cross-alignment power of diffusion models rather than their generative capability. Specifically, TDSM aligns skeleton features with text prompts by incorporating text features into the reverse diffusion process, where skeleton features are denoised under text guidance, forming a unified skeleton-text latent space for robust matching. To enhance discriminative power, we introduce a triplet diffusion (TD) loss that encourages our TDSM to correct skeleton-text matches while pushing them apart for different action classes. Our TDSM significantly outperforms very recent state-of-the-art methods with significantly large margins of 2.36%-point to 13.05%-point, demonstrating superior accuracy and scalability in zero-shot settings through effective skeleton-text matching.

Figures

Figures reproduced from arXiv: 2411.10745 by the authors.

Figure 1
Figure 1. Overview of our Triplet Diffusion for Skeleton-Text [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Training framework of our TDSM for zero-shot skeleton-based action recognition. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Inference framework of our TDSM for ZSAR. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Effect of varying inference timesteps ttest across multiple datasets. Each plot shows the top-1 accuracy trend on the NTU-60 and NTU-120 datasets under different splits. The solid red line represents the average accuracy of our method, with the shaded orange area indic…
Figure 5
Figure 5. Figure 5: Effect of varying inference timesteps ttest across multiple datasets. Each plot shows the top-1 accuracy trend on the NTU-60 and NTU-120 datasets under different splits. The solid red line represents the average accuracy of our method, with the shaded orange area indic…
Figure 6
Figure 6. Figure 6: Confusion matrix and per-class top-1 accuracy visualization for NTU-60 55/5 Split 2. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Confusion matrix and per-class top-1 accuracy visualization for PKU-MMD 46/5 Split 1. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: A detail structure of our CrossDiT Block. [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

81 extracted references · 49 canonical work pages

  1. [1]

    Language models are few-shot learners

    Tom B Brown. Language models are few-shot learners. arXiv preprint arXiv:2005.14165, 2020. 2, 12

    arXiv 2005

  2. [2]

    Peekaboo: Text to image diffusion models are zero-shot segmentors

    Ryan Burgert, Kanchana Ranasinghe, Xiang Li, and Michael S Ryoo. Peekaboo: Text to image diffusion models are zero-shot segmentors. arXiv preprint arXiv:2211.13224,

    arXiv

  3. [3]

    Ske2grid: Skeleton-to-grid representation learning for action recognition

    Dongqi Cai, Yangyuxuan Kang, Anbang Yao, and Yurong Chen. Ske2grid: Skeleton-to-grid representation learning for action recognition. In International Conference on Machine Learning, 2023. 13

    2023

  4. [4]

    Realtime multi-person 2d pose estimation using part affinity fields

    Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. Realtime multi-person 2d pose estimation using part affinity fields. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7291–7299, 2017. 1

    2017

  5. [5]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on ma- chine learning, pages 1597–1607. PMLR, 2020. 2

    2020

  6. [6]

    Executing your commands via motion diffusion in latent space

    Xin Chen, Biao Jiang, Wen Liu, Zilong Huang, Bin Fu, Tao Chen, and Gang Yu. Executing your commands via motion diffusion in latent space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 18000–18010, 2023. 11

    work page 2023

  7. [7]

    Channel-wise topology refinement graph convolution for skeleton-based action recognition

    Yuxin Chen, Ziqi Zhang, Chunfeng Yuan, Bing Li, Ying Deng, and Weiming Hu. Channel-wise topology refinement graph convolution for skeleton-based action recognition. In Proceedings of the IEEE/CVF international conference on computer vision, pages 13359–13368, 2021. 1, 13

    work page 2021

  8. [8]

    Fine-grained side information guided dual-prompts for zero-shot skeleton action recognition

    Yang Chen, Jingcai Guo, Tian He, and Ling Wang. Fine-grained side information guided dual-prompts for zero-shot skeleton action recognition. arXiv preprint arXiv:2404.07487, 2024. 1, 2, 7

    arXiv 2024

Show all 81 references
  1. [9]

    Skeleton-based action recognition with shift graph convolutional network

    Ke Cheng, Yifan Zhang, Xiangyu He, Weihan Chen, Jian Cheng, and Hanqing Lu. Skeleton-based action recognition with shift graph convolutional network. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 183–192, 2020. 1, 6, 12, 13

    2020

  2. [10]

    In- fogcn: Representation learning for human skeleton-based action recognition

    Hyung-gun Chi, Myoung Hoon Ha, Seunggeun Chi, Sang Wan Lee, Qixing Huang, and Karthik Ramani. In- fogcn: Representation learning for human skeleton-based action recognition. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 20186–201...

    2022

  3. [11]

    Text-to-image diffusion mod- els are zero shot classifiers

    Kevin Clark and Priyank Jaini. Text-to-image diffusion mod- els are zero shot classifiers. Advances in Neural Information Processing Systems, 36, 2024. 3

    2024

  4. [12]

    Skateformer: Skeletal- temporal transformer for human action recognition

    Jeonghyeok Do and Munchurl Kim. Skateformer: Skeletal- temporal transformer for human action recognition. arXiv preprint arXiv:2403.09508, 2024. 1, 13

    2024 arXiv

  5. [13]

    Revisiting skeleton-based action recognition

    Haodong Duan, Yue Zhao, Kai Chen, Dahua Lin, and Bo Dai. Revisiting skeleton-based action recognition. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2969–2978, 2022. 1, 13

    2022

  6. [14]

    Skeletr: Towards skeleton-based action recognition in the wild

    Haodong Duan, Mingze Xu, Bing Shuai, Davide Mod- olo, Zhuowen Tu, Joseph Tighe, and Alessandro Bergamo. Skeletr: Towards skeleton-based action recognition in the wild. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 13634–13644, 2023. 13

    2023

  7. [15]

    Scaling recti- fied flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. In Forty-first International Conference on Mach...

    2024

  8. [16]

    Zero-shot action recognition in videos: A survey

    Valter Estevam, Helio Pedrini, and David Menotti. Zero-shot action recognition in videos: A survey. Neurocomputing, 439:159–175, 2021. 1

    2021

  9. [17]

    Action detection via an image diffusion process

    Lin Geng Foo, Tianjiao Li, Hossein Rahmani, and Jun Liu. Action detection via an image diffusion process. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18351–18361, 2024. 11

    2024

  10. [18]

    De- vise: A deep visual-semantic embedding model

    Andrea Frome, Greg S Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Marc’Aurelio Ranzato, and Tomas Mikolov. De- vise: A deep visual-semantic embedding model. Advances in neural information processing systems, 26, 2013. 10

    2013

  11. [19]

    Diffpose: Toward more reliable 3d pose estimation

    Jia Gong, Lin Geng Foo, Zhipeng Fan, Qiuhong Ke, Hos- sein Rahmani, and Jun Liu. Diffpose: Toward more reliable 3d pose estimation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 13041–13051, 2023. 11

    2023

  12. [20]

    Syntactically guided generative embeddings for zero-shot skeleton action recognition

    Pranay Gupta, Divyanshu Sharma, and Ravi Kiran Sarvadev- abhatla. Syntactically guided generative embeddings for zero-shot skeleton action recognition. In2021 IEEE Interna- tional Conference on Image Processing (ICIP) , pages 439–

  13. [21]

    Lotus: Diffusion-based visual foundation model for high-quality dense prediction

    Jing He, Haodong Li, Wei Yin, Yixun Liang, Leheng Li, Kaiqiang Zhou, Hongbo Liu, Bingbing Liu, and Ying- Cong Chen. Lotus: Diffusion-based visual foundation model for high-quality dense prediction. arXiv preprint arXiv:2409.18124, 2024. 9

    2024 arXiv

  14. [22]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 3

    2020

  15. [23]

    Deep metric learning using triplet network

    Elad Hoffer and Nir Ailon. Deep metric learning using triplet network. In Similarity-based pattern recognition: third inter- national workshop, SIMBAD 2015, Copenhagen, Denmark, October 12-14, 2015. Proceedings 3, pages 84–92. Springer,

    2015

  16. [24]

    Diffpose: Multi- hypothesis human pose estimation using diffusion models

    Karl Holmquist and Bastian Wandt. Diffpose: Multi- hypothesis human pose estimation using diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 15977–15987, 2023. 11

    2023

  17. [25]

    Stablemofusion: Towards robust and efficient diffusion-based motion generation framework

    Yiheng Huang, Hui Yang, Chuanchen Luo, Yuxi Wang, Shibiao Xu, Zhaoxiang Zhang, Man Zhang, and Jun- ran Peng. Stablemofusion: Towards robust and efficient diffusion-based motion generation framework. In Proceed- ings of the 32nd ACM International Conference on Multime- dia, pag...

    2024

  18. [26]

    Learning robust visual-semantic embed- dings

    Yao-Hung Hubert Tsai, Liang-Kang Huang, and Ruslan Salakhutdinov. Learning robust visual-semantic embed- dings. In Proceedings of the IEEE International conference on Computer Vision, pages 3571–3580, 2017. 7, 10

    2017

  19. [27]

    Open clip, 2021

    Gabriel Ilharco, Mitchell Wortsman, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok 14 Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. Open clip, 2021. 5, 6, 12

    2021

  20. [28]

    The kinetics hu- man action video dataset

    Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, et al. The kinetics hu- man action video dataset. arXiv preprint arXiv:1705.06950,

    arXiv

  21. [29]

    Learning clip representations for skeleton-based 3d action recognition

    Qiuhong Ke, Mohammed Bennamoun, Senjian An, Ferdous Sohel, and Farid Boussaid. Learning clip representations for skeleton-based 3d action recognition. IEEE Transactions on Image Processing, 27(6):2842–2855, 2018. 13

    2018

  22. [30]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 2

    2013 arXiv

  23. [31]

    Human action recognition and predic- tion: A survey

    Yu Kong and Yun Fu. Human action recognition and predic- tion: A survey. International Journal of Computer Vision , 130(5):1366–1401, 2022. 1

    2022

  24. [32]

    Zero-shot skeleton-based action recognition with dual visual-text alignment

    Jidong Kuang, Hongsong Wang, Chaolei Han, and Jie Gui. Zero-shot skeleton-based action recognition with dual visual-text alignment. arXiv preprint arXiv:2409.14336 ,

    arXiv

  25. [33]

    Leveraging spatio- temporal dependency for skeleton-based action recognition

    Jungho Lee, Minhyeok Lee, Suhwan Cho, Sungmin Woo, Sungjun Jang, and Sangyoun Lee. Leveraging spatio- temporal dependency for skeleton-based action recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10255–10264, 2023. 13

    2023

  26. [34]

    Hierarchically decomposed graph convolutional net- works for skeleton-based action recognition

    Jungho Lee, Minhyeok Lee, Dogyoon Lee, and Sangyoun Lee. Hierarchically decomposed graph convolutional net- works for skeleton-based action recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 10444–10453, 2023. 13

    2023

  27. [35]

    Your diffusion model is secretly a zero-shot classifier

    Alexander C Li, Mihir Prabhudesai, Shivam Duggal, Ellis Brown, and Deepak Pathak. Your diffusion model is secretly a zero-shot classifier. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 2206–2217,

  28. [36]

    Multi-semantic fusion model for generalized zero-shot skeleton-based action recognition

    Ming-Zhe Li, Zhen Jia, Zhang Zhang, Zhanyu Ma, and Liang Wang. Multi-semantic fusion model for generalized zero-shot skeleton-based action recognition. InInternational Conference on Image and Graphics, pages 68–80. Springer,

  29. [37]

    Independently recurrent neural network (indrnn): Building a longer and deeper rnn

    Shuai Li, Wanqing Li, Chris Cook, Ce Zhu, and Yanbo Gao. Independently recurrent neural network (indrnn): Building a longer and deeper rnn. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 5457–5466, 2018. 13

    2018

  30. [38]

    Sa-dvae: Im- proving zero-shot skeleton-based action recognition by disentangled variational autoencoders

    Sheng-Wei Li, Zi-Xiang Wei, Wei-Jie Chen, Yi-Hsin Yu, Chih-Yuan Yang, and Jane Yung-jen Hsu. Sa-dvae: Im- proving zero-shot skeleton-based action recognition by disentangled variational autoencoders. arXiv preprint arXiv:2407.13460, 2024. 1, 2, 7

    2024 arXiv

  31. [39]

    Pku-mmd: A large scale benchmark for continu- ous multi-modal human action understanding.arXiv preprint arXiv:1703.07475, 2017

    Chunhui Liu, Yueyu Hu, Yanghao Li, Sijie Song, and Jiay- ing Liu. Pku-mmd: A large scale benchmark for continu- ous multi-modal human action understanding.arXiv preprint arXiv:1703.07475, 2017. 6, 12

    2017 arXiv

  32. [40]

    Beyond-skeleton: Zero-shot skeleton action recognition enhanced by supplementary rgb visual in- formation

    Hongjie Liu, Yingchun Niu, Kun Zeng, Chun Liu, Mengjie Hu, and Qing Song. Beyond-skeleton: Zero-shot skeleton action recognition enhanced by supplementary rgb visual in- formation. Expert Systems with Applications , 273:126814,

  33. [41]

    Spatio-temporal lstm with trust gates for 3d human action recognition

    Jun Liu, Amir Shahroudy, Dong Xu, and Gang Wang. Spatio-temporal lstm with trust gates for 3d human action recognition. In Computer Vision–ECCV 2016: 14th Euro- pean Conference, Amsterdam, The Netherlands, October 11- 14, 2016, Proceedings, Part III 14, pages 816–833. Springer,

    2016

  34. [42]

    Ntu rgb+ d 120: A large- scale benchmark for 3d human activity understanding

    Jun Liu, Amir Shahroudy, Mauricio Perez, Gang Wang, Ling-Yu Duan, and Alex C Kot. Ntu rgb+ d 120: A large- scale benchmark for 3d human activity understanding. IEEE transactions on pattern analysis and machine intelligence , 42(10):2684–2701, 2019. 6

    2019

  35. [43]

    Disentangling and unifying graph convo- lutions for skeleton-based action recognition

    Ziyu Liu, Hongwen Zhang, Zhenghao Chen, Zhiyong Wang, and Wanli Ouyang. Disentangling and unifying graph convo- lutions for skeleton-based action recognition. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 143–152, 2020. 13

    2020

  36. [44]

    Sgdr: Stochas- tic gradient descent with warm restarts

    Ilya Loshchilov and Frank Hutter. Sgdr: Stochas- tic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983, 2016. 6

    2016 arXiv

  37. [45]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 6

    2017 arXiv

  38. [46]

    Igformer: Interaction graph transformer for skeleton-based human interaction recognition

    Yunsheng Pang, Qiuhong Ke, Hossein Rahmani, James Bai- ley, and Jun Liu. Igformer: Interaction graph transformer for skeleton-based human interaction recognition. In European Conference on Computer Vision , pages 605–622. Springer,

  39. [47]

    Automatic differentiation in pytorch

    Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Al- ban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017. 6

    2017

  40. [48]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 4195–4205,

  41. [49]

    Film: Visual reasoning with a general conditioning layer

    Ethan Perez, Florian Strub, Harm De Vries, Vincent Du- moulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer. In Proceedings of the AAAI con- ference on artificial intelligence, 2018. 11

    2018

  42. [50]

    3d skeleton-based human action classification: A survey

    Liliana Lo Presti and Marco La Cascia. 3d skeleton-based human action classification: A survey. Pattern Recognition, 53:130–147, 2016. 1

    2016

  43. [51]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  44. [52]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2, 3, 4

    2022

  45. [53]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. In Medical image computing and computer-assisted 15 intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings,...

    2015

  46. [54]

    Generalized zero-and few-shot learning via aligned variational autoencoders

    Edgar Schonfeld, Sayna Ebrahimi, Samarth Sinha, Trevor Darrell, and Zeynep Akata. Generalized zero-and few-shot learning via aligned variational autoencoders. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8247–8255, 2019. 1, 2, 7

    2019

  47. [55]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural In- f...

    2022

  48. [56]

    Ntu rgb+ d: A large scale dataset for 3d human activity anal- ysis

    Amir Shahroudy, Jun Liu, Tian-Tsong Ng, and Gang Wang. Ntu rgb+ d: A large scale dataset for 3d human activity anal- ysis. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1010–1019, 2016. 6

    2016

  49. [57]

    Deep high-resolution representation learning for human pose es- timation

    Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. Deep high-resolution representation learning for human pose es- timation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5693–5703,

  50. [58]

    Human action recog- nition from various data modalities: A review

    Zehua Sun, Qiuhong Ke, Hossein Rahmani, Mohammed Bennamoun, Gang Wang, and Jun Liu. Human action recog- nition from various data modalities: A review. IEEE trans- actions on pattern analysis and machine intelligence , 2022. 1

    2022

  51. [59]

    Diffuse attend and segment: Un- supervised zero-shot segmentation using stable diffusion

    Junjiao Tian, Lavisha Aggarwal, Andrea Colaco, Zsolt Kira, and Mar Gonzalez-Franco. Diffuse attend and segment: Un- supervised zero-shot segmentation using stable diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 3554–3563...

    2024

  52. [60]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017. 11

    2017

  53. [61]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. 11

    2017

  54. [62]

    3mformer: Multi-order multi- mode transformer for skeletal action recognition

    Lei Wang and Piotr Koniusz. 3mformer: Multi-order multi- mode transformer for skeletal action recognition. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5620–5631, 2023. 13

    2023

  55. [63]

    A comparative review of recent kinect-based action recognition algorithms

    Lei Wang, Du Q Huynh, and Piotr Koniusz. A comparative review of recent kinect-based action recognition algorithms. IEEE Transactions on Image Processing, 29:15–28, 2019. 1

    2019

  56. [64]

    A survey of zero-shot learning: Settings, methods, and appli- cations

    Wei Wang, Vincent W Zheng, Han Yu, and Chunyan Miao. A survey of zero-shot learning: Settings, methods, and appli- cations. ACM Transactions on Intelligent Systems and Tech- nology (TIST), 10(2):1–37, 2019. 1

    2019

  57. [65]

    Recovering realistic texture in image super-resolution by deep spatial feature transform

    Xintao Wang, Ke Yu, Chao Dong, and Chen Change Loy. Recovering realistic texture in image super-resolution by deep spatial feature transform. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 606–615, 2018. 11

    2018

  58. [66]

    Learning pyramid-structured long-range dependencies for 3d human pose estimation

    Mingjie Wei, Xuemei Xie, Yutong Zhong, and Guangming Shi. Learning pyramid-structured long-range dependencies for 3d human pose estimation. IEEE Transactions on Multi- media, 2025. 11

    2025

  59. [67]

    Fine-grained action retrieval through multiple parts- of-speech embeddings

    Michael Wray, Diane Larlus, Gabriela Csurka, and Dima Damen. Fine-grained action retrieval through multiple parts- of-speech embeddings. In Proceedings of the IEEE/CVF in- ternational conference on computer vision , pages 450–459,

  60. [68]

    Generative action description prompts for skeleton-based action recognition

    Wangmeng Xiang, Chao Li, Yuxuan Zhou, Biao Wang, and Lei Zhang. Generative action description prompts for skeleton-based action recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 10276–10285, 2023. 13

    2023

  61. [69]

    An information compensation framework for zero-shot skeleton-based action recognition

    Haojun Xu, Yan Gao, Jie Li, and Xinbo Gao. An information compensation framework for zero-shot skeleton-based action recognition. arXiv preprint arXiv:2406.00639, 2024. 1, 2, 3

    2024 arXiv

  62. [70]

    Topology-aware convolutional neural network for efficient skeleton-based action recognition

    Kailin Xu, Fanfan Ye, Qiaoyong Zhong, and Di Xie. Topology-aware convolutional neural network for efficient skeleton-based action recognition. In Proceedings of the AAAI Conference on Artificial Intelligence , pages 2866– 2874, 2022. 13

    2022

  63. [71]

    Spatial tempo- ral graph convolutional networks for skeleton-based action recognition

    Sijie Yan, Yuanjun Xiong, and Dahua Lin. Spatial tempo- ral graph convolutional networks for skeleton-based action recognition. In Proceedings of the AAAI conference on arti- ficial intelligence, 2018. 1, 6, 12, 13

    2018

  64. [72]

    Ig captioner: Information gain captioners are strong zero-shot classifiers

    Chenglin Yang, Siyuan Qiao, Yuan Cao, Yu Zhang, Tao Zhu, Alan Yuille, and Jiahui Yu. Ig captioner: Information gain captioners are strong zero-shot classifiers. arXiv preprint arXiv:2311.17072, 2023. 3

    2023 arXiv

  65. [73]

    A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Luisa Pola- nia Cabrera, Varun Jampani, Deqing Sun, and Ming-Hsuan Yang. A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence. Advances in Neural Information Processing Systems, 36, 2024. 3

    2024

  66. [74]

    View adaptive recurrent neural networks for high performance human action recog- nition from skeleton data

    Pengfei Zhang, Cuiling Lan, Junliang Xing, Wenjun Zeng, Jianru Xue, and Nanning Zheng. View adaptive recurrent neural networks for high performance human action recog- nition from skeleton data. In Proceedings of the IEEE inter- national conference on computer vision , pages 2...

  67. [75]

    Learn- ing discriminative representations for skeleton based action recognition

    Huanyu Zhou, Qingjie Liu, and Yunhong Wang. Learn- ing discriminative representations for skeleton based action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10608– 10617, 2023. 1, 13

    2023

  68. [76]

    Hypergraph trans- former for skeleton-based action recognition

    Yuxuan Zhou, Chao Li, Zhi-Qi Cheng, Yifeng Geng, Xu- ansong Xie, and Margret Keuper. Hypergraph trans- former for skeleton-based action recognition. arXiv preprint arXiv:2211.09590, 2022. 13

    2022 arXiv

  69. [77]

    Zero-shot skeleton-based action recogni- tion via mutual information estimation and maximization

    Yujie Zhou, Wenwen Qiang, Anyi Rao, Ning Lin, Bing Su, and Jiaqi Wang. Zero-shot skeleton-based action recogni- tion via mutual information estimation and maximization. In Proceedings of the 31st ACM International Conference on Multimedia, pages 5302–5310, 2023. 1, 2, 6, 7, 9, 10

    2023

  70. [78]

    Blockgcn: Redefine topology aware- ness for skeleton-based action recognition

    Yuxuan Zhou, Xudong Yan, Zhi-Qi Cheng, Yan Yan, Qi Dai, and Xian-Sheng Hua. Blockgcn: Redefine topology aware- ness for skeleton-based action recognition. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2049–2058, 2024. 1, 13 16

    2024

  71. [79]

    Part-aware unified representation of language and skele- ton for zero-shot action recognition

    Anqi Zhu, Qiuhong Ke, Mingming Gong, and James Bai- ley. Part-aware unified representation of language and skele- ton for zero-shot action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18761–18770, 2024. 1, 2, 6, 7, 9, 10

    2024

  72. [80]

    Co-occurrence fea- ture learning for skeleton based action recognition using reg- ularized deep lstm networks

    Wentao Zhu, Cuiling Lan, Junliang Xing, Wenjun Zeng, Yanghao Li, Li Shen, and Xiaohui Xie. Co-occurrence fea- ture learning for skeleton based action recognition using reg- ularized deep lstm networks. In Proceedings of the AAAI conference on artificial intelligence, 2016. 13 17

    2016

  73. [443]

    1, 2, 6, 7

    IEEE, 2021. 1, 2, 6, 7

    2021

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.