Pith. sign in

REVIEW 4 major objections 6 minor 52 references

Heterogeneous Skeleton-Based Action Representation Learning

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that one self-supervised model can learn unified action representations from both 2D 17-joint and 3D 25-joint skeletons, and that this unified approach beats prior homogeneous-data methods.

desk verdict First real attempt at heterogeneous skeleton learning; results are promising but the 3D-lifting and prompt alignment are under-tested, so treat the mechanism claim with caution. read the letter →

arxiv 2506.03481 v1 pith:PYCYV3RJ submitted 2025-06-04 cs.CV

Hongsong Wang , Xiaoyan Ma , Jidong Kuang , Jie Gui This is my paper
classification cs.CV
keywords skeleton-basedactionrecognitionheterogeneousskeletondataself-supervisedrepresentationlearningunified3Dposeestimationsemanticmotionencodingpromptretrieval
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Human skeletons arrive in different shapes: depth sensors give 3D 25-joint skeletons, while RGB pose estimators give 2D 17-joint skeletons. This paper argues that previous skeleton-action models ignore this heterogeneity and instead train separate models for each skeleton type. Its proposal is a single self-supervised framework that lifts 2D skeletons to 3D, pads both skeleton types into one prompted 30-joint canonical format using trainable skeleton-specific prompts, and trains a shared transformer with a consistency loss plus a semantic motion encoding supplied by a language model. If correct, one action-recognition encoder handles both skeleton sources without retraining and transfers across datasets, outperforming homogeneous-data baselines.

What carries the argument

The load-bearing object is the prompted unified skeleton: a fixed 30-joint canonical ordering (facial joints, common joints, head-hand-foot joints) into which each heterogeneous skeleton is placed, with trainable skeleton-specific prompt vectors ($\mathrm{prompt}_J \in \mathbb{R}^{5 \times 3}$ for the 25-joint skeleton and $\mathrm{prompt}_C \in \mathbb{R}^{10 \times 3}$ for the 17-joint skeleton) concatenated to fill the slots that a given skeleton lacks. This makes all inputs the same tensor shape $u \in \mathbb{R}^{m \times t \times 30 \times 3}$, so a single transformer encoder sees every modality. Around this core, the framework adds a 3D pose estimation loss that supervises the lifted 2D skeletons against the real 3D skeletons on common joints, and a consistency loss that pulls skeleton-specific projections toward an early-fusion embedding.

What would settle it

An experiment that reports mean per-joint error between the lifted 2D skeletons and the true 3D skeletons on the same NTU-60 sequences, especially for the three interpolated spine joints, would settle whether the assumed alignment holds.

Watch

Extended reading notes

Core claim

The paper's central claim is that data heterogeneity in skeletons is not a nuisance to normalize away but a resource: a 2D 17-joint skeleton and a 3D 25-joint skeleton can be processed into a common representation and learned jointly by one encoder. The 2D skeleton is first interpolated with three spine joints and lifted to 3D by a 4-layer MLP; skeleton-specific prompt vectors fill missing joints so both inputs become a 30-joint prompted unified skeleton. A third modality, semantic motion encoding derived from a pretrained language model's embeddings of seven direction words, is fused with the two skeleton modalities, and a feature-consistency loss among skeleton-specific projections ties the modalities together. The paper asserts this is the first framework that studies and unifies heterogeneous skeleton data, and its experiments show top-1 accuracy gains over prior state of the art on NTU-60, NTU-120, and PKU-MMD II, plus transfer to a 2D-only dataset.

Load-bearing premise

The framework's key assumption is that a 2D skeleton lifted to 3D by a small network, with three spine joints added by simple midpoint interpolation, is aligned closely enough with a true 3D skeleton that training the two sources together in one encoder helps rather than hurts.

Editorial extensions

If this is right

  • A single pretrained encoder can recognize actions from both 2D 17-joint and 3D 25-joint skeletons without per-type retraining, so a deployment can switch sensors without retraining the model.
  • Adding the lifted 2D modality and semantic motion encoding improves top-1 accuracy over prior single-skeleton methods: 87.8% on NTU-60 x-sub versus 84.4% for the 3s-UmURL baseline.
  • The frozen encoder transfers across datasets: 64.3% on PKU-MMD II after NTU-60 pretraining and 75.3% on the 2D FineGYM dataset after NTU-120 pretraining.
  • Each design piece contributes: removing 3D pose estimation, semantic motion encoding, or skeleton-specific prompts drops PKU-MMD II accuracy from 58.2% to 55.8%, 57.9%, and 57.2%, respectively.
  • The two skeleton types complement each other: 25-joint data helps hand and foot actions such as hand-waving and pointing, while 17-joint data helps head and face actions such as head-shaking and face-wiping.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The prompting trick is modular: a third skeleton topology (for example a 21-joint or 33-joint detector) could be added by defining one new prompt vector and a joint-order mapping, because the unified format is already fixed at 30 joints.
  • The weak 1%-label semi-supervised results suggest that heterogeneous skeletons need enough paired data to cohere, so collecting more multi-sensor data may matter more than further architecture changes.
  • Replacing the 4-layer MLP lifter with a stronger 3D pose estimator would test how much of the gain comes from lifting quality versus prompting and consistency training.
  • A finer motion vocabulary than the seven direction words, such as speed-weighted per-axis motion, is an untested variant that could make semantic motion encoding more informative.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper proposes a self-supervised framework for learning unified action representations from heterogeneous skeleton data, focusing on two common formats: 3D 25-joint Kinect skeletons and 2D 17-joint skeletons. The method consists of two stages: heterogeneous skeleton processing, which lifts 2D skeletons to 3D with an MLP, interpolates three spine joints, and uses trainable skeleton-specific prompts to map both skeletons into a common 30-joint format; and unified representation learning, which uses a shared Transformer encoder, a feature consistency loss, VICReg regularization, and an auxiliary semantic motion encoding derived from CLIP text embeddings of direction words. Experiments on NTU-60, NTU-120, and PKU-MMD II cover action recognition, retrieval, semi-supervised learning, and transfer learning. The central empirical claim is that one self-supervised model can recognize actions from both 2D-17-joint and 3D-25-joint skeletons without retraining, and Table 1 reports gains over prior state-of-the-art methods (e.g., 87.8 vs. 84.4 on NTU-60 x-sub for J+C+S versus 3s-UmURL).

Significance. If the results are reproducible, the paper addresses a real gap: most self-supervised skeleton-based action recognition methods assume a single homogeneous skeleton format, whereas real deployments (e.g., robots with different humanoid structures) receive skeletons with varying joint counts and coordinate dimensions. The prompt-based topology unification is simple and practical, and the semantic motion encoding is a creative way to inject static joint-name semantics into a dynamic representation. The paper also evaluates across three datasets and four downstream tasks, and it reports FLOPs, which is helpful for practitioners. However, the evidence that the specific proposed mechanism (rather than the mere addition of more input streams) drives the improvements is currently limited, and the load-bearing 3D pose estimation module is never quantitatively evaluated.

major comments (4)
  1. [§4.2, Table 1] The comparison with prior state-of-the-art methods is not fully controlled. The proposed J+C+S model adds a second skeleton source (2D HRNet detections) and a semantic stream, whereas the compared multi-modal methods such as 3s-UmURL combine joint, motion, and bone streams all derived from the same 3D skeleton. The reported margin (87.8 vs. 84.4 on NTU-60 x-sub) could therefore reflect the use of additional input data rather than the benefit of the proposed heterogeneous-skeleton unification. Please add a baseline that uses the same three input streams but with a simpler alignment, such as zero-padding the 2D skeleton to 30 joints and using late fusion of independently trained encoder heads, to isolate the contribution of the prompt-based unified skeleton and shared encoder.
  2. [§3.1, Eq. (11)] The 3D pose estimation module is load-bearing for the J+C gain because it converts the 2D skeleton into the 3D space used by the shared encoder. Yet the paper never reports a quantitative measure of lifted 3D pose quality, nor the coordinate normalization applied before the reconstruction loss in Eq. (11). The loss is computed only on common joints, and the text does not state whether the HRNet 2D skeletons and the Kinect 3D skeletons are temporally aligned and expressed in the same camera/world frame. Without this information and an alignment error metric, the reader cannot verify that the two modalities are actually comparable after lifting.
  3. [§4.3, Table 3] The ablations for the three main components (3D pose estimation, semantic motion, and skeleton-specific prompts) are confined to the PKU-MMD II dataset, with margins of 2.4, 0.3, and 1.0 percentage points, respectively. No standard deviation over multiple seeds is reported. Since this table is the only direct support for the claim that each component contributes to the overall accuracy, please report the same ablations on NTU-60 and NTU-120 and include at least three random seeds.
  4. [§4.4, Table 6] The transfer comparison on FineGYM is not informative for the paper's claims: all listed baselines use RGB or RGB+Point modalities, while the proposed method uses skeleton data. The observed superiority (75.3 vs. the best RGB method at 73.4) could be entirely due to the difference in input modality rather than to representation quality. Please compare against skeleton-based transfer methods on FineGYM, or remove this table and soften the corresponding claim.
minor comments (6)
  1. [§3.2, Eq. (7)] The symbol ⊙ is defined as concatenation but is conventionally used for element-wise product; please replace it with ⊕ or [·;·] to avoid confusion.
  2. [§3.1] The phrase "So we has a total of 30 different joints" is ungrammatical, and the derivation of the number 30 (25 + 20 − 15 shared joints) should be made explicit.
  3. [§4.3, Table 3] Please clarify in the table caption or the text that the "w/o 3D pose estimation" row uses the original 2D skeleton as the C input rather than a 3D-lifted skeleton, so that readers can correctly interpret the ablation.
  4. [§4.1 and Eq. (16)] The hyperparameter λ in the total loss is not specified in the implementation details; please report its value and, if possible, a small sensitivity study.
  5. [Figure 4] The font sizes for the class names and accuracy values in Figure 4 are very small; please enlarge them and add a clear legend for the color coding.
  6. [§5] The conclusion states that the model is limited to processing skeleton data for up to two people, but Section 4 does not describe how multi-person sequences are handled in the experiments; please clarify whether all reported results use single-person sequences or how two-person inputs are processed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported accuracies are held-out measurements of a self-supervised training framework, not consequences of fitted parameters or self-citations.

full rationale

This is an empirical systems paper: the claimed contribution is a training framework plus measured accuracies on held-out test splits of NTU-60, NTU-120, and PKU-MMD II, not a derivation of a quantity from fitted parameters. Every potentially 'circular-looking' component is an internal training objective rather than a reported prediction. The 3D-lifting MLP is trained with L_rec (Eq. 11) against the 3D joint coordinates of the same dataset, but this is standard cross-modal auxiliary supervision: the lifted 2D stream is one input to the shared encoder, and the reported downstream accuracies are computed by a frozen encoder plus a linear probe on held-out subjects (Section 4.2), so the final numbers are not forced by the fitted lifting weights or prompts. The prompt vectors and the feature-reduction projection for the CLIP direction semantics are trainable parameters fit to the training split, but nothing in the paper reports them as 'predictions'; they are internal mechanisms. The fused embedding in Eq. 8 and consistency loss Eq. 10 define the learning objective rather than being derived from the test-set result. Citations to prior work, including the co-authored USDRL [38] and early-fusion reference [31], are methodological influences, not load-bearing uniqueness or ansatz-justifying citations; the paper does not invoke any self-cited theorem to forbid alternatives. The only notable weakness is an untested 2D-to-3D alignment assumption, but an untested assumption is a correctness risk, not circularity. Accordingly, no specific reduction of a claimed result to its inputs can be exhibited, and the appropriate circularity score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 2 invented entities

The framework's central claim rests on the trainable prompts, the untested alignment between interpolated 2D spine joints and Kinect 3D joints, and the assumption that CLIP-based directional semantics help. These are all internal design choices; the paper introduces no new physical entities.

free parameters (5)
  • promptJ = learned during training
    Trainable 5 by 3 prompt vector for the 25-joint skeleton, used to fill missing joints in the unified 30-joint skeleton (Section 3.1).
  • promptC = learned during training
    Trainable 10 by 3 prompt vector for the 17-joint skeleton, used to fill missing joints.
  • semantic embedding dimension = 1
    Hand-set reduction dimension for semantic motion features (Section 3.2). The CLIP text embedding is projected to a scalar per axis.
  • VICReg loss coefficients = not stated
    Weights gamma and mu in Eqs. 12 and 15; the paper does not report their values.
  • 3D pose estimation MLP weights = learned during training
    4-layer MLP parameters trained with the reconstruction loss L_rec (Eq. 11).
assumptions (5)
  • domain assumption The midpoint of the shoulders and hips gives the spine joints of the 17-joint skeleton
    Eqs. 1-3. This anatomical simplification assumes that interpolated joints align with the Kinect 3D spine joints, forming the basis of the L_rec common-joint loss.
  • domain assumption CLIP text embeddings of direction words carry semantic information useful for action recognition
    Section 3.2. The paper assumes that the semantic motion encoding, after reduction to a scalar, improves the learned representation. The ablation on PKU-MMD II shows only a 0.3 point gain, suggesting this assumption is weakly supported.
  • domain assumption The Kinect 3D skeleton and the HRNet 2D skeleton of the same sample represent the same action with paired joints
    The consistency loss (Eq. 10) and reconstruction loss (Eq. 11) treat the two skeletons as views of one action. This holds by data collection, but assumes HRNet keypoints are correctly localized and in correspondence with the Kinect joints.
  • standard math VICReg's variance and covariance regularization prevents representational collapse in this multi-stream setting
    Eqs. 12-15. This is a published self-supervised objective applied without modification.
  • domain assumption Two single-head Transformer encoders (spatial and temporal) are sufficient to model skeleton sequences
    Section 4.1. The paper does not study encoder depth or head count; the architecture is justified only by the final accuracy.
invented entities (2)
  • Prompted unified skeleton (30-joint format)
    purpose: Unifies the 25-joint 3D and the 17-joint 2D skeletons into a common representation with skeleton-specific trainable prompts.
    Introduced in Section 3.1. Its effectiveness is evaluated only via internal ablations and downstream accuracy; there is no external falsifiable handle.
  • Semantic motion encoding
    purpose: Represents each joint's motion direction as a text-derived semantic signal to aid action representation learning.
    Defined in Section 3.2. The ablation shows a minor gain, and the encoding is only evaluated within the paper's benchmarks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Heterogeneous Skeleton-Based Action Representation Learning." pith.science (2026). https://pith.science/paper/PYCYV3RJ

@misc{pith2026250603481,
  author       = {Pith},
  title        = {Pith review of: Heterogeneous Skeleton-Based Action Representation Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PYCYV3RJ}},
  note         = {Machine review of arXiv:2506.03481}
}
read the original abstract

Skeleton-based human action recognition has received widespread attention in recent years due to its diverse range of application scenarios. Due to the different sources of human skeletons, skeleton data naturally exhibit heterogeneity. The previous works, however, overlook the heterogeneity of human skeletons and solely construct models tailored for homogeneous skeletons. This work addresses the challenge of heterogeneous skeleton-based action representation learning, specifically focusing on processing skeleton data that varies in joint dimensions and topological structures. The proposed framework comprises two primary components: heterogeneous skeleton processing and unified representation learning. The former first converts two-dimensional skeleton data into three-dimensional skeleton via an auxiliary network, and then constructs a prompted unified skeleton using skeleton-specific prompts. We also design an additional modality named semantic motion encoding to harness the semantic information within skeletons. The latter module learns a unified action representation using a shared backbone network that processes different heterogeneous skeletons. Extensive experiments on the NTU-60, NTU-120, and PKU-MMD II datasets demonstrate the effectiveness of our method in various tasks of action understanding. Our approach can be applied to action recognition in robots with different humanoid structures.

Figures

Figures reproduced from arXiv: 2506.03481 by the authors.

Figure 1
Figure 1. Comparison between our approach (right) and existing [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The structure of our framework. This framework comprises heterogeneous skeleton processing and unified representation [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Comparison between two popular human skeletons: the [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Comparison of skeletons in terms of single-class action [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

52 extracted references · 43 canonical work pages

  1. [1]

    Vi- creg: Variance-invariance-covariance regularization for self- supervised learning.arXiv preprint arXiv:2105.04906, 2021

    Adrien Bardes, Jean Ponce, and Yann LeCun. Vi- creg: Variance-invariance-covariance regularization for self- supervised learning.arXiv preprint arXiv:2105.04906, 2021. 5

    arXiv 2021

  2. [2]

    Frame-wise action representations for long videos via sequence contrastive learning

    Minghao Chen, Fangyun Wei, Chong Li, and Deng Cai. Frame-wise action representations for long videos via sequence contrastive learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13801–13810, 2022. 8

    work page 2022

  3. [3]

    Hi- erarchically self-supervised transformer for human skeleton representation learning

    Yuxiao Chen, Long Zhao, Jianbo Yuan, Yu Tian, Zhaoyang Xia, Shijie Geng, Ligong Han, and Dimitris N Metaxas. Hi- erarchically self-supervised transformer for human skeleton representation learning. InEuropean Conference on Com- puter Vision, pages 185–202. Springer, 2022. 2, 7

    work page 2022

  4. [4]

    In- fogcn: Representation learning for human skeleton-based action recognition

    Hyung-gun Chi, Myoung Hoon Ha, Seunggeun Chi, Sang Wan Lee, Qixing Huang, and Karthik Ramani. In- fogcn: Representation learning for human skeleton-based action recognition. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 20186–20196, 2022. 2, 3

    work page 2022

  5. [5]

    A large- scale study of spatiotemporal representation learning with a new benchmark on action recognition

    Andong Deng, Taojiannan Yang, and Chen Chen. A large- scale study of spatiotemporal representation learning with a new benchmark on action recognition. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 20519–20531, 2023. 8

    work page 2023

  6. [6]

    Hierarchical contrast for unsu- pervised skeleton-based action representation learning

    Jianfeng Dong, Shengkai Sun, Zhonglin Liu, Shujie Chen, Baolong Liu, and Xun Wang. Hierarchical contrast for unsu- pervised skeleton-based action representation learning. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 525–533, 2023. 2, 7, 8

    work page 2023

  7. [7]

    Representation learning of temporal dynamics for skeleton-based action recognition

    Yong Du, Yun Fu, and Liang Wang. Representation learning of temporal dynamics for skeleton-based action recognition. IEEE Transactions on Image Processing, 25(7):3010–3022,

  8. [8]

    Hyperbolic self-paced learning for self-supervised skeleton-based action representations

    Luca Franco, Paolo Mandica, Bharti Munjal, and Fabio Galasso. Hyperbolic self-paced learning for self-supervised skeleton-based action representations. InInternational Con- ference on Learning Representations, 2023. 6

    work page 2023

Show all 52 references
  1. [9]

    Contrastive learning from ex- tremely augmented skeleton sequences for self-supervised action recognition

    Tianyu Guo, Hong Liu, Zhan Chen, Mengyuan Liu, Tao Wang, and Runwei Ding. Contrastive learning from ex- tremely augmented skeleton sequences for self-supervised action recognition. InProceedings of the AAAI Conference on Artificial Intelligence, pages 762–770, 2022. 2, 3, 6, 7

    2022

  2. [10]

    Video rep- resentation learning by dense predictive coding

    Tengda Han, Weidi Xie, and Andrew Zisserman. Video rep- resentation learning by dense predictive coding. InProceed- ings of the IEEE/CVF International Conference on Com- puter Vision Workshops, pages 0–0, 2019. 8

    2019

  3. [11]

    Global-local motion transformer for unsupervised skeleton-based action learning

    Boeun Kim, Hyung Jin Chang, Jungho Kim, and Jin Young Choi. Global-local motion transformer for unsupervised skeleton-based action learning. InEuropean Conference on Computer Vision, pages 209–225. Springer, 2022. 3, 6, 7

    2022

  4. [12]

    3d human action rep- resentation learning via cross-view consistency pursuit

    Linguo Li, Minsi Wang, Bingbing Ni, Hang Wang, Jiancheng Yang, and Wenjun Zhang. 3d human action rep- resentation learning via cross-view consistency pursuit. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 4741–4750, 2021. 3, 6, 7, 8

    2021

  5. [13]

    Ms2l: Multi-task self-supervised learning for skeleton based action recognition

    Lilang Lin, Sijie Song, Wenhan Yang, and Jiaying Liu. Ms2l: Multi-task self-supervised learning for skeleton based action recognition. InProceedings of the ACM international con- ference on multimedia, pages 2490–2498, 2020. 8

    2020

  6. [14]

    Actionlet- dependent contrastive learning for unsupervised skeleton- based action recognition

    Lilang Lin, Jiahang Zhang, and Jiaying Liu. Actionlet- dependent contrastive learning for unsupervised skeleton- based action recognition. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2363–2372, 2023. 2, 6

    2023

  7. [15]

    Idempotent unsupervised representation learning for skeleton-based action recognition

    Lilang Lin, Lehong Wu, Jiahang Zhang, and Jiaying Liu. Idempotent unsupervised representation learning for skeleton-based action recognition. InEuropean Conference on Computer Vision, pages 75–92. Springer, 2025. 6

    2025

  8. [16]

    Ntu rgb+ d 120: A large- scale benchmark for 3d human activity understanding.IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(10):2684–2701, 2019

    Jun Liu, Amir Shahroudy, Mauricio Perez, Gang Wang, Ling-Yu Duan, and Alex C Kot. Ntu rgb+ d 120: A large- scale benchmark for 3d human activity understanding.IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(10):2684–2701, 2019. 5

    2019

  9. [17]

    A benchmark dataset and comparison study for multi-modal human action analytics.ACM Transactions on Multimedia Computing, Communications, and Applications, 16(2):1–24, 2020

    Jiaying Liu, Sijie Song, Chunhui Liu, Yanghao Li, and Yueyu Hu. A benchmark dataset and comparison study for multi-modal human action analytics.ACM Transactions on Multimedia Computing, Communications, and Applications, 16(2):1–24, 2020. 5

    2020

  10. [18]

    Cmd: Self-supervised 3d action representation learning with cross-modal mutual distillation

    Yunyao Mao, Wengang Zhou, Zhenbo Lu, Jiajun Deng, and Houqiang Li. Cmd: Self-supervised 3d action representation learning with cross-modal mutual distillation. InEuropean Conference on Computer Vision, pages 734–752. Springer,

  11. [19]

    Masked motion predictors are strong 3d action representation learners

    Yunyao Mao, Jiajun Deng, Wengang Zhou, Yao Fang, Wanli Ouyang, and Houqiang Li. Masked motion predictors are strong 3d action representation learners. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 10181–10191, 2023. 2

    2023

  12. [20]

    Unsupervised 3d human pose representation with viewpoint and pose disen- tanglement

    Qiang Nie, Ziwei Liu, and Yunhui Liu. Unsupervised 3d human pose representation with viewpoint and pose disen- tanglement. InEuropean Conference on Computer Vision, pages 102–118. Springer, 2020. 2

    2020

  13. [21]

    Skeleton-based action recognition via spatial and temporal transformer networks.Computer Vision and Image Under- standing, 208:103219, 2021

    Chiara Plizzari, Marco Cannici, and Matteo Matteucci. Skeleton-based action recognition via spatial and temporal transformer networks.Computer Vision and Image Under- standing, 208:103219, 2021. 5

    2021

  14. [22]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. InInternational Conference on Machine Learning,...

    2021

  15. [23]

    Self-supervised video transformer

    Kanchana Ranasinghe, Muzammal Naseer, Salman Khan, Fahad Shahbaz Khan, and Michael S Ryoo. Self-supervised video transformer. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 2874–2884, 2022. 8

    2022

  16. [24]

    Halp: Hal- lucinating latent positives for skeleton-based self-supervised learning of actions

    Anshul Shah, Aniket Roy, Ketul Shah, Shlok Mishra, David Jacobs, Anoop Cherian, and Rama Chellappa. Halp: Hal- lucinating latent positives for skeleton-based self-supervised learning of actions. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognit...

    2023

  17. [25]

    Ntu rgb+ d: A large scale dataset for 3d human activity anal- ysis

    Amir Shahroudy, Jun Liu, Tian-Tsong Ng, and Gang Wang. Ntu rgb+ d: A large scale dataset for 3d human activity anal- ysis. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1010–1019, 2016. 5

    2016

  18. [26]

    Finegym: A hierarchical video dataset for fine-grained action understand- ing

    Dian Shao, Yue Zhao, Bo Dai, and Dahua Lin. Finegym: A hierarchical video dataset for fine-grained action understand- ing. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 2616–2625,

  19. [27]

    Adversarial self-supervised learn- ing for semi-supervised 3d action recognition

    Chenyang Si, Xuecheng Nie, Wei Wang, Liang Wang, Tie- niu Tan, and Jiashi Feng. Adversarial self-supervised learn- ing for semi-supervised 3d action recognition. InEuropean Conference on Computer Visio, pages 35–51. Springer, 2020. 7

    2020

  20. [28]

    Predict & cluster: Unsupervised skeleton based action recognition

    Kun Su, Xiulong Liu, and Eli Shlizerman. Predict & cluster: Unsupervised skeleton based action recognition. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9631–9640, 2020. 2, 7

    2020

  21. [29]

    Self-supervised 3d skeleton action representation learning with motion con- sistency and continuity

    Yukun Su, Guosheng Lin, and Qingyao Wu. Self-supervised 3d skeleton action representation learning with motion con- sistency and continuity. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 13328– 13338, 2021. 2, 7

    2021

  22. [30]

    Deep high-resolution representation learning for human pose es- timation

    Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. Deep high-resolution representation learning for human pose es- timation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5693– 5703, 2019. 5

    2019

  23. [31]

    Uni- fied multi-modal unsupervised representation learning for skeleton-based action understanding

    Shengkai Sun, Daizong Liu, Jianfeng Dong, Xiaoye Qu, Junyu Gao, Xun Yang, Xun Wang, and Meng Wang. Uni- fied multi-modal unsupervised representation learning for skeleton-based action understanding. InProceedings of the ACM International Conference on Multimedia, pages 2973– 2...

    2023

  24. [32]

    Learning the predictability of the future

    D ´ıdac Sur´ıs, Ruoshi Liu, and Carl V ondrick. Learning the predictability of the future. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12607–12617, 2021. 8

    2021

  25. [33]

    Skeleton-contrastive 3d action representation learn- ing

    Fida Mohammad Thoker, Hazel Doughty, and Cees GM Snoek. Skeleton-contrastive 3d action representation learn- ing. InProceedings of the ACM international conference on multimedia, pages 1655–1663, 2021. 7, 8

    2021

  26. [34]

    Modeling temporal dynamics and spatial configurations of actions using two- stream recurrent neural networks

    Hongsong Wang and Liang Wang. Modeling temporal dynamics and spatial configurations of actions using two- stream recurrent neural networks. InProceedings of the IEEE Conference on Computer Vision and Pattern Recog- nition, pages 499–508, 2017. 2

    2017

  27. [35]

    Beyond joints: Learning representations from primitive geometries for skeleton-based action recognition and detection.IEEE Transactions on Im- age Processing, 27(9):4382–4394, 2018

    Hongsong Wang and Liang Wang. Beyond joints: Learning representations from primitive geometries for skeleton-based action recognition and detection.IEEE Transactions on Im- age Processing, 27(9):4382–4394, 2018. 2

    2018

  28. [36]

    Contrast-reconstruction representation learning for self-supervised skeleton-based action recognition.IEEE Transactions on Image Processing, 31:6224–6238, 2022

    Peng Wang, Jun Wen, Chenyang Si, Yuntao Qian, and Liang Wang. Contrast-reconstruction representation learning for self-supervised skeleton-based action recognition.IEEE Transactions on Image Processing, 31:6224–6238, 2022. 2

    2022

  29. [37]

    Molo: Motion- augmented long-short contrastive learning for few-shot ac- tion recognition

    Xiang Wang, Shiwei Zhang, Zhiwu Qing, Changxin Gao, Yingya Zhang, Deli Zhao, and Nong Sang. Molo: Motion- augmented long-short contrastive learning for few-shot ac- tion recognition. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 1...

    2023

  30. [38]

    Usdrl: Unified skeleton-based dense repre- sentation learning with multi-grained feature decorrelation

    Wanjiang Weng, Hongsong Wang, Junbo Wang, Lei He, and Guosen Xie. Usdrl: Unified skeleton-based dense repre- sentation learning with multi-grained feature decorrelation. InProceedings of the AAAI Conference on Artificial Intelli- gence, 2025. 2, 6

    2025

  31. [39]

    Scd- net: Spatiotemporal clues disentanglement network for self- supervised skeleton-based action recognition

    Cong Wu, Xiao-Jun Wu, Josef Kittler, Tianyang Xu, Sara Ahmed, Muhammad Awais, and Zhenhua Feng. Scd- net: Spatiotemporal clues disentanglement network for self- supervised skeleton-based action recognition. InProceed- ings of the AAAI Conference on Artificial Intelligence, pag...

    2024

  32. [40]

    Generative action description prompts for skeleton-based action recognition

    Wangmeng Xiang, Chao Li, Yuxuan Zhou, Biao Wang, and Lei Zhang. Generative action description prompts for skeleton-based action recognition. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 10276–10285, 2023. 2, 3

    2023

  33. [41]

    Unsupervised motion representation learning with capsule autoencoders.Advances in Neural Information Processing Systems, 34:3205–3217, 2021

    Ziwei Xu, Xudong Shen, Yongkang Wong, and Mohan S Kankanhalli. Unsupervised motion representation learning with capsule autoencoders.Advances in Neural Information Processing Systems, 34:3205–3217, 2021. 2

    2021

  34. [42]

    Spatial tempo- ral graph convolutional networks for skeleton-based action recognition

    Sijie Yan, Yuanjun Xiong, and Dahua Lin. Spatial tempo- ral graph convolutional networks for skeleton-based action recognition. InProceedings of the AAAI Conference on Ar- tificial Intelligence, 2018. 2

    2018

  35. [43]

    Skeleton cloud colorization for unsupervised 3d action representation learning

    Siyuan Yang, Jun Liu, Shijian Lu, Meng Hwa Er, and Alex C Kot. Skeleton cloud colorization for unsupervised 3d action representation learning. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 13423– 13433, 2021. 7

    2021

  36. [44]

    Contrastive positive mining for unsupervised 3d action representation learning

    Haoyuan Zhang, Yonghong Hou, Wenjing Zhang, and Wan- qing Li. Contrastive positive mining for unsupervised 3d action representation learning. InEuropean Conference on Computer Vision, pages 36–51. Springer, 2022. 2, 3, 6, 7

    2022

  37. [45]

    Hierarchi- cal consistent contrastive learning for skeleton-based action recognition with growing augmentations

    Jiahang Zhang, Lilang Lin, and Jiaying Liu. Hierarchi- cal consistent contrastive learning for skeleton-based action recognition with growing augmentations. InProceedings of the AAAI Conference on Artificial Intelligence, pages 3427– 3435, 2023. 3, 6, 7

    2023

  38. [46]

    Prompted con- trast with masked motion modeling: Towards versatile 3d ac- tion representation learning

    Jiahang Zhang, Lilang Lin, and Jiaying Liu. Prompted con- trast with masked motion modeling: Towards versatile 3d ac- tion representation learning. InProceedings of the 31st ACM International Conference on Multimedia, pages 7175–7183,

  39. [47]

    Unsupervised representation learning with long-term dynamics for skeleton based action recognition

    Nenggan Zheng, Jun Wen, Risheng Liu, Liangqu Long, Jian- hua Dai, and Zhefeng Gong. Unsupervised representation learning with long-term dynamics for skeleton based action recognition. InProceedings of the AAAI Conference on Ar- tificial Intelligence, 2018. 2, 7, 8

    2018

  40. [48]

    Hypergraph transformer for skeleton-based action recognition.arXiv preprint arXiv:2211.09590, 2022

    Yuxuan Zhou, Zhi-Qi Cheng, Chao Li, Yanwen Fang, Yifeng Geng, Xuansong Xie, and Margret Keuper. Hypergraph transformer for skeleton-based action recognition.arXiv preprint arXiv:2211.09590, 2022. 2

    2022 arXiv

  41. [49]

    Self-supervised action representation learning from partial spatio-temporal skeleton sequences

    Yujie Zhou, Haodong Duan, Anyi Rao, Bing Su, and Ji- aqi Wang. Self-supervised action representation learning from partial spatio-temporal skeleton sequences. InProceed- ings of the AAAI Conference on Artificial Intelligence, pages 3825–3833, 2023. 3, 6

    2023

  42. [50]

    Blockgcn: Redefine topology aware- ness for skeleton-based action recognition

    Yuxuan Zhou, Xudong Yan, Zhi-Qi Cheng, Yan Yan, Qi Dai, and Xian-Sheng Hua. Blockgcn: Redefine topology aware- ness for skeleton-based action recognition. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2049–2058, 2024. 2

    2024

  43. [51]

    Motionbert: A unified perspective on learning human motion representations

    Wentao Zhu, Xiaoxuan Ma, Zhaoyang Liu, Libin Liu, Wayne Wu, and Yizhou Wang. Motionbert: A unified perspective on learning human motion representations. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 15085–15099, 2023. 2

    2023

  44. [52]

    Modeling the relative visual tempo for self-supervised skeleton-based action recognition

    Yisheng Zhu, Hu Han, Zhengtao Yu, and Guangcan Liu. Modeling the relative visual tempo for self-supervised skeleton-based action recognition. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 13913–13922, 2023. 2, 6

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.