Pith. sign in

REVIEW 2 major objections 5 minor 64 references

A novel Framework for Open-Vocabulary Multi-Object Recognition using CLIP

T0 review · 2 major / 5 minor · reviewed 2026-07-15 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Skeleton sequences can be turned into images so that ImageNet-pretrained vision models learn skeleton representations without any skeleton-specific architecture.

desk verdict Solid empirical skeleton SSL paper: S2I lets you run ImageNet MAE/DiffMAE on joint sequences and get competitive numbers plus a real cross-format angle; not foundational, but useful and referee-worthy. read the letter →

arxiv 2603.05962 v2 pith:YROG3EJD submitted 2026-03-06 cs.CV

Wei Yu Chen , Ying Dai This is my paper · ORCID
classification cs.CV
keywords skeletonrepresentationlearningskeleton-to-imageencodingvision-pretrainedmodelsmaskedautoencodersself-supervisedcross-formattransferactionrecognition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Existing skeleton methods train custom networks on fixed joint layouts and cannot easily reuse the powerful models already trained on images. This paper shows that a simple reformatting step—partitioning joints into five body parts, ordering them kinematically, mapping coordinates to RGB, and resizing to 224×224—produces image-like tensors that MAE and DiffMAE can process directly. Initializing from ImageNet weights and continuing with masked reconstruction on these tensors yields competitive or state-of-the-art self-supervised results on NTU-60, NTU-120 and PKU-MMD. Because the representation is format-agnostic, the same model can be pretrained on mixed datasets that use 13-, 20- or 25-joint skeletons and still transfer across them. The practical payoff is that the large visual pretraining ecosystem becomes available for skeleton action recognition without redesigning networks or forcing joint alignment.

What carries the argument

Skeleton-to-Image Encoding (S2I): the fixed pipeline that turns any T×J×3 skeleton sequence into a 3×224×224 pseudo-image so that an off-the-shelf vision transformer can be pretrained and transferred unchanged.

What would settle it

Train the identical MAE/DiffMAE pipeline on S2I images but with randomly shuffled joint order or pure Gaussian noise instead of real coordinates; if linear-probe and fine-tune accuracies on NTU-60 collapse to chance, the encoding itself is not preserving the kinematics the method relies on.

Watch

Extended reading notes

Core claim

Skeleton-to-Image Encoding (S2I) converts a sparse 3-D joint sequence into a dense image by body-part partitioning, top-down kinematic ordering, (x,y,z)→RGB mapping and bilinear resize, enabling ImageNet-pretrained MAE and DiffMAE to perform self-supervised skeleton representation learning for the first time and to support cross-format and multi-dataset pretraining without skeleton-specific architectures.

Load-bearing premise

That mapping joint coordinates to RGB channels, grouping them into five fixed body parts and resizing to a square image still keeps enough motion structure for a vision transformer’s masked reconstruction to transfer useful ImageNet knowledge.

Editorial extensions

If this is right

  • Any future larger vision foundation model can be dropped in for skeleton pretraining without redesigning a graph or transformer backbone.
  • Heterogeneous skeleton datasets (different joint counts or sensors) can be pooled for a single universal pretraining run.
  • Multi-modal action systems can share the same visual encoder for both RGB video and skeleton streams.
  • Cross-dataset transfer no longer requires manual joint subset selection or interpolation.
  • Masked modeling recipes developed for images become immediately applicable to skeleton sequences.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the same encoding works for generative vision models, skeleton motion synthesis could inherit the same ImageNet priors.
  • The five-part partition may be replaceable by learned clustering, opening a route to entirely sensor-agnostic skeleton pretraining.
  • Because the output is an ordinary image, standard vision interpretability tools (attention maps, occlusion) can be applied directly to skeleton decisions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The manuscript proposes Skeleton-to-Image Encoding (S2I), which partitions skeleton joints into five semantic body parts, reorders them top-down along kinematic chains, maps (x,y,z) coordinates to RGB channels, stacks frames temporally, and resizes the resulting T imes J map to 224 imes224 via linear interpolation. This image-like representation allows ImageNet-pretrained MAE and DiffMAE (ViT-B) to be used directly for self-supervised skeleton pretraining via masked reconstruction (Eqs. 1–2), followed by linear probing or fine-tuning for action recognition (Eq. 3). The method is evaluated on NTU-60/120, PKU-MMD I/II, NW-UCLA, and Toyota under linear, fine-tune, semi-supervised, transfer, cross-format, and universal multi-dataset pretraining protocols, reporting competitive or SOTA numbers (especially with 3-stream joint/bone/motion fusion) without skeleton-specific architectures.

Significance. If the central claim holds, S2I provides a simple, format-agnostic bridge that lets the community reuse large-scale vision-pretrained models and weights for skeleton representation learning. The cross-format transfer (Table 8) and universal pretraining (Table 9) results are particularly valuable, because they address the long-standing problem of heterogeneous joint layouts across capture devices without manual joint subsetting or model redesign. The work also supplies extensive multi-protocol ablations (Tables 1–3) that isolate the contribution of ImageNet initialization, masking strategy, and multi-stream fusion, giving the community a concrete, reproducible baseline for vision-to-skeleton transfer.

major comments (2)
  1. Section 3.1 and Figure 2: the claim that S2I “preserves essential spatial-temporal patterns” rests on an untested assumption that bilinear resize of the T imes J map to 224 imes224 (and the fixed five-part ordering) retains the articulated kinematics that matter for action. No ablation compares S2I against alternative layouts (e.g., graph-distance ordering, no body-part partition, or different interpolation) or quantifies information loss; without such evidence the transfer gains in Tables 4–9 cannot be confidently attributed to the encoding rather than to the strong MAE/DiffMAE backbones and multi-stream fusion.
  2. Tables 4–5 and 8: several “SOTA” claims for 3s-S2I are obtained by late fusion of three independently trained streams (joint/bone/motion). The single-stream S2I numbers are competitive but not consistently superior to recent specialized methods (e.g., MAMP, MacDiff). The paper should either (a) report single-stream results as the primary comparison or (b) apply the same multi-stream fusion to the strongest baselines so that the contribution of S2I itself is isolated.
minor comments (5)
  1. The supplied abstract and title describe an unrelated OVOR/CLIP object-recognition framework; the actual manuscript is about skeleton-to-image encoding. This metadata mismatch must be corrected before any further review cycle.
  2. Section 3.3 and Table 2: “Group masking” is never formally defined; a short equation or diagram would clarify how it differs from block masking.
  3. Appendix E / Table 14: the joint-to-body-part mapping for the 13-joint Toyota skeleton collapses the spine to a single “head” joint; a brief justification of how this affects the five-part layout would help readers reproduce the universal-pretraining setting.
  4. Figure 1 and Figure 4 contain placeholder or low-resolution glyphs that render as boxes; higher-resolution versions are needed for print.
  5. Minor typos: “spatio-temporal” vs. “spatial-temporal” inconsistency; “DiffMAE’s consistently superior performance” (Table 1 shows only a 0.5–1.7 point edge).

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: empirical S2I encoding + standard MAE/DiffMAE pretraining evaluated on external public benchmarks; minor self-citations appear only as baselines.

full rationale

The paper's central claim is an engineering representation (S2I: body-part partition, kinematic reorder, (x,y,z)->RGB, bilinear resize to 224x224) that lets ImageNet-pretrained MAE/DiffMAE be applied to skeleton sequences, followed by ordinary linear-probe / fine-tune / semi-supervised / transfer evaluation on NTU-60/120, PKU-MMD, NW-UCLA and Toyota. The reconstruction losses (Eqs. 1-2) and cross-entropy (Eq. 3) are the standard MAE/DiffMAE/CE objectives; none algebraically force the reported accuracies. Ablations (Tables 1-3) and cross-format/universal results (Tables 8-9) are measured on held-out public splits. Self-citations (e.g., Colorization [40]) serve only as comparison baselines, not as uniqueness theorems or definitional premises that close a loop. The method is therefore self-contained against external benchmarks; the only residual risk is the (non-circular) empirical assumption that the pseudo-image preserves enough kinematics.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The work is empirical deep learning. It inherits ViT/MAE/DiffMAE math and ImageNet weights as standard tools, assumes five-part body semantics generalize across Kinect and LCRNet skeletons, and chooses several training hyperparameters by ablation on NTU-60. No new physical entities; free parameters are ordinary ML knobs that affect reported numbers.

free parameters (4)
  • masking_ratio = 0.75
    Chosen as 75% after NTU-60 C-sub sweep (Figure 3); default for main results.
  • pretrain_epochs_and_batch = 800/1600 epochs, bs 512
    MAE 800 epochs / DiffMAE 1600 epochs, batch 512 (768 for universal pretrain); set by authors relative to ImageNet recipes and skeleton data scale (Appendix B).
  • downstream_lr_and_optimizer = SGD 0.2 / AdamW 1e-3
    Linear probe SGD lr 0.2; fine-tune AdamW lr 1e-3 with layer decay 0.75; selected as training protocol (Section 4.2, Appendix C).
  • diffusion_rho_and_T = T=1000, ρ=1.0
    DiffMAE noise schedule uses T=1000 and ρ=1.0 by default (Appendix B).
assumptions (4)
  • domain assumption ViT-B/16 MAE and DiffMAE architectures and ImageNet-pretrained weights are valid starting points for representation learning on non-natural images.
    Invoked throughout Section 3.2 and Table 1; standard transfer assumption in modern CV.
  • domain assumption Human skeletons can be partitioned into five semantic parts (spine, left/right arm, left/right leg) with a consistent top-down kinematic order across 13-, 20-, and 25-joint formats.
    Core of S2I (Section 3.1, Appendix E Table 14); required for format-agnostic images.
  • ad hoc to paper Linear interpolation of the T×J map to 224×224 preserves task-relevant motion for masked modeling.
    Stated as the final S2I step (Section 3.1); not independently validated against other resizers or native resolutions.
  • domain assumption Standard NTU/PKU/NW-UCLA/Toyota splits and cross-subject/view/setup protocols are the right evaluation targets.
    Section 4.1; community convention.
invented entities (1)
  • Skeleton-to-Image Encoding (S2I)
    purpose: Unified image-like representation of heterogeneous skeleton sequences for vision backbones.
    Defined in Section 3.1; evaluated only inside this paper’s experiments, though falsifiable by reimplementation on public datasets.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A novel Framework for Open-Vocabulary Multi-Object Recognition using CLIP." pith.science (2026). https://pith.science/paper/YROG3EJD

@misc{pith2026260305962,
  author       = {Pith},
  title        = {Pith review of: A novel Framework for Open-Vocabulary Multi-Object Recognition using CLIP},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YROG3EJD}},
  note         = {Machine review of arXiv:2603.05962}
}
read the original abstract

To address the limitations of existing open-vocabulary object recognition methods, including high system complexity, substantial training costs, and limited generalization capability, this paper proposes a novel Open-Vocabulary Object Recognition (OVOR) framework based on a streamlined two-stage strategy: object segmentation followed by recognition. The proposed framework eliminates the need for complex retraining procedures and labor-intensive annotation. After extracting object regions, object-level image embeddings and category-level text embeddings are generated using CLIP, enabling recognition over arbitrary vocabularies. To reduce dependence on CLIP and enhance encoding flexibility, we further introduce a CNN/MLP-based approach that extracts convolutional neural network (CNN) feature maps and employs a multilayer perceptron (MLP) to align visual features with text embeddings. The resulting embeddings are then concatenated for cross-modal matching. Finally, object recognition is performed through similarity matching between image and text embeddings. Experiments on COCO, Pascal VOC, and ADE20K demonstrate that CLIP-based image encoding achieves the highest average AP, outperforming current state-of-the-art methods. Meanwhile, the results reveal the potential of CLIP-independent image encoding as a promising alternative for OVOR.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

64 extracted references · 5 linked inside Pith

  1. [1]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” inInternational Conference on Learning Representations, 2021. [Online]. Available: https://openreview.net/forum?id=YicbFdNTTy

    2021

  2. [2]

    Masked autoencoders are scalable vision learners,

    K. He, X. Chen, S. Xie, Y. Li, P . Doll ´ar, and R. Girshick, “Masked autoencoders are scalable vision learners,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 16 000–16 009

    2022

  3. [3]

    Scaling up visual and vision-language representation learning with noisy text supervision,

    C. Jia, Y. Yang, Y. Xia, Y.-T. Chen, Z. Parekh, H. Pham, Q. Le, Y.-H. Sung, Z. Li, and T. Duerig, “Scaling up visual and vision-language representation learning with noisy text supervision,” inProceedings of the 38th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, M. Meila and T. Zhang, Eds., vol. 139. PMLR, 18...

    2021

  4. [4]

    Flava: A foundational language and vision alignment model,

    A. Singh, R. Hu, V . Goswami, G. Couairon, W. Galuba, M. Rohrbach, and D. Kiela, “Flava: A foundational language and vision alignment model,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2022, pp. 15 638–15 650

    2022

  5. [5]

    Large language models can un- derstanding depth from monocular images,

    Z. Xia and T. Wu, “Large language models can un- derstanding depth from monocular images,”arXiv preprint arXiv:2409.01133, 2024

    arXiv 2024

  6. [6]

    Depth anything: Unleashing the power of large-scale unlabeled data,

    L. Yang, B. Kang, Z. Huang, X. Xu, J. Feng, and H. Zhao, “Depth anything: Unleashing the power of large-scale unlabeled data,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 10 371–10 381

    2024

  7. [7]

    Pad: Self-supervised pre-training with patchwise-scale adapter for infrared images,

    T. Zhang, K. Ding, J. Wen, Y. Xiong, Z. Zhang, S. Xi- ang, and C. Pan, “Pad: Self-supervised pre-training with patchwise-scale adapter for infrared images,” 2023

    2023

  8. [8]

    F-vita: Foundation model guided visible to thermal transla- tion,

    J. N. Paranjape, C. de Melo, and V . M. Patel, “F-vita: Foundation model guided visible to thermal transla- tion,”arXiv preprint arXiv:2504.02801, 2025

    arXiv 2025

Show all 64 references
  1. [9]

    VideoMAE: Masked autoencoders are data-efficient learners for self-supervised video pre-training,

    Z. Tong, Y. Song, J. Wang, and L. Wang, “VideoMAE: Masked autoencoders are data-efficient learners for self-supervised video pre-training,” inAdvances in Neural Information Processing Systems, 2022

    2022

  2. [10]

    Videomae v2: Scaling video masked autoencoders with dual masking,

    L. Wang, B. Huang, Z. Zhao, Z. Tong, Y. He, Y. Wang, Y. Wang, and Y. Qiao, “Videomae v2: Scaling video masked autoencoders with dual masking,” inProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2023, pp. 14 549–14 560

    2023

  3. [11]

    Omnivl: One foundation model for image-language and video- language tasks,

    J. Wang, D. Chen, Z. Wu, C. Luo, L. Zhou, Y. Zhao, Y. Xie, C. Liu, Y.-G. Jiang, and L. Yuan, “Omnivl: One foundation model for image-language and video- language tasks,”Advances in neural information process- ing systems, vol. 35, pp. 5696–5710, 2022

    2022

  4. [12]

    P2p: Tuning pre-trained image models for point cloud analysis with point-to-pixel prompting,

    Z. Wang, X. Yu, Y. Rao, J. Zhou, and J. Lu, “P2p: Tuning pre-trained image models for point cloud analysis with point-to-pixel prompting,”Advances in neural information processing systems, vol. 35, 2022

    2022

  5. [13]

    Pointclip: Point cloud understanding by clip,

    R. Zhang, Z. Guo, W. Zhang, K. Li, X. Miao, B. Cui, Y. Qiao, P . Gao, and H. Li, “Pointclip: Point cloud understanding by clip,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 8552–8562

    2022

  6. [14]

    Diffusion models as masked autoencoders,

    C. Wei, K. Mangalam, P .-Y. Huang, Y. Li, H. Fan, H. Xu, H. Wang, C. Xie, A. Yuille, and C. Feichten- hofer, “Diffusion models as masked autoencoders,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 16 284–16 294

    2023

  7. [15]

    Hierarchical recurrent neural network for skeleton based action recogni- tion,

    Y. Du, W. Wang, and L. Wang, “Hierarchical recurrent neural network for skeleton based action recogni- tion,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1110–1118

    2015

  8. [16]

    Skeleton-based action recognition using spatio-temporal lstm network with trust gates,

    J. Liu, A. Shahroudy, D. Xu, A. C. Kot, and G. Wang, “Skeleton-based action recognition using spatio-temporal lstm network with trust gates,”IEEE transactions on pattern analysis and machine intelligence, vol. 40, no. 12, pp. 3007–3021, 2017

    2017

  9. [17]

    View adaptive recurrent neural networks for high performance human action recognition from skeleton data,

    P . Zhang, C. Lan, J. Xing, W. Zeng, J. Xue, and N. Zheng, “View adaptive recurrent neural networks for high performance human action recognition from skeleton data,” inProceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2117–2126

    2017

  10. [18]

    Skeleton based action recognition with convolutional neural network,

    Y. Du, Y. Fu, and L. Wang, “Skeleton based action recognition with convolutional neural network,” in 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR). IEEE, 2015, pp. 579–583

    2015

  11. [19]

    A new representation of skeleton sequences for 3d action recognition,

    Q. Ke, M. Bennamoun, S. An, F. Sohel, and F. Bous- said, “A new representation of skeleton sequences for 3d action recognition,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 3288–3297

    2017

  12. [20]

    Co-occurrence feature learning from skeleton data for action recog- nition and detection with hierarchical aggregation,

    C. Li, Q. Zhong, D. Xie, and S. Pu, “Co-occurrence feature learning from skeleton data for action recog- nition and detection with hierarchical aggregation,” inProceedings of the 27th International Joint Conference on Artificial Intelligence, 2018, pp. 786–792

    2018

  13. [21]

    Spatial temporal graph convolutional networks for skeleton-based ac- tion recognition,

    S. Yan, Y. Xiong, and D. Lin, “Spatial temporal graph convolutional networks for skeleton-based ac- tion recognition,” inThirty-Second AAAI conference on artificial intelligence, 2018. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 12

    2018

  14. [22]

    Two- stream adaptive graph convolutional networks for skeleton-based action recognition,

    L. Shi, Y. Zhang, J. Cheng, and H. Lu, “Two- stream adaptive graph convolutional networks for skeleton-based action recognition,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019

    2019

  15. [23]

    Channel-wise topology refinement graph convolution for skeleton-based action recognition,

    Y. Chen, Z. Zhang, C. Yuan, B. Li, Y. Deng, and W. Hu, “Channel-wise topology refinement graph convolution for skeleton-based action recognition,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 13 359–13 368

    2021

  16. [24]

    Stst: Spatial-temporal specialized transformer for skeleton- based action recognition,

    Y. Zhang, B. Wu, W. Li, L. Duan, and C. Gan, “Stst: Spatial-temporal specialized transformer for skeleton- based action recognition,” inProceedings of the 29th ACM International Conference on Multimedia, 2021

    2021

  17. [25]

    Hypergraph transformer for skeleton-based action recognition,

    Y. Zhou, Z.-Q. Cheng, C. Li, Y. Fang, Y. Geng, X. Xie, and M. Keuper, “Hypergraph transformer for skeleton-based action recognition,”arXiv, 2022

    2022

  18. [26]

    3d human action representation learning via cross-view consistency pursuit,

    L. Li, M. Wang, B. Ni, H. Wang, J. Yang, and W. Zhang, “3d human action representation learning via cross-view consistency pursuit,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 4741–4750

    2021

  19. [27]

    Contrastive learning from extremely aug- mented skeleton sequences for self-supervised action recognition,

    T. Guo, H. Liu, Z. Chen, M. Liu, T. Wang, and R. Ding, “Contrastive learning from extremely aug- mented skeleton sequences for self-supervised action recognition,” inProceedings of the AAAI conference on artificial intelligence, vol. 36, no. 1, 2022, pp. 762–770

    2022

  20. [28]

    Contrastive positive mining for unsupervised 3d action represen- tation learning,

    H. Zhang, Y. Hou, W. Zhang, and W. Li, “Contrastive positive mining for unsupervised 3d action represen- tation learning,” inEuropean Conference on Computer Vision. Springer, 2022, pp. 36–51

    2022

  21. [29]

    Masked motion predictors are strong 3d action representation learners,

    Y. Mao, J. Deng, W. Zhou, Y. Fang, W. Ouyang, and H. Li, “Masked motion predictors are strong 3d action representation learners,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 10 181–10 191

    2023

  22. [30]

    Macdiff: Unified skeleton modeling with masked conditional diffusion,

    L. Wu, L. Lin, J. Zhang, Y. Ma, and J. Liu, “Macdiff: Unified skeleton modeling with masked conditional diffusion,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 110–128

    2024

  23. [31]

    Skeletonmae: Spatial-temporal masked au- toencoders for self-supervised skeleton action recog- nition,

    W. Wu, Y. Hua, C. Zheng, S. Wu, C. Chen, and A. Lu, “Skeletonmae: Spatial-temporal masked au- toencoders for self-supervised skeleton action recog- nition,” in2023 IEEE international conference on multi- media and expo workshops. IEEE, 2023, pp. 224–229

    2023

  24. [32]

    Momen- tum contrast for unsupervised visual representation learning,

    K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, “Momen- tum contrast for unsupervised visual representation learning,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020

    2020

  25. [33]

    A simple framework for contrastive learning of visual representations,

    T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” inInternational conference on machine learning. PmLR, 2020, pp. 1597–1607

    2020

  26. [34]

    Spatiotemporal contrastive video representation learning,

    R. Qian, T. Meng, B. Gong, M.-H. Yang, H. Wang, S. Belongie, and Y. Cui, “Spatiotemporal contrastive video representation learning,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 6964–6974

    2021

  27. [35]

    BEit: BERT pre-training of image transformers,

    H. Bao, L. Dong, S. Piao, and F. Wei, “BEit: BERT pre-training of image transformers,” inInternational Conference on Learning Representations, 2022

    2022

  28. [36]

    Masked feature prediction for self- supervised visual pre-training,

    C. Wei, H. Fan, S. Xie, C.-Y. Wu, A. Yuille, and C. Feichtenhofer, “Masked feature prediction for self- supervised visual pre-training,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 14 668–14 678

    2022

  29. [37]

    Dynamic multiscale graph neural networks for 3d skeleton based human motion prediction,

    M. Li, S. Chen, Y. Zhao, Y. Zhang, Y. Wang, and Q. Tian, “Dynamic multiscale graph neural networks for 3d skeleton based human motion prediction,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 214–223

    2020

  30. [38]

    Exploiting spatial-temporal relationships for 3d pose estimation via graph con- volutional networks,

    Y. Cai, L. Ge, J. Liu, J. Cai, T.-J. Cham, J. Yuan, and N. M. Thalmann, “Exploiting spatial-temporal relationships for 3d pose estimation via graph con- volutional networks,” inProceedings of the IEEE/CVF international conference on computer vision, 2019

    2019

  31. [39]

    Multi-scale semantics-guided neural net- works for efficient skeleton-based human action recognition,

    P . Zhang, C. Lan, W. Zeng, J. Xing, J. Xue, and N. Zheng, “Multi-scale semantics-guided neural net- works for efficient skeleton-based human action recognition,”arXiv preprint arXiv:2111.03993, 2021

    2021 arXiv

  32. [40]

    Skele- ton cloud colorization for unsupervised 3d action rep- resentation learning,

    S. Yang, J. Liu, S. Lu, M. H. Er, and A. C. Kot, “Skele- ton cloud colorization for unsupervised 3d action rep- resentation learning,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2021

    2021

  33. [41]

    Collaborating domain-shared and target-specific feature clustering for cross-domain 3d action recognition,

    Q. Liu and Z. Wang, “Collaborating domain-shared and target-specific feature clustering for cross-domain 3d action recognition,” inEuropean Conference on Com- puter Vision. Springer, 2022, pp. 137–155

    2022

  34. [42]

    Ntu rgb+d: A large scale dataset for 3d human activity analysis,

    A. Shahroudy, J. Liu, T.-T. Ng, and G. Wang, “Ntu rgb+d: A large scale dataset for 3d human activity analysis,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 2016

    2016

  35. [43]

    Ntu rgb+d 120: A large-scale bench- mark for 3d human activity understanding,

    J. Liu, A. Shahroudy, M. Perez, G. Wang, L. Y. Duan, and A. C. Kot, “Ntu rgb+d 120: A large-scale bench- mark for 3d human activity understanding,”IEEE Transactions on Pattern Analysis and Machine Intelli- gence, vol. 42, no. 10, pp. 2684–2701, 2020

    2020

  36. [44]

    A bench- mark dataset and comparison study for multi-modal human action analytics,

    J. Liu, S. Song, C. Liu, Y. Li, and Y. Hu, “A bench- mark dataset and comparison study for multi-modal human action analytics,”ACM Transactions on Mul- timedia Computing, Communications, and Applications (TOMM), vol. 16, no. 2, pp. 1–24, 2020

    2020

  37. [45]

    Cross- view action modeling, learning and recognition,

    J. Wang, X. Nie, Y. Xia, Y. Wu, and S.-C. Zhu, “Cross- view action modeling, learning and recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 2649–2656

    2014

  38. [46]

    Toyota smarthome: Real-world activities of daily living,

    S. Das, R. Dai, M. Koperski, L. Minciullo, L. Garattoni, F. Bremond, and G. Francesca, “Toyota smarthome: Real-world activities of daily living,” inThe IEEE International Conference on Computer Vision, 2019

    2019

  39. [47]

    Semantics-guided neural networks for efficient skeleton-based human action recognition,

    P . Zhang, C. Lan, W. Zeng, J. Xing, J. Xue, and N. Zheng, “Semantics-guided neural networks for efficient skeleton-based human action recognition,” inproceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 1112–1121

    2020

  40. [48]

    Skeleton-based action recognition with shift graph convolutional network,

    K. Cheng, Y. Zhang, X. He, W. Chen, J. Cheng, and H. Lu, “Skeleton-based action recognition with shift graph convolutional network,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 183–192

    2020

  41. [49]

    Unsupervised representation learning with JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 13 long-term dynamics for skeleton based action recogni- tion,

    N. Zheng, J. Wen, R. Liu, L. Long, J. Dai, and Z. Gong, “Unsupervised representation learning with JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 13 long-term dynamics for skeleton based action recogni- tion,” inProceedings of the AAAI conference on artificial intel...

    2021

  42. [50]

    Predict & cluster: Unsupervised skeleton based action recognition,

    K. Su, X. Liu, and E. Shlizerman, “Predict & cluster: Unsupervised skeleton based action recognition,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 9631–9640

    2020

  43. [51]

    Ms2l: Multi- task self-supervised learning for skeleton based action recognition,

    L. Lin, S. Song, W. Yang, and J. Liu, “Ms2l: Multi- task self-supervised learning for skeleton based action recognition,” inProceedings of the 28th ACM interna- tional conference on multimedia, 2020, pp. 2490–2498

    2020

  44. [52]

    Skeleton- contrastive 3d action representation learning,

    F. M. Thoker, H. Doughty, and C. G. Snoek, “Skeleton- contrastive 3d action representation learning,” inPro- ceedings of the 29th ACM international conference on multimedia, 2021, pp. 1655–1663

    2021

  45. [53]

    Global- local motion transformer for unsupervised skeleton- based action learning,

    B. Kim, H. J. Chang, J. Kim, and J. Y. Choi, “Global- local motion transformer for unsupervised skeleton- based action learning,” inEuropean conference on com- puter vision. Springer, 2022, pp. 209–225

    2022

  46. [54]

    Cmd: Self-supervised 3d action representation learning with cross-modal mutual distillation,

    Y. Mao, W. Zhou, Z. Lu, J. Deng, and H. Li, “Cmd: Self-supervised 3d action representation learning with cross-modal mutual distillation,” inEuropean Confer- ence on Computer Vision. Springer, 2022, pp. 734–752

    2022

  47. [55]

    Actionlet-dependent contrastive learning for unsupervised skeleton-based action recognition,

    L. Lin, J. Zhang, and J. Liu, “Actionlet-dependent contrastive learning for unsupervised skeleton-based action recognition,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 2363–2372

    2023

  48. [56]

    Self-supervised 3d skeleton action representation learning with motion consis- tency and continuity,

    Y. Su, G. Lin, and Q. Wu, “Self-supervised 3d skeleton action representation learning with motion consis- tency and continuity,” inProceedings of the IEEE/CVF international conference on computer vision, 2021

    2021

  49. [57]

    View-invariant skele- ton action representation learning via motion retarget- ing,

    D. Yang, Y. Wang, A. Dantcheva, L. Garattoni, G. Francesca, and F. Br ´emond, “View-invariant skele- ton action representation learning via motion retarget- ing,”International Journal of Computer Vision, vol. 132, no. 7, pp. 2351–2366, 2024

    2024

  50. [58]

    Hierarchically self- supervised transformer for human skeleton represen- tation learning,

    Y. Chen, L. Zhao, J. Yuan, Y. Tian, Z. Xia, S. Geng, L. Han, and D. N. Metaxas, “Hierarchically self- supervised transformer for human skeleton represen- tation learning,” inEuropean Conference on Computer Vision. Springer, 2022, pp. 185–202

    2022

  51. [59]

    Adversarial self-supervised learning for semi-supervised 3d action recognition,

    C. Si, X. Nie, W. Wang, L. Wang, T. Tan, and J. Feng, “Adversarial self-supervised learning for semi-supervised 3d action recognition,” inEuropean Conference on Computer Vision, 2020, pp. 35–51

    2020

  52. [60]

    Unik: A uni- fied framework for real-world skeleton-based action recognition,

    D. Yang, Y. Wang, A. Dantcheva, L. Garattoni, G. Francesca, and F. Br ´emond, “Unik: A uni- fied framework for real-world skeleton-based action recognition,”arXiv preprint arXiv:2107.08580, 2021

    2021 arXiv

  53. [61]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,”Advances in neural infor- mation processing systems, vol. 30, 2017

    2017

  54. [62]

    Layer normal- ization,

    J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normal- ization,”arXiv preprint arXiv:1607.06450, 2016

    2016 arXiv

  55. [63]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P . Abbeel, “Denoising diffusion probabilistic models,”Advances in neural information processing systems, vol. 33, pp. 6840–6851, 2020

    2020

  56. [64]

    Lcr-net++: Multi-person 2d and 3d pose detection in natural im- ages,

    G. Rogez, P . Weinzaepfel, and C. Schmid, “Lcr-net++: Multi-person 2d and 3d pose detection in natural im- ages,”IEEE transactions on pattern analysis and machine intelligence, vol. 42, no. 5, pp. 1146–1161, 2019. Siyuan Yangreceived the B.Eng. degree from Harbin Institute of ...

    2019

Pith tools

Reviewed July 15, 2026 · model on record in the stance chip above.