Pith. sign in

REVIEW 4 major objections 6 minor 67 references

How do Foundation Models Compare to Skeleton-Based Approaches for Gesture Recognition in Human-Robot Interaction?

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read On a new dataset of 15 whole-body navigation gestures, a skeleton-based graph network (HD-GCN) reaches 94.4% top-1 accuracy, a frozen video foundation model with a learned head (V-JEPA) reaches 90.1%, and a zero-shot multimodal model…

desk verdict A useful new HRI gesture dataset and a mostly honest FM-vs-skeleton comparison whose headline gap needs error bars before the 4% HD-GCN advantage is taken at face value. read the letter →

arxiv 2506.20795 v1 pith:O2W3WOB4 submitted 2025-06-25 cs.CV cs.HCcs.RO

Stephanie Käs , Anton Burenko , Louis Markert , Onur Alp Culha , Dennis Mack , Timm Linder , Bastian Leibe This is my paper
classification cs.CVcs.HCcs.RO
keywords gesturerecognitionhuman-robotinteractionfoundationmodelsvision-languageskeleton-basedactiongraphconvolutionalnetworkszero-shotclassificationintralogisticsdataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether general-purpose vision foundation models can replace task-specific, skeleton-based pipelines for recognizing whole-body gestures in human-robot communication. The authors introduce NUGGET, a 15-class dataset of 4-second navigation-gesture recordings, and compare a skeleton-based graph convolutional network (HD-GCN), a frozen video foundation model with a small learned classification head (V-JEPA), and a zero-shot vision-language model (Gemini Flash 2.0). On the single-person test split, HD-GCN reaches 94.4% top-1 accuracy, V-JEPA reaches 90.1%, and Gemini reaches 42.1%. The central conclusion is that a foundation model with a task-specific head can come close to a dedicated skeleton pipeline, while a purely prompted VLM cannot yet resolve fine-grained gesture distinctions. This matters for robot design because a single shared video backbone that performs well across several perception tasks would simplify onboard systems.

What carries the argument

Three pipelines are compared on the paper's new dataset. NUGGET contains 11,569 four-second sequences from 13 participants performing 15 upper-body gesture classes (14 gestures plus an 'Other' class) in indoor and outdoor intralogistics settings, with up to three people per frame in the recordings. MeTRAbs + HD-GCN uses a frozen 3D pose estimator (MeTRAbs) to extract skeleton keypoint sequences, which a Hierarchically Decomposed Graph Convolutional Network classifies using spatio-temporal graph structure. V-JEPA is a frozen video foundation model whose features are pooled with a cross-attention 'attentive probe' and then passed through a small MLP classifier; only this head is trained. Gemini Flash 2.0 is used zero-shot as a multiple-choice video classifier, prompted with textual descriptions of the gestures plus the instruction to return a class ID.

What would settle it

Run all three methods on a NUGGET test split that includes multi-person frames (the recordings already contain up to three people per frame); if HD-GCN's advantage over V-JEPA shrinks or disappears there, the reported ranking is an artifact of the single-person test setup.

Watch

Extended reading notes

Core claim

The paper's central finding is that, for dynamic full-body gesture recognition on its NUGGET dataset, the specialized skeleton-based pipeline MeTRAbs + HD-GCN remains the most accurate method (94.4% top-1, 87.0% Jaccard, 93.0% F1), while V-JEPA with an attentive-probing classifier reaches 90.1% top-1 (77.0% Jaccard, 86.5% F1) and zero-shot Gemini Flash 2.0 trails at 42.1% top-1 (18.0% Jaccard, 28.0% F1). The authors interpret the V-JEPA result as evidence that a frozen video foundation model, given a simple task-specific attention-based head, can nearly match a top skeleton-based approach and could serve as a shared multi-task perception backbone. Gemini's poor performance, including near-chance recognition of directional and object-related gestures, leads them to conclude that text-prompted zero-shot VLMs are not yet suitable for fine-grained gesture recognition without further fine-tuning or a different input representation.

Load-bearing premise

The single-person test split of NUGGET is representative of the multi-person intralogistics interactions the dataset was created for, so the measured rankings generalize beyond the zero-multi-person test set.

Editorial extensions

If this is right

  • A frozen video foundation model with a lightweight learned head can stay within about four accuracy points of a dedicated skeleton pipeline, so robot perception systems may be able to share one video backbone across multiple tasks.
  • Zero-shot text-prompted vision-language models are not yet reliable for fine-grained gesture commands, so practical HRI systems using such models would need fine-tuning or gesture representations beyond text descriptions.
  • The skeleton-based method's margin holds on both NUGGET and the external TCG dataset, supporting graph convolutional networks as the current reference point for symbolic gesture recognition.
  • NUGGET's multi-person recordings, while not used in the test split, provide a resource for studying multi-user robot interactions in future work.
  • Gemini's worst failures are directional and object-related gestures (Left, Right, Pick up, Give me), identifying the classes that need better representations for zero-shot recognition.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the test split contains no multi-person sequences, the reported ranking may not transfer to the multi-user interactions the dataset was designed for; a multi-person test split could reverse V-JEPA's and HD-GCN's relative standing if tracking failures accumulate.
  • The V-JEPA pipeline's accuracy comes from freezing the backbone and training only the probe, so the same video features could plausibly be reused for object detection and other robotics tasks; the relevant comparison is then total system complexity, not classification accuracy alone.
  • The Gemini gap may be narrower than it looks: the prompts encode gestures as text descriptions, and a representation that shows motion directly (such as skeleton overlays or example frames in the prompt) could improve zero-shot performance without fine-tuning.
  • The large 'Other' class, which dominates the training set, likely inflates learned models' bias toward it; replacing the abstract non-gesture definition with concrete distractor actions might sharpen the accuracy numbers for both HD-GCN and V-JEPA.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents an empirical comparison of three approaches for dynamic, full-body gesture recognition in human-robot interaction: a skeleton-based pipeline (MeTRAbs pose estimation plus HD-GCN), a video foundation model (V-JEPA with linear or attentive probing heads), and a vision-language model (Gemini Flash 2.0 in zero-shot multiple-choice mode). The authors introduce NUGGET, a new publicly available dataset of 11,569 four-second sequences covering 15 upper-body gesture classes from 13 participants in intralogistics scenarios. On a single-person test split, MeTRAbs+HD-GCN achieves 94.4% top-1 accuracy, V-JEPA with an attentive probe reaches 90.1%, while zero-shot Gemini attains only 42.1%. The paper concludes that HD-GCN remains the most accurate approach, that V-JEPA is a viable alternative with a simple task-specific head, and that zero-shot Gemini is not yet suitable for fine-grained gesture recognition.

Significance. If the reported ranking is robust, the paper provides a useful data point for robotics practitioners deciding between specialized skeleton-based models and general-purpose foundation models. The NUGGET dataset is a concrete contribution, and the external validation of HD-GCN on the TCG dataset strengthens the baseline. The paper also includes useful ablations of V-JEPA probing strategies, Gemini prompt/input variants, and sliding-window sizes. The main weakness is statistical: the headline comparison rests on a single train/test split without confidence intervals or repeated trials, and the test set is small, class-imbalanced, and single-person only. The paper's own discussion acknowledges the class-imbalance bias but does not quantify its effect on the comparative gap.

major comments (4)
  1. [Section V-B, Table IV] The central claim that MeTRAbs+HD-GCN outperforms V-JEPA by about 4% (94.4% versus 90.1% top-1 accuracy) is based on a single train/test split of the NUGGET dataset. No confidence intervals, standard deviations, or repeated-seed results are reported. Given that the test set contains only 1,766 sequences from 13 participants and is heavily class-imbalanced ('Other' is 55.7% of test samples), the 4.3-point gap could easily be within subject-level noise. Please provide per-subject or bootstrap confidence intervals for the key metrics, or explicitly state that the ranking is not statistically validated.
  2. [Section V-C] The discussion acknowledges that both models are biased toward the 'Other' class, but it does not quantify how this bias affects the head-to-head comparison. Since 'Other' comprises 76.5% of the training set, the reported accuracies may overstate real-world performance and the comparative gap may be concentrated in specific gesture classes. Please report macro-averaged metrics, per-class confidence intervals, or class-balanced evaluations so that the HD-GCN > V-JEPA ordering can be assessed independently of the class-imbalance bias.
  3. [Section VI, Table V] The optimal 4-second sliding window for HD-GCN was selected using 'NUGGET' without specifying whether the selection was made on a held-out validation split or on the same test set used for Table IV. If the test set was used for window-size selection, the reported 94.4% accuracy is optimistic. Please clarify the model-selection protocol, and also state how the V-JEPA hyperparameters (e.g., attentive-probe architecture, number of layers) were chosen.
  4. [Section III, Table III] The reported experiments use a single-person test split, while NUGGET is explicitly motivated by multi-person intralogistics scenarios. The paper notes this limitation, but the conclusion that HD-GCN and V-JEPA are suitable for HRI gesture recognition should either be scoped to the single-person setting or supported by a multi-person evaluation, otherwise the central applicability claim is not fully substantiated.
minor comments (6)
  1. [Section V-B, Fig. 4b] The text states that V-JEPA 'reaches 100% TPR for Other', but Figure 4b shows a true-positive rate of 97.7% for the 'Other' class; please correct this discrepancy.
  2. [Section V-B] The phrase 'Left vs. Right (90.2% and 50.9% false negative rate)' is inaccurate: 90.2% is the true-positive rate for 'Left', not a false-negative rate. Please rephrase to distinguish TPR and FNR.
  3. [Section VI] The list of ablation studies contains a duplicated '(3)' numbering; also, 'heaviliy' in Section V-C should be 'heavily'.
  4. [Table IV] The Top-1 and Top-3 columns are missing the '%' sign used in the other columns; please format consistently.
  5. [Section IV-C] The exact zero-shot prompt used for Gemini is not included in the paper, despite the reference to 'Sec. VI, Prompting Strategies for Gemini'. Since reproducibility of the Gemini results depends on the prompt wording, please include the full prompt text either in the main text or in an appendix.
  6. [Section III] The paper states that NUGGET is publicly available, but no download link or repository URL is provided; please add one.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity: the central comparison is a self-contained empirical evaluation with external grounding on TCG; only a minor non-load-bearing self-citation (MeTRAbs) is present.

full rationale

This paper is an empirical benchmark rather than a derivation chain. HD-GCN is trained from scratch on the NUGGET train split, V-JEPA uses a frozen pretrained backbone with a classification head trained on the same train split, and Gemini is prompted zero-shot; the headline numbers in Table IV are measured on the held-out test split, not derived from the definitions of the models. HD-GCN is additionally validated on the external TCG dataset in Table X, where it beats the TCG baselines by at least 5%, so the skeleton baseline does not rest solely on in-house claims. The only self-citation is the MeTRAbs pose estimator [11,45,46], used as a frozen component, and no uniqueness theorem or unverified self-cited premise is invoked to force the ranking, so this is not load-bearing. Borderline issues are statistical rather than circular: the 4 s window was selected after evaluating window sizes on NUGGET in Table V and the same 94.4% accuracy is then reported in Table IV; the attentive probe was selected from Table VI on the same test set; and Gemini's textual descriptions were generated from ground-truth videos. These practices can bias or inflate the reported numbers, but none makes a result equal to its input by construction. The paper also explicitly discloses the single-person test split (Table III) and the class-imbalance bias toward 'Other' (Section V-C), which are validation concerns rather than circularity. Overall, no load-bearing step reduces to its own inputs.

Assumptions & free parameters 3 free parameters · 5 assumptions · 1 invented entities

The central comparison depends on the quality and representativeness of NUGGET (especially the single-person test split and the dominant 'Other' class), on the adequacy of MeTRAbs pose estimates for HD-GCN, and on the quality of textual descriptions for Gemini. These are domain assumptions rather than free parameters in a derivation. No fitted constants are presented as predictions.

free parameters (3)
  • Sliding window size for HD-GCN = 4 seconds
    Selected by ablation (Table V) as the best trade-off, reaching 94.4% accuracy. This is a hyperparameter tuned to NUGGET.
  • Number of frames for Gemini input = 16 frames
    In the frames input setting, a sequence of 16 frames at 10 FPS is used; this is a design choice rather than a theoretically derived value.
  • Attentive probe architecture hyperparameters = not reported
    The probe follows [17] and [12], but layer choices, pooling settings, and training details are not specified in the paper, making the configuration a hidden fitting choice.
assumptions (5)
  • domain assumption Each NUGGET sequence has one correct gesture label.
    All classification losses assume a single ground-truth class per 4-second sequence, which may not hold for ambiguous or mixed gestures.
  • domain assumption MeTRAbs provides sufficient 3D pose accuracy for HD-GCN on NUGGET.
    The skeleton-based pipeline depends entirely on the quality of pose estimates; no pose error analysis is provided for the varied NUGGET conditions.
  • domain assumption Textual class descriptions used for Gemini are adequate representations of gestures.
    Descriptions were generated by three annotators and merged via Mistral LLM; they may not capture motion dynamics, and the paper acknowledges the 'Other' definition was too abstract.
  • domain assumption The 'Other' class is well-defined and distinct from the 14 gestures.
    The paper itself notes this definition caused misinterpretation for Gemini, and the class dominates training (76.5%), affecting accuracy and bias.
  • standard math Standard supervised learning assumptions hold (i.i.d. train and test from the same distribution).
    All reported metrics assume independent samples; the data come from only 13 participants with potential correlation between clips from the same person.
invented entities (1)
  • NUGGET dataset independent evidence
    purpose: A benchmark for full-body dynamic gesture recognition in human-robot interaction, tailored to intralogistics environments.
    The dataset is claimed to be public and can be used by other researchers for external evaluation. However, no repository link or download instructions are provided in this preprint, so its availability is not yet verifiable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of How do Foundation Models Compare to Skeleton-Based Approaches for Gesture Recognition in Human-Robot Interaction?." pith.science (2026). https://pith.science/paper/O2W3WOB4

@misc{pith2026250620795,
  author       = {Pith},
  title        = {Pith review of: How do Foundation Models Compare to Skeleton-Based Approaches for Gesture Recognition in Human-Robot Interaction?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O2W3WOB4}},
  note         = {Machine review of arXiv:2506.20795}
}
read the original abstract

Gestures enable non-verbal human-robot communication, especially in noisy environments like agile production. Traditional deep learning-based gesture recognition relies on task-specific architectures using images, videos, or skeletal pose estimates as input. Meanwhile, Vision Foundation Models (VFMs) and Vision Language Models (VLMs) with their strong generalization abilities offer potential to reduce system complexity by replacing dedicated task-specific modules. This study investigates adapting such models for dynamic, full-body gesture recognition, comparing V-JEPA (a state-of-the-art VFM), Gemini Flash 2.0 (a multimodal VLM), and HD-GCN (a top-performing skeleton-based approach). We introduce NUGGET, a dataset tailored for human-robot communication in intralogistics environments, to evaluate the different gesture recognition approaches. In our experiments, HD-GCN achieves best performance, but V-JEPA comes close with a simple, task-specific classification head - thus paving a possible way towards reducing system complexity, by using it as a shared multi-task model. In contrast, Gemini struggles to differentiate gestures based solely on textual descriptions in the zero-shot setting, highlighting the need of further research on suitable input representations for gestures.

Figures

Figures reproduced from arXiv: 2506.20795 by the authors.

Figure 1
Figure 1. We compare Gemini Flash 2.0 and V-JEPA, two [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Class distributions by percentage of samples per class [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Photos of the 15 classes in our NUGGET dataset, [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Confusion matrices on NUGGET test for MeTRAbs + HD-GCN, V-JEPA (Attentive Probe), and Gemini Flash [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

67 extracted references · 63 canonical work pages

  1. [1]

    Tracking body and hands for gesture recognition: NATOPS aircraft handling signals database,

    Y . Song, D. Demirdjian, and R. Davis, “Tracking body and hands for gesture recognition: NATOPS aircraft handling signals database,” in IEEE AFGR., 2011

    work page 2011

  2. [2]

    Visual recognition of traffic police gestures with convo- lutional pose machine and handcrafted features,

    J. He et al., “Visual recognition of traffic police gestures with convo- lutional pose machine and handcrafted features,”Neurocomputing, vol. 390, 2020

    work page 2020

  3. [3]

    Chinese traffic police gesture recognition based on graph convolutional network in natural scene,

    K. Liu et al., “Chinese traffic police gesture recognition based on graph convolutional network in natural scene,”Applied Sciences, vol. 11, no. 24, 2021

    work page 2021

  4. [4]

    Traffic control gesture recognition for au- tonomous vehicles,

    J. Wiederer et al., “Traffic control gesture recognition for au- tonomous vehicles,” inIROS, 2020

    work page 2020

  5. [5]

    Learning deep and compact models for gesture recognition,

    K. Mullick and A. M. Namboodiri, “Learning deep and compact models for gesture recognition,” inICIP, 2017

    work page 2017

  6. [6]

    HGR-ViT: Hand gesture recognition with vision transformer,

    C. K. Tan et al., “HGR-ViT: Hand gesture recognition with vision transformer,”Sensors, vol. 23, 2023

    work page 2023

  7. [7]

    A transformer-based network for dynamic hand gesture recognition,

    A. D’Eusanio et al., “A transformer-based network for dynamic hand gesture recognition,” in3DV, 2020

    work page 2020

  8. [8]

    Dynamic gesture recognition based on LSTM-CNN,

    Y . Wu, B. Zheng, and Y . Zhao, “Dynamic gesture recognition based on LSTM-CNN,” inCAC, 2018

    work page 2018

Show all 67 references
  1. [9]

    Body gesture recognition to control a social mobile robot,

    J. Laplaza et al., “Body gesture recognition to control a social mobile robot,” inHRI, 2023

    2023

  2. [10]

    A fast-response dynamic-static parallel attention GCN network for body–hand gesture recognition in HRI,

    X. Guo et al., “A fast-response dynamic-static parallel attention GCN network for body–hand gesture recognition in HRI,”TIE, 2023

    2023

  3. [11]

    MeTRAbs: Metric-scale truncation-robust heatmaps for absolute 3D human pose estimation,

    I. Sarandi et al., “MeTRAbs: Metric-scale truncation-robust heatmaps for absolute 3D human pose estimation,”IEEE T-BIOM, vol. 3, no. 1, 2021

    2021

  4. [13]

    DINOv2: Learning robust visual features without supervision,

    M. Oquab et al., “DINOv2: Learning robust visual features without supervision,”TMLR, 2024

    2024

  5. [14]

    The Gemini 2.0 model family,

    DeepMind, “The Gemini 2.0 model family,”https : / / deepmind . google / technologies / gemini/, Accessed: 2025-03-25

    2025

  6. [15]

    GPT-4: OpenAI’s language model,

    OpenAI, “GPT-4: OpenAI’s language model,”https : / / openai.com/research/gpt-4, Accessed: 2025-03-25, 2025

    2025

  7. [16]

    Foundation models for video understanding: A survey,

    N. Madan et al., “Foundation models for video understanding: A survey,” 2024. arXiv:2405.03770 [cs.CV]

    2024 arXiv

  8. [17]

    VideoGLUE: Video general understanding eval- uation of foundation models,

    L. Yuan et al., “VideoGLUE: Video general understanding eval- uation of foundation models,”Transactions on Machine Learning Research, 2024

    2024

  9. [18]

    Hierarchically decomposed graph convolutional networks for skeleton-based action recognition,

    J. Lee et al., “Hierarchically decomposed graph convolutional networks for skeleton-based action recognition,” inICCV, 2023

    2023

  10. [19]

    Gemini: A family of highly capable multimodal mod- els,

    DeepMind, “Gemini: A family of highly capable multimodal mod- els,” 2023. arXiv:2312.11805 [cs.CL]

    2023 arXiv

  11. [20]

    Gesture recognition: A survey,

    S. Mitra and T. Acharya, “Gesture recognition: A survey,”SMCC, vol. 37, no. 3, 2007

    2007

  12. [21]

    Survey on hand gesture recognition from visual input,

    M. Linardakis, I. Varlamis, and G. T. Papadopoulos, “Survey on hand gesture recognition from visual input,” 2025. arXiv:2501. 11992 [cs.CV]

    2025

  13. [22]

    Computer vision-based hand gesture recognition for human-robot interaction: A review,

    J. Qi et al., “Computer vision-based hand gesture recognition for human-robot interaction: A review,”Complex Intell. Syst., vol. 10, 2024

    2024

  14. [23]

    Robust dynamic gesture recognition at ultra-long distances,

    E. B. Beeri, E. Nissinman, and A. Sintov, “Robust dynamic gesture recognition at ultra-long distances,” 2024. arXiv:2411 . 18413 [cs.RO]

    2024

  15. [24]

    UA V-GESTURE: A dataset for UA V control and gesture recognition,

    A. G. Perera, Y . Wei Law, and J. Chahl, “UA V-GESTURE: A dataset for UA V control and gesture recognition,” inECCV, 2018

    2018

  16. [25]

    An overview of hand gesture languages for autonomous UA V handling,

    F. Patrona, I. Mademlis, and I. Pitas, “An overview of hand gesture languages for autonomous UA V handling,” inAIRPHARO, 2021

    2021

  17. [26]

    Synthetic-to-real domain adaptation for action recognition: A dataset and baseline performances,

    A. V . Reddy et al., “Synthetic-to-real domain adaptation for action recognition: A dataset and baseline performances,” inICRA, 2023

    2023

  18. [27]

    Real-time multi-modal hu- man–robot collaboration using gestures and speech,

    H. Chen, M. C. Leu, and Z. Yin, “Real-time multi-modal hu- man–robot collaboration using gestures and speech,”J. Manuf. Sci. Eng., vol. 144, no. 10, 2022

    2022

  19. [28]

    Vision-based hand gesture recognition for human- robot collaboration: A survey,

    Z. Xia et al., “Vision-based hand gesture recognition for human- robot collaboration: A survey,” inICCAR, 2019

    2019

  20. [29]

    A human-centered approach to robot gesture based communication within collaborative working processes,

    T. Ende et al., “A human-centered approach to robot gesture based communication within collaborative working processes,” inIROS, 2011

    2011

  21. [30]

    FollowMe: Person following and gesture recognition with a quadrocopter,

    T. Naseer, J. Sturm, and D. Cremers, “FollowMe: Person following and gesture recognition with a quadrocopter,” inIROS, 2013

    2013

  22. [31]

    Innovative collabora- tive method for interaction between a human operator and robotic manipulator using pointing gestures,

    M. ˇCorˇn´ak, M. T ¨olgyessy, and P. Hubinsk ´y, “Innovative collabora- tive method for interaction between a human operator and robotic manipulator using pointing gestures,”Applied Sciences, 2022

    2022

  23. [32]

    Gesture recognition for human-robot collab- oration: A review,

    H. Liu and L. Wang, “Gesture recognition for human-robot collab- oration: A review,”Int. J. Ind. Ergonom., vol. 68, 2018

    2018

  24. [33]

    Application of human activity/action recognition: A review,

    N. Sedaghati, S. Ardebili, and A. Ghaffari, “Application of human activity/action recognition: A review,”Multim. Tools Appl., 2025

    2025

  25. [34]

    A survey on vision-based human action recognition,

    R. Poppe, “A survey on vision-based human action recognition,” Image and Vision Computing, vol. 28, no. 6, 2010

    2010

  26. [35]

    Review of dynamic gesture recognition,

    Y . Shi et al., “Review of dynamic gesture recognition,”Virtual Reality & Intelligent Hardware, vol. 3, no. 3, 2021

    2021

  27. [36]

    Recent advances of monocular 2D and 3D human pose estimation: A deep learning perspective,

    W. Liu et al., “Recent advances of monocular 2D and 3D human pose estimation: A deep learning perspective,”ACM Comput. Surv., vol. 55, no. 4, 2022

    2022

  28. [37]

    TSGCNeXt: Dynamic-static multi-graph convolu- tion for efficient skeleton-based action recognition with long-term learning potential,

    D. Liu et al., “TSGCNeXt: Dynamic-static multi-graph convolu- tion for efficient skeleton-based action recognition with long-term learning potential,” 2023. arXiv:2304.11631 [cs.CV]

    2023 arXiv

  29. [38]

    Language knowledge-assisted in topology construction for skeleton-based action recognition,

    T. Nguyen Quang and T.-O. Nguyen, “Language knowledge-assisted in topology construction for skeleton-based action recognition,” in SOICT, 2023

    2023

  30. [39]

    ViPLO: Vision transformer based pose-conditioned self-loop graph for human-object interaction detection,

    J. Park, J.-W. Park, and J.-S. Lee, “ViPLO: Vision transformer based pose-conditioned self-loop graph for human-object interaction detection,” inCVPR, 2023

    2023

  31. [40]

    SkeleTR: Towards skeleton-based action recogni- tion in the wild,

    H. Duan et al., “SkeleTR: Towards skeleton-based action recogni- tion in the wild,” inICCV, 2023

    2023

  32. [41]

    SkeletonMAE: Graph-based masked autoencoder for skeleton sequence pre-training,

    H. Yan et al., “SkeletonMAE: Graph-based masked autoencoder for skeleton sequence pre-training,” inICCV, 2023

    2023

  33. [42]

    MotionGPT: Human motion as a foreign language,

    B. Jiang et al., “MotionGPT: Human motion as a foreign language,” NeurIPS, vol. 36, 2024

    2024

  34. [43]

    MotionGPT-2: A general-purpose motion-language model for motion generation and understanding,

    Y . Wang et al., “MotionGPT-2: A general-purpose motion-language model for motion generation and understanding,” 2024. arXiv: 2410.21747 [cs.CV]

    2024 arXiv

  35. [44]

    Spatial temporal graph convolutional networks for skeleton-based action recognition,

    S. Yan, Y . Xiong, and D. Lin, “Spatial temporal graph convolutional networks for skeleton-based action recognition,” inAAAI, 2018

    2018

  36. [45]

    Learning 3D human pose estimation from dozens of datasets using a geometry-aware autoencoder to bridge between skeleton formats,

    I. S ´ar´andi, A. Hermans, and B. Leibe, “Learning 3D human pose estimation from dozens of datasets using a geometry-aware autoencoder to bridge between skeleton formats,” inWACV, 2023

    2023

  37. [46]

    Systematic comparison of projection methods for monocular 3D human pose estimation on fisheye images,

    S. K ¨as et al., “Systematic comparison of projection methods for monocular 3D human pose estimation on fisheye images,” inICRA, 2025

    2025

  38. [47]

    A proposed set of communicative gestures for human robot interaction and an RGB image-based gesture recognizer implemented in ROS,

    J. Tan et al., “A proposed set of communicative gestures for human robot interaction and an RGB image-based gesture recognizer implemented in ROS,” inICRA, 2022

    2022

  39. [48]

    Skeleton-based action and gesture recognition for human-robot collaboration,

    M. Terreran, M. Lazzaretto, and S. Ghidoni, “Skeleton-based action and gesture recognition for human-robot collaboration,” inIAS, 2023

    2023

  40. [49]

    Recognizing actions by shape- motion prototype trees,

    Z. Lin, Z. Jiang, and L. S. Davis, “Recognizing actions by shape- motion prototype trees,” inICCV, 2009

    2009

  41. [50]

    Fast gesture recognition with multiple stream discrete HMMs on 3D skeletons,

    G. Borghi, R. Vezzani, and R. Cucchiara, “Fast gesture recognition with multiple stream discrete HMMs on 3D skeletons,” inICPR, 2016

    2016

  42. [51]

    A full-body gesture database for automatic gesture recognition,

    B.-W. Hwang, S. Kim, and S.-W. Lee, “A full-body gesture database for automatic gesture recognition,” inIEEE AFGR., 2006

    2006

  43. [52]

    Towards controlling mobile robot using upper human body gesture based on convolutional neural network,

    M. Fuad et al., “Towards controlling mobile robot using upper human body gesture based on convolutional neural network,”JRC, vol. 4, no. 6, 2023

    2023

  44. [53]

    Self-feedback DETR for temporal action detection,

    J. Kim, M. Lee, and J.-P. Heo, “Self-feedback DETR for temporal action detection,” inICCV, 2023

    2023

  45. [54]

    Cross-modal learning with 3D deformable attention for action recognition,

    S. Kim, D. Ahn, and B. C. Ko, “Cross-modal learning with 3D deformable attention for action recognition,” inICCV, 2023

    2023

  46. [55]

    Zero-shot action recogni- tion in videos: A survey,

    V . Estevam, H. Pedrini, and D. Menotti, “Zero-shot action recogni- tion in videos: A survey,”Neurocomputing, vol. 439, 2021

    2021

  47. [56]

    Foundation model and temporal priors-guided trans- ductive few-shot action recognition,

    B. Vu et al., “Foundation model and temporal priors-guided trans- ductive few-shot action recognition,” inICASSP, 2025

    2025

  48. [57]

    Few-shot action recognition with captioning foundation models,

    X. Wang et al., “Few-shot action recognition with captioning foundation models,” 2023. arXiv:2310.10125 [cs.CV]

    2023 arXiv

  49. [58]

    An evaluation of large pre-trained models for gesture recognition using synthetic videos,

    A. Reddy et al., “An evaluation of large pre-trained models for gesture recognition using synthetic videos,” inSynth. Data for Artif. Intell. and Mach. Learning: Tools, Techniques, and Applications II, 2024

    2024

  50. [59]

    motpy – simple multi object tracking library,

    W. Muro ´n, “motpy – simple multi object tracking library,”https: //github.com/wmuron/motpy, 2021

    2021

  51. [60]

    NTU RGB+D 120: A large-scale benchmark for 3D human activity understanding,

    J. Liu et al., “NTU RGB+D 120: A large-scale benchmark for 3D human activity understanding,”PAMI, vol. 42, no. 10, 2020

    2020

  52. [61]

    DeGCN: Deformable graph convolutional net- works for skeleton-based action recognition,

    W. Myung et al., “DeGCN: Deformable graph convolutional net- works for skeleton-based action recognition,”Trans. Image Proc., vol. 33, 2024

    2024

  53. [62]

    An image is worth 16x16 words: Transform- ers for image recognition at scale,

    A. Dosovitskiy et al., “An image is worth 16x16 words: Transform- ers for image recognition at scale,” inICLR, 2021

    2021

  54. [63]

    A comprehensive review of YOLO architectures in computer vision: From YOLOv1 to YOLOv8 and YOLO-NAS,

    J. Terven, D.-M. C ´ordova-Esparza, and J.-A. Romero-Gonz ´alez, “A comprehensive review of YOLO architectures in computer vision: From YOLOv1 to YOLOv8 and YOLO-NAS,”Machine Learning and Knowledge Extraction, vol. 5, no. 4, 2023

    2023

  55. [64]

    Mask R-CNN,

    K. He et al., “Mask R-CNN,” inICCV, 2017

    2017

  56. [65]

    Revisiting feature prediction for learning visual representations from video,

    A. Bardes et al., “Revisiting feature prediction for learning visual representations from video,”Transactions on Machine Learning Research, 2024

    2024

  57. [66]

    Video-MME: The first-ever comprehensive evaluation benchmark of multi-modal LLMs in video analysis,

    C. Fu et al., “Video-MME: The first-ever comprehensive evaluation benchmark of multi-modal LLMs in video analysis,” inCVPR, 2025

    2025

  58. [67]

    Mistral 7B,

    A. Q. Jiang et al., “Mistral 7B,” 2023. arXiv:2310 . 06825 [cs.CL]

    2023

  59. [68]

    3D human pose estimation in video with temporal convolutions and semi-supervised training,

    D. Pavllo et al., “3D human pose estimation in video with temporal convolutions and semi-supervised training,” inCVPR, 2019

    2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.