Pith. sign in

REVIEW 4 major objections 5 minor 76 references

Skeleton Motion Words for Unsupervised Skeleton-Based Temporal Action Segmentation

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SMQ shows that unsupervised skeleton-based temporal action segmentation can be done by learning a small codebook of discrete skeleton motion words, and that joint-disentangled, patch-level quantization outperforms prior unsupervised methods

desk verdict First unsupervised skeleton-based TAS method; solid empirical study, but the motion-word story is fragile and the evaluation has several fixable leaks. read the letter →

arxiv 2508.04513 v1 pith:EGSXP345 submitted 2025-08-06 cs.CV

Uzay Gökay , Federico Spurio , Dominik R. Bach , Juergen Gall This is my paper · ORCID
classification cs.CV
keywords unsupervisedtemporalactionsegmentationskeleton-basedvectorquantizationmotionwordsautoencoderjoint-disentangledembeddingcodebooklearninghumananalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Unsupervised temporal action segmentation has been studied mostly on RGB video; skeleton recordings—robust, privacy-preserving, and common in wearables and motion capture—have lacked a working unsupervised recipe. This paper tries to close that gap by recasting the task as learning a small vocabulary of discrete skeleton motion words: a temporal autoencoder maps each joint separately into a latent space, cuts the latent sequence into one-second patches, and quantizes every patch to its nearest codebook vector. The codebook index becomes the frame's action label, so segmentation and clustering happen in a single pass with no labels. Across HuGaDB, LARa, and BABEL subsets, the method reports consistent gains over prior unsupervised video-action-segmentation methods and over self-supervised skeleton representation baselines, with the largest margins on segment-level metrics.

What carries the argument

The central object is the skeleton motion word: a codebook vector $c_k \in \mathbb{R}^{P \times (V \cdot D)}$ representing a prototypical short skeleton motion over a $P$-frame patch. It carries the argument through four moves: (1) the encoder processes each joint independently so the embedding space stays joint-disentangled; (2) Euclidean nearest-neighbor assignment converts each latent patch to a word and yields the segmentation $Y_{nt}$ directly from the assigned word index; (3) EMA codebook updates and a stop-gradient commitment loss stabilize the learned vocabulary; and (4) a translation/rotation-invariant inter-joint distance reconstruction loss shapes the words to preserve pose-level

What would settle it

Concrete test: take a trained SMQ model on LARa and compute, for each ground-truth action, the entropy of the codebook-word distribution over its patches. If a single repeated action instance is split across many words (high intra-action entropy), or if the same word appears across actions with dissimilar poses, the action-consistency claim fails. Alternatively, evaluate SMQ on a dataset with two actions that share poses but differ in temporal order: if quantization groups by pose rather than action, the two actions will collapse into one segment.

Watch

Extended reading notes

Core claim

SMQ treats unsupervised skeleton-based temporal action segmentation as online time-series clustering: a dilated-TCN sequence-to-sequence autoencoder maps each joint separately into a latent space, the per-joint embeddings are patched into non-overlapping one-second segments, and each patch is assigned to the nearest vector in a learned codebook whose entries are the "skeleton motion words." The assigned codebook index directly yields the frame-level action label, and training with an inter-joint-distance reconstruction loss plus a commitment loss makes the words capture actions rather than pose noise. On HuGaDB, LARa, and the three BABEL subsets, SMQ reports higher MoF, edit score, and segme

Load-bearing premise

The method stands or falls on whether assigning each motion patch to the closest learned codebook word, guided only by reconstruction and commitment losses, yields labels that match actions; if the codebook organizes by pose details instead of action identity, the segmentation fragments and the segmental metrics collapse.

Editorial extensions

If this is right

  • Because labels are the codebook indices themselves, SMQ needs no separate clustering step or Viterbi-style post-processing, avoiding the over-fragmentation of CTE/TOT and the over-merging of ASOT.
  • Patch-level quantization matters: frame-wise assignment over-segments, while one-second patches give the best segmental metrics on LARa; patch size is reported as a key temporal-resolution knob.
  • Keeping joints disentangled in the embedding improves performance on all three datasets, so skeleton structure itself carries signal that joint-concatenated representations lose.
  • Reconstruction through inter-joint distances outperforms per-coordinate MSE, indicating that a pose-level, translation- and rotation-invariant objective is part of what makes the learned motion words useful.
  • SMQ degrades gracefully when joints are randomly dropped (25% and 50% missing), indicating the learned motion words are not tied to any single joint.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One step the paper leaves implicit: the same codebook could feed a hierarchical scheme—coarse words for actions, finer words for sub-motion—which would directly address the boundary detection the paper names as future work.
  • The supplementary silhouette-score analysis suggests $K$ (the number of motion words) can be chosen from unlabeled data; testing that heuristic on HuGaDB and BABEL, rather than only LARa, is a concrete next check.
  • If motion words are shared across actions, they may function as reusable movement primitives; inspecting per-word mean poses and joint-velocity profiles could test whether the vocabulary composes actions from smaller parts.
  • The graceful degradation under missing joints points beyond benchmarks: in deployments with partial occlusion or dropped IMU nodes, SMQ might keep segmenting with only a small score drop—a property worth verifying on other datasets.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces SMQ, an unsupervised method for skeleton-based temporal action segmentation. A sequence-to-sequence temporal autoencoder processes each joint independently to obtain a disentangled latent embedding; the latent sequence is split into non-overlapping patches, each patch is quantized to the nearest of K learnable codebook vectors ('skeleton motion words'), and the decoder reconstructs the input from the quantized patches. Frame-level segmentation is read out directly from the nearest-codebook assignments. The method is evaluated on HuGaDB, LARa, BABEL (three subsets), and PKU-MMD v2, and is reported to outperform prior unsupervised temporal action segmentation methods and self-supervised skeleton representation baselines on most metrics.

Significance. If the empirical claims hold, SMQ would be a meaningful advance for unsupervised skeleton-based temporal action segmentation, a setting where prior unsupervised video-segmentation methods transfer poorly. The paper provides a clear architectural contribution—joint-disentangled embeddings plus patch-level vector quantization—and backs it with a broad set of ablations, qualitative results, runtime measurements, and public code. The main numerical gains on HuGaDB and LARa are large, and the extension to the BABEL protocol is a useful stress test. However, the central interpretive claim that the learned codebook entries are action-consistent 'motion words' is not established; in fact, the paper's own K-robustness experiment provides evidence against a one-code-per-action mapping. The evaluation also has methodological weaknesses—hyperparameter selection on the reported test set, missing error bars, and a modified BABEL protocol—that must be clarified before the comparison can be fully trusted.

major comments (4)
  1. [Table 16 and Suppl. 6.1–6.2] The codebook size K is set to the number of ground-truth action classes (Suppl. 6.1), and evaluation aligns clusters to labels via a global one-to-one Hungarian matching (Suppl. 6.2). The K-robustness study (Table 16) shows that on LARa, K=4 achieves MoF 44.0 while the true K=8 achieves only 37.4. If each learned motion word corresponded to an action, providing the true number of actions should not be clearly worse than providing half that number. This is internal evidence that the learned codebook entries fragment action classes and that the one-to-one Hungarian mapping then distorts the evaluation. The paper's central claim that skeleton motion words are semantically meaningful action clusters is therefore not supported by the current evidence. Please report a codebook-to-ground-truth contingency analysis (e.g., per-code action purity or a many-to-one mapping), and discuss why the true
  2. [Sec. 4.3, Tables 5, 7, 8] The patch size, reconstruction loss weight λ, and EMA decay α are selected on LARa by comparing test-set metrics, and the final LARa results are then reported as the method's performance. This is selection on the test set and can inflate the reported numbers, especially in an unsupervised setting where no validation split is described. The same selected hyperparameters are then fixed across the other datasets, so the issue propagates. Please either use a validation split for hyperparameter selection, report all configurations' performance across datasets, or at minimum disclose this dependence and supplement the main tables with multiple-seed means and standard deviations. Given the absence of error bars, it is currently impossible to tell whether the reported gains are within run-to-run variation.
  3. [Sec. 4.1 (BABEL protocol)] The BABEL evaluation modifies the protocol of [69] by excluding sequences containing more than 50% background actions. This is a significant alteration: it changes the difficulty and composition of the benchmark, and it affects the comparability of the reported numbers with any previously published numbers. The paper needs to state how many sequences are removed from each subset, report results on the original unmodified subset as well, and confirm explicitly that every baseline method was run on exactly the same filtered sequences. Without this, the BABEL comparison in Table 2 is not fully controlled.
  4. [Tables 1–3 and 16] No variance or significance information is reported for the main results. Unsupervised clustering methods, including SMQ, depend on initialization and stochastic training; the differences between SMQ and the strongest baseline on some BABEL metrics are small (e.g., ASOT vs SMQ on Subset-1 Edit and F1@10 in Table 2). Reporting single runs is insufficient to support the claim of consistent superiority. Please report at least three seeds with means and standard deviations for all principal comparisons, and indicate whether the differences are stable across seeds.
minor comments (5)
  1. [Eq. (6)] The summation notation in the commitment loss is garbled ('Σ_{i∈ p}'); it should read as a sum over all patches in the batch.
  2. [Suppl. 6.1] The sentence 'The codebook size corresponds to the number of ground-truth actions in the dataset as it is required by the protocol' is a key design choice and should be stated prominently in the main text, not only in the supplement. It also deserves a justification beyond 'required by the protocol.'
  3. [Table 16 caption] The column header 'Num of actions (K)' conflates the codebook size with the number of ground-truth actions. Since the two are set equal by the protocol, this is understandable, but a clearer caption would avoid confusion about what is being varied.
  4. [Sec. 4.2, text after Table 2] The text says 'ASOT performs slightly better for the other metrics' for BABEL Subset-1, but the table shows ASOT also outperforms SMQ on Edit and all F1 thresholds there. The wording understates the comparison; please be precise.
  5. [Figure 3] The qualitative figure would benefit from clearer axis labels and legends indicating which color corresponds to which action; the current figure is hard to read without the caption.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the motion-word segmentation is produced by reconstruction-trained codebook assignment, and the use of GT class count K plus Hungarian evaluation is a standard protocol rather than a fitted input.

full rationale

The claimed derivation chain is: a joint-disentangled encoder maps skeleton sequences to latent embeddings; embeddings are patched; each patch is assigned to the nearest learned codebook vector (Eq. 1); codebook vectors are updated by EMA of assigned patches (Eqs. 2-4); the decoder reconstructs the input from quantized patches under inter-joint-distance MSE (Eq. 5) plus a commitment loss (Eq. 6); total loss is Eq. 7. None of these equations involves ground-truth action labels. Segmentation Y is read directly from the codebook indices, and its agreement with human actions is an empirical property evaluated afterward. The only GT-derived scalar is the codebook size K (Suppl. 6.1: "The codebook size corresponds to the number of ground-truth actions in the dataset as it is required by the protocol"), and the only alignment step is global one-to-one Hungarian matching (Suppl. 6.2). Both are standard evaluation protocols used identically for the compared methods; they do not make the predicted labels equal to the ground truth by construction. The K-robustness study (Table 16) is an ablation, not a fitted prediction, and the silhouette-score suggestion (Suppl. 8.8) is an unsupervised heuristic validated by correlation. Self-citations (MS-TCN architecture, HVQ in related work) are not load-bearing: the architecture choice is ablated in Table 12, and no uniqueness or correctness claim is imported from the authors' prior work. Thus the central segmentation result is self-contained and not circular.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The approach relies on the empirical assumption that a vector-quantized autoencoder trained with reconstruction and commitment losses produces a codebook whose indices align with action semantics; this is the key domain assumption, and it is only validated by downstream evaluation metrics, not by any formal argument. K, the number of actions, is taken from ground truth in the main experiments, and several hyperparameters (patch size, alpha, lambda) are chosen on LARa.

free parameters (5)
  • Patch size P = 50 frames (1 s) on LARa; 60 on HuGaDB, 30 on BABEL at their frame rates
    Chosen by ablation on LARa (Table 5); then kept at one second across all datasets (Section 4.3).
  • Codebook size K = GT action count: 10 (HuGaDB), 8 (LARa), 4 (BABEL), 41 (PKU-MMD)
    Set to the number of ground-truth action classes (suppl. 6.1), i.e., label-count supervision; a silhouette-score heuristic is proposed post hoc.
  • EMA decay alpha = 0.5
    Tuned on LARa via ablation (Table 8).
  • Reconstruction loss weight lambda = 0.001
    Tuned on LARa via ablation (Table 7).
  • Latent dimension D and per-joint channel sizes = not reported
    Architectural choice not specified in the paper, needed for exact reproduction.
assumptions (5)
  • domain assumption Euclidean nearest-neighbor quantization (Eq. 1) produces action-consistent labels.
    The segmentation is read directly from codebook assignments; the training objective does not contain an action-consistency term, so this is an empirical assumption.
  • domain assumption A finite, shared codebook of motion words can represent all actions across subjects.
    Section 3.2: 'skeleton sequences are mapped to a sequence of skeleton motion words...' assumes a discrete vocabulary of atomic motions.
  • domain assumption Per-joint disentangled encoding preserves cross-joint information needed for actions.
    The encoder processes each joint independently (Section 3.1); the paper validates this by ablation (Table 4) but cannot guarantee it for new datasets.
  • domain assumption Inter-joint distance MSE is a sufficient reconstruction objective for learning pose semantics.
    Eq. 5 discards absolute positions and rotations, assuming these are irrelevant for action identity.
  • standard math Hungarian matching between predicted and GT clusters is a fair evaluation protocol.
    Used following prior work (Section 6.2); it is a standard permutation-alignment step in unsupervised clustering evaluation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Skeleton Motion Words for Unsupervised Skeleton-Based Temporal Action Segmentation." pith.science (2026). https://pith.science/paper/EGSXP345

@misc{pith2026250804513,
  author       = {Pith},
  title        = {Pith review of: Skeleton Motion Words for Unsupervised Skeleton-Based Temporal Action Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EGSXP345}},
  note         = {Machine review of arXiv:2508.04513}
}
read the original abstract

Current state-of-the-art methods for skeleton-based temporal action segmentation are predominantly supervised and require annotated data, which is expensive to collect. In contrast, existing unsupervised temporal action segmentation methods have focused primarily on video data, while skeleton sequences remain underexplored, despite their relevance to real-world applications, robustness, and privacy-preserving nature. In this paper, we propose a novel approach for unsupervised skeleton-based temporal action segmentation. Our method utilizes a sequence-to-sequence temporal autoencoder that keeps the information of the different joints disentangled in the embedding space. Latent skeleton sequences are then divided into non-overlapping patches and quantized to obtain distinctive skeleton motion words, driving the discovery of semantically meaningful action clusters. We thoroughly evaluate the proposed approach on three widely used skeleton-based datasets, namely HuGaDB, LARa, and BABEL. The results demonstrate that our model outperforms the current state-of-the-art unsupervised temporal action segmentation methods. Code is available at https://github.com/bachlab/SMQ .

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

76 extracted references · 73 canonical work pages

  1. [69]

    Frame-level label refinement for skeleton-based weakly-supervised action recognition

    Qing Yu and Kent Fujiwara. Frame-level label refinement for skeleton-based weakly-supervised action recognition. In Proceedings of the AAAI Conference on Artificial Intelli- gence (AAAI), pages 3322–3330, 2023. 5

    work page 2023

  2. [1]

    Motion segmentation via generalized curvatures

    Robert T Arn, Pradyumna Narayana, Tegan Emerson, Bruce A Draper, Michael Kirby, and Chris Peterson. Motion segmentation via generalized curvatures. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) , 41 (12):2919–2932, 2018. 2

    work page 2018

  3. [2]

    Segmenting motion capture data into distinct behaviors

    Jernej Barbi ˇc, Alla Safonova, Jia-Yu Pan, Christos Falout- sos, Jessica K Hodgins, and Nancy S Pollard. Segmenting motion capture data into distinct behaviors. In Proceedings of Graphics Interface (GI), pages 185–194, 2004. 2

    work page 2004

  4. [3]

    Unified fully and timestamp supervised temporal action segmentation via sequence to se- quence translation

    Nadine Behrmann, S Alireza Golestaneh, Zico Kolter, Juer- gen Gall, and Mehdi Noroozi. Unified fully and timestamp supervised temporal action segmentation via sequence to se- quence translation. In Proceedings of the European Confer- ence on Computer Vision (ECCV) , pages 52–68. Springer,

    work page

  5. [4]

    Leveraging triplet loss for unsupervised action segmentation

    Elena Bel ´en Bueno-Benito, Biel Tura Vecino, and Mariella Dimiccoli. Leveraging triplet loss for unsupervised action segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4921–4929, 2023. 2, 4, 5

    work page 2023

  6. [5]

    Deep clustering for unsupervised learning of visual features

    Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze. Deep clustering for unsupervised learning of visual features. In Proceedings of the European confer- ence on computer vision (ECCV), pages 132–149, 2018. 2

    work page 2018

  7. [6]

    Unsupervised learning of visual features by contrasting cluster assignments

    Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Pi- otr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. Ad- vances in neural information processing systems , 33:9912– 9924, 2020

    work page 2020

  8. [7]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 9650–9660, 2021

    work page 2021

Show all 76 references
  1. [8]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on ma- chine learning, pages 1597–1607. PmLR, 2020. 2

    2020

  2. [9]

    Hugadb: Hu- man gait database for activity recognition from wearable in- ertial sensor networks

    Roman Chereshnev and Attila Kert ´esz-Farkas. Hugadb: Hu- man gait database for activity recognition from wearable in- ertial sensor networks. In Analysis of Images, Social Net- works and Texts (AIST), pages 131–141. Springer, 2018. 2, 5

    2018

  3. [10]

    Tempo- ral action segmentation: An analysis of modern techniques

    Guodong Ding, Fadime Sener, and Angela Yao. Tempo- ral action segmentation: An analysis of modern techniques. IEEE Transactions on Pattern Analysis and Machine Intelli- gence (TPAMI), 2023. 2, 6, 1

    2023

  4. [11]

    Fast and unsupervised action boundary detection for action seg- mentation

    Zexing Du, Xue Wang, Guoqing Zhou, and Qing Wang. Fast and unsupervised action boundary detection for action seg- mentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3323–3332, 2022. 2

    2022

  5. [12]

    Ms-tcn: Multi-stage temporal convolutional network for action segmentation

    Yazan Abu Farha and Juergen Gall. Ms-tcn: Multi-stage temporal convolutional network for action segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR) , pages 3575–3584,

  6. [13]

    Masked autoencoders as spatiotemporal learners

    Christoph Feichtenhofer, Yanghao Li, Kaiming He, et al. Masked autoencoders as spatiotemporal learners. Advances in neural information processing systems, 35:35946–35958,

  7. [14]

    Skeleton-based action segmentation with multi-stage spatial- temporal graph convolutional neural networks

    Benjamin Filtjens, Bart Vanrumste, and Peter Slaets. Skeleton-based action segmentation with multi-stage spatial- temporal graph convolutional neural networks. IEEE Trans- actions on Emerging Topics in Computing , 12(1):202–212,

  8. [15]

    Bootstrap your own latent-a new approach to self-supervised learning

    Jean-Bastien Grill, Florian Strub, Florent Altch ´e, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Ghesh- laghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. Advances in Neur...

    2020

  9. [16]

    Contrastive learning from ex- tremely augmented skeleton sequences for self-supervised action recognition

    Tianyu Guo, Hong Liu, Zhan Chen, Mengyuan Liu, Tao Wang, and Runwei Ding. Contrastive learning from ex- tremely augmented skeleton sequences for self-supervised action recognition. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), pages 762–770, 2022. 1, 3, 6, 7

    2022

  10. [17]

    Momentum contrast for unsupervised visual rep- resentation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual rep- resentation learning. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 9729–9738, 2020. 2

    2020

  11. [18]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16000–16009, 2022. 2

    2022

  12. [19]

    Deep-aligned convolutional neural network for skeleton- based action recognition and segmentation

    Babak Hosseini, Romain Montagne, and Barbara Hammer. Deep-aligned convolutional neural network for skeleton- based action recognition and segmentation. Data Science and Engineering (DSE), 5(2):126–139, 2020. 1

    2020

  13. [20]

    Improving action segmentation via graph-based temporal reasoning

    Yifei Huang, Yusuke Sugano, and Yoichi Sato. Improving action segmentation via graph-based temporal reasoning. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 14024–14034,

  14. [21]

    Language-assisted skeleton action understanding for skeleton-based temporal action seg- mentation

    Haoyu Ji, Bowen Chen, Xinglong Xu, Weihong Ren, Zhiy- ong Wang, and Honghai Liu. Language-assisted skeleton action understanding for skeleton-based temporal action seg- mentation. In Proceedings of the European Conference on Computer Vision (ECCV), 2024. 1, 2 9

    2024

  15. [22]

    Kipf and Max Welling

    Thomas N. Kipf and Max Welling. Semi-supervised classi- fication with graph convolutional networks. In International Conference on Learning Representations, 2017. 2

    2017

  16. [23]

    Efficient unsupervised temporal segmentation of motion data

    Bj ¨orn Kr ¨uger, Anna V ¨ogele, Tobias Willig, Angela Yao, Reinhard Klein, and Andreas Weber. Efficient unsupervised temporal segmentation of motion data. IEEE Transactions on Multimedia (T-MM), 19(4):797–812, 2016. 2

    2016

  17. [24]

    An end-to- end generative framework for video segmentation and recog- nition

    Hilde Kuehne, Juergen Gall, and Thomas Serre. An end-to- end generative framework for video segmentation and recog- nition. In Proceedings of the IEEE/CVF Winter Confer- ence on Applications of Computer Vision (WACV), pages 1–

  18. [25]

    A hy- brid rnn-hmm approach for weakly supervised temporal ac- tion segmentation

    Hilde Kuehne, Alexander Richard, and Juergen Gall. A hy- brid rnn-hmm approach for weakly supervised temporal ac- tion segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 42(4):765–779, 2020. 6

    2020

  19. [26]

    Unsupervised learning of action classes with continu- ous temporal embedding

    Anna Kukleva, Hilde Kuehne, Fadime Sener, and Juergen Gall. Unsupervised learning of action classes with continu- ous temporal embedding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12066–12074, 2019. 1, 2, 6, 3

    2019

  20. [27]

    Unsupervised action segmentation by joint representation learning and on- line clustering

    Sateesh Kumar, Sanjay Haresh, Awais Ahmed, Andrey Konin, M Zeeshan Zia, and Quoc-Huy Tran. Unsupervised action segmentation by joint representation learning and on- line clustering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pa...

    2022

  21. [28]

    Segmental spatiotemporal cnns for fine-grained action seg- mentation

    Colin Lea, Austin Reiter, Ren ´e Vidal, and Gregory D Hager. Segmental spatiotemporal cnns for fine-grained action seg- mentation. In Proceedings of the European Conference on Computer Vision (ECCV), pages 36–52. Springer, 2016. 1

    2016

  22. [29]

    Temporal convolutional networks for action segmentation and detection

    Colin Lea, Michael D Flynn, Rene Vidal, Austin Reiter, and Gregory D Hager. Temporal convolutional networks for action segmentation and detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 156–165, 2017. 2, 6, 1

    2017

  23. [30]

    Action shuffle alternating learn- ing for unsupervised action segmentation

    Jun Li and Sinisa Todorovic. Action shuffle alternating learn- ing for unsupervised action segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12628–12636, 2021. 1, 2

    2021

  24. [31]

    3d human action repre- sentation learning via cross-view consistency pursuit

    Linguo Li, Minsi Wang, Bingbing Ni, Hang Wang, Jiancheng Yang, and Wenjun Zhang. 3d human action repre- sentation learning via cross-view consistency pursuit. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 4741–4750, 2021. 1, 3

    2021

  25. [32]

    Ms-tcn++: Multi-stage temporal convolu- tional network for action segmentation

    Shijie Li, Yazan Abu Farha, Yun Liu, Ming-Ming Cheng, and Juergen Gall. Ms-tcn++: Multi-stage temporal convolu- tional network for action segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) , 45 (6):6647–6658, 2023. 2, 3

    2023

  26. [33]

    A decoupled spatio- temporal framework for skeleton-based action segmentation

    Yunheng Li, Zhongyu Li, Shanghua Gao, Qilong Wang, Qibin Hou, and Ming-Ming Cheng. A decoupled spatio- temporal framework for skeleton-based action segmentation. arXiv preprint arXiv:2312.05830, 2023. 1, 2

    2023 arXiv

  27. [34]

    Involving distinguished temporal graph convolu- tional networks for skeleton-based temporal action segmen- tation

    Yun-Heng Li, Kai-Yuan Liu, Sheng-Lan Liu, Lin Feng, and Hong Qiao. Involving distinguished temporal graph convolu- tional networks for skeleton-based temporal action segmen- tation. IEEE Transactions on Circuits and Systems for Video Technology, 34(1):647–660, 2023. 2

    2023

  28. [35]

    Ms2l: Multi-task self-supervised learning for skeleton based action recognition

    Lilang Lin, Sijie Song, Wenhan Yang, and Jiaying Liu. Ms2l: Multi-task self-supervised learning for skeleton based action recognition. In Proceedings of the ACM International Con- ference on Multimedia (ACM MM), pages 2490–2498, 2020. 1, 3

    2020

  29. [36]

    Actionlet- dependent contrastive learning for unsupervised skeleton- based action recognition

    Lilang Lin, Jiahang Zhang, and Jiaying Liu. Actionlet- dependent contrastive learning for unsupervised skeleton- based action recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2363–2372, 2023. 1, 3, 6, 7

    2023

  30. [37]

    Taec: Unsupervised action segmentation with temporal-aware embedding and clustering

    Wei Lin, Anna Kukleva, Horst Possegger, Hilde Kuehne, and Horst Bischof. Taec: Unsupervised action segmentation with temporal-aware embedding and clustering. In Computer Vi- sion Winter Workshop, 2023. 1, 2

    2023

  31. [38]

    Pku-mmd: A large scale benchmark for continu- ous multi-modal human action understanding.arXiv preprint arXiv:1703.07475, 2017

    Chunhui Liu, Yueyu Hu, Yanghao Li, Sijie Song, and Jiay- ing Liu. Pku-mmd: A large scale benchmark for continu- ous multi-modal human action understanding.arXiv preprint arXiv:1703.07475, 2017. 5, 1

    2017 arXiv

  32. [39]

    Diffusion action segmenta- tion

    Daochang Liu, Qiyue Li, Anh-Dung Dinh, Tingting Jiang, Mubarak Shah, and Chang Xu. Diffusion action segmenta- tion. In Proceedings of the IEEE/CVF International Confer- ence on Computer Vision (ICCV), pages 10139–10149, 2023. 2

    2023

  33. [40]

    Spatial focus attention for fine-grained skeleton-based action tasks

    Kaiyuan Liu, Yunheng Li, Yuanfeng Xu, Shuai Liu, and Shenglan Liu. Spatial focus attention for fine-grained skeleton-based action tasks. IEEE Signal Processing Letters, 29:1883–1887, 2022. 2

    2022

  34. [41]

    Matthew Loper, Naureen Mahmood, Javier Romero, Ger- ard Pons-Moll, and Michael J. Black. SMPL: A skinned multi-person linear model. ACM Trans. Graphics (Proc. SIGGRAPH Asia), 34(6):248:1–248:16, 2015. 5

    2015

  35. [42]

    FACT: Frame-action cross- attention temporal modeling for efficient supervised action segmentation

    Zijia Lu and Ehsan Elhamifar. FACT: Frame-action cross- attention temporal modeling for efficient supervised action segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 2

    2024

  36. [43]

    Fine-grained unsu- pervised temporal action segmentation and distributed repre- sentation for skeleton-based human motion analysis

    Hao Ma, Zaiyue Yang, and Haoyang Liu. Fine-grained unsu- pervised temporal action segmentation and distributed repre- sentation for skeleton-based human motion analysis. IEEE Transactions on Cybernetics (T-CYB), 52(12):13411–13424,

  37. [44]

    Troje, Ger- ard Pons-Moll, and Michael J

    Naureen Mahmood, Nima Ghorbani, Nikolaus F. Troje, Ger- ard Pons-Moll, and Michael J. Black. AMASS: Archive of motion capture as surface shapes. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 5442–5451, 2019. 5

    2019

  38. [45]

    Lara: Creating a dataset for human activity recognition in logistics using semantic attributes

    Friedrich Niemann, Christopher Reining, Fernando Moya Rueda, Nilah Ravi Nair, Janine Anika Steffens, Gernot A Fink, and Michael Ten Hompel. Lara: Creating a dataset for human activity recognition in logistics using semantic attributes. Sensors, 20(15):4083, 2020. 2, 5

    2020

  39. [46]

    Unsupervised human action recognition with skeletal graph laplacian and self-supervised viewpoints 10 invariance

    Giancarlo Paoletti, Jacopo Cavazza, Cigdem Beyan, and Alessio Del Bue. Unsupervised human action recognition with skeletal graph laplacian and self-supervised viewpoints 10 invariance. In Proceedings of the British Machine Vision Conference (BMVC), 2021. 1

    2021

  40. [47]

    Punnakkal, Arjun Chandrasekaran, Nikos Athanasiou, Alejandra Quiros-Ramirez, and Michael J

    Abhinanda R. Punnakkal, Arjun Chandrasekaran, Nikos Athanasiou, Alejandra Quiros-Ramirez, and Michael J. Black. BABEL: Bodies, action and behavior with english labels. In Proceedings IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pages 722–731, 2021. 2, 5

    2021

  41. [48]

    Effi- cient parameter-free clustering using first neighbor relations

    Saquib Sarfraz, Vivek Sharma, and Rainer Stiefelhagen. Effi- cient parameter-free clustering using first neighbor relations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 8934–8943,

  42. [49]

    Temporally-weighted hierarchical clustering for unsupervised action segmenta- tion

    Saquib Sarfraz, Naila Murray, Vivek Sharma, Ali Diba, Luc Van Gool, and Rainer Stiefelhagen. Temporally-weighted hierarchical clustering for unsupervised action segmenta- tion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages...

    2021

  43. [50]

    Motion words: a text-like representation of 3d skeleton sequences

    Jan Sedmidubsky, Petra Budikova, Vlastislav Dohnal, and Pavel Zezula. Motion words: a text-like representation of 3d skeleton sequences. In Proceedings of the European Con- ference on Information Retrieval (ECIR) , pages 527–541. Springer, 2020. 2

    2020

  44. [51]

    Unsupervised learning and segmentation of complex activities from video

    Fadime Sener and Angela Yao. Unsupervised learning and segmentation of complex activities from video. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8368–8376, 2018. 2, 1

    2018

  45. [52]

    Hierarchical vector quantization for unsuper- vised action segmentation

    Federico Spurio, Emad Bahrami, Gianpiero Francesca, and Juergen Gall. Hierarchical vector quantization for unsuper- vised action segmentation. In Proceedings of the AAAI Con- ference on Artificial Intelligence (AAAI) , pages 6996–7005,

  46. [53]

    Elucidating the hierarchical nature of be- havior with masked autoencoders

    Lucas Stoffl, Andy Bonnetto, St ´ephane d’Ascoli, and Alexander Mathis. Elucidating the hierarchical nature of be- havior with masked autoencoders. In Proceedings of the Eu- ropean Conference on Computer Vision (ECCV), pages 106– 125, 2024. 3

    2024

  47. [54]

    Predict & cluster: Unsupervised skeleton based action recognition

    Kun Su, Xiulong Liu, and Eli Shlizerman. Predict & cluster: Unsupervised skeleton based action recognition. InProceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 9631–9640, 2020. 1, 3, 6, 7

    2020

  48. [55]

    Hierarchical spatial-temporal network for skeleton-based temporal action segmentation

    Chenwei Tan, Tao Sun, Talas Fu, Yuhan Wang, Minjie Xu, and Shenglan Liu. Hierarchical spatial-temporal network for skeleton-based temporal action segmentation. In Chi- nese Conference on Pattern Recognition and Computer Vi- sion (PRCV), pages 28–39. Springer, 2023. 1

    2023

  49. [56]

    Tslearn, a machine learning toolkit for time series data.Jour- nal of Machine Learning Research (JMLR) , 21(118):1–6,

    Romain Tavenard, Johann Faouzi, Gilles Vandewiele, Felix Divo, Guillaume Androz, Chester Holtz, Marie Payne, Ro- man Yurchak, Marc Rußwurm, Kushal Kolar, and Eli Woods. Tslearn, a machine learning toolkit for time series data.Jour- nal of Machine Learning Research (JMLR) , 21(...

  50. [57]

    Stga-net: Spatial-temporal graph attention network for skeleton-based temporal action segmentation

    Xiaoyan Tian, Ye Jin, Zhao Zhang, Peng Liu, and Xianglong Tang. Stga-net: Spatial-temporal graph attention network for skeleton-based temporal action segmentation. In Pro- ceedings of the IEEE International Conference on Multime- dia and Expo Workshops (ICMEW) , pages 218–223. IEEE,

  51. [58]

    Spatial-temporal graph transformer network for skeleton-based temporal action segmentation

    Xiaoyan Tian, Ye Jin, Zhao Zhang, Peng Liu, and Xiang- long Tang. Spatial-temporal graph transformer network for skeleton-based temporal action segmentation. Multimedia Tools and Applications (MTAP), 83(15):44273–44297, 2024. 1

    2024

  52. [59]

    Permutation-aware activity segmentation via unsuper- vised frame-to-segment alignment

    Quoc-Huy Tran, Ahmed Mehmood, Muhammad Ahmed, Muhammad Naufil, Anas Zafar, Andrey Konin, and Zeeshan Zia. Permutation-aware activity segmentation via unsuper- vised frame-to-segment alignment. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision ...

    2024

  53. [60]

    Neural discrete representation learning

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in Neural Information Processing Systems (NeurIPS), 30, 2017. 3, 5

    2017

  54. [61]

    Efficient unsupervised temporal segmentation of human motion

    Anna V ¨ogele, Bj ¨orn Kr ¨uger, and Reinhard Klein. Efficient unsupervised temporal segmentation of human motion. In Proceedings of the ACM SIGGRAPH/Eurographics Sympo- sium on Computer Animation (SCA) , pages 167–176, 2014. 2

    2014

  55. [62]

    Unsupervised temporal segmentation of repetitive human ac- tions based on kinematic modeling and frequency analysis

    Qifei Wang, Gregorij Kurillo, Ferda Ofli, and Ruzena Bajcsy. Unsupervised temporal segmentation of repetitive human ac- tions based on kinematic modeling and frequency analysis. In Proceedings of the IEEE International Conference on 3D Vision (3DV), pages 562–570. IEEE, 2015. 2

    2015

  56. [63]

    An efficient framework for few-shot skeleton-based temporal ac- tion segmentation

    Leiyang Xu, Qiang Wang, Xiaotian Lin, and Lin Yuan. An efficient framework for few-shot skeleton-based temporal ac- tion segmentation. Proceedings of the European Conference on Computer Vision (ECCV), 232:103707, 2023. 1, 2

    2023

  57. [64]

    Temporally consistent unbal- anced optimal transport for unsupervised action segmenta- tion

    Ming Xu and Stephen Gould. Temporally consistent unbal- anced optimal transport for unsupervised action segmenta- tion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 14618– 14627, 2024. 1, 2, 6, 3

    2024

  58. [65]

    Unsupervised motion representation learning with capsule autoencoders

    Ziwei Xu, Xudong Shen, Yongkang Wong, and Mohan S Kankanhalli. Unsupervised motion representation learning with capsule autoencoders. Advances in Neural Information Processing Systems (NeurIPS), 34:3205–3217, 2021. 3

    2021

  59. [66]

    Spatial tempo- ral graph convolutional networks for skeleton-based action recognition

    Sijie Yan, Yuanjun Xiong, and Dahua Lin. Spatial tempo- ral graph convolutional networks for skeleton-based action recognition. In Proceedings of the AAAI Conference on Ar- tificial Intelligence (AAAI), 2018. 2, 6

    2018

  60. [67]

    Lac-latent action composition for skeleton-based ac- tion segmentation

    Di Yang, Yaohui Wang, Antitza Dantcheva, Quan Kong, Lorenzo Garattoni, Gianpiero Francesca, and Francois Bre- mond. Lac-latent action composition for skeleton-based ac- tion segmentation. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision (ICCV) , pag...

    2023

  61. [68]

    ASFormer: Transformer for action segmentation

    Fangqiu Yi, Hongyu Wen, and Tingting Jiang. ASFormer: Transformer for action segmentation. In Proceedings of the British Machine Vision Conference (BMVC), 2021. 2

    2021

  62. [70]

    Contrastive positive mining for unsupervised 3d action representation learning

    Haoyuan Zhang, Yonghong Hou, Wenjing Zhang, and Wan- qing Li. Contrastive positive mining for unsupervised 3d action representation learning. In Proceedings of the Euro- 11 pean Conference on Computer Vision (ECCV), pages 36–51. Springer, 2022. 1, 3

    2022

  63. [71]

    Unsupervised representation learning with long-term dynamics for skeleton based action recognition

    Nenggan Zheng, Jun Wen, Risheng Liu, Liangqu Long, Jian- hua Dai, and Zhefeng Gong. Unsupervised representation learning with long-term dynamics for skeleton based action recognition. In Proceedings of the AAAI Conference on Ar- tificial Intelligence (AAAI), 2018. 3

    2018

  64. [72]

    Feng Zhou, Fernando De la Torre, and Jessica K. Hodgins. Aligned cluster analysis for temporal segmentation of hu- man motion. In Proceedings of the IEEE International Con- ference on Automatic Face and Gesture Recognition (FG) , pages 1–7, 2008. 2

    2008

  65. [73]

    Hierarchical aligned cluster analysis for temporal clustering of human motion

    Feng Zhou, Fernando De la Torre, and Jessica K Hodgins. Hierarchical aligned cluster analysis for temporal clustering of human motion. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 35(3):582–596, 2012. 2 12 Skeleton Motion Words for Unsupervised Skele...

    2012

  66. [74]

    Implementation Details and Evaluation Metrics 6.1. Implementation Details In SMQ, both the encoder and decoder use a two-stage Temporal Convolutional Network (MS-TCN), with each stage comprising three dilated residual layers to effectively capture temporal dependencies. The co...

  67. [75]

    Each sequence lasts approximately 1 to 2 minutes and includes around 7 action instances

    Results on PKU-MMD v2 PKU-MMD v2 [38] contains 1009 skeleton sequences span- ning 41 action categories, performed by 13 subjects. Each sequence lasts approximately 1 to 2 minutes and includes around 7 action instances. The data were recorded at 30 Method MoF Edit F1@ �10, 25, ...

  68. [76]

    Additional Ablations 8.1. Impact of Disentangled Embedding To further analyze the impact of the disentangled embed- ding, we conducted an ablation study comparing two asym- metric encoder-decoder configurations. In the first con- figuration, the encoder processes each joint in...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.