Improving Viewpoint-Invariance and Temporal Consistency for Action Detection

Cedric Demonceaux; Renato Martins; Thomas Chalumeau; Yannick Porto

arxiv: 2605.22695 · v1 · pith:GYEDXH2Bnew · submitted 2026-05-21 · 💻 cs.CV

Improving Viewpoint-Invariance and Temporal Consistency for Action Detection

Yannick Porto , Renato Martins , Thomas Chalumeau , Cedric Demonceaux This is my paper

Pith reviewed 2026-05-22 05:42 UTC · model grok-4.3

classification 💻 cs.CV

keywords action detectionviewpoint invariancetemporal consistencystate-space modelsvideo analysissynthetic augmentationmulti-scale encodinguntrimmed videos

0 comments

The pith

A two-stage detector extracts motion from synthetic viewpoints at training and uses selective state-space modelling to aggregate across views and time scales.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to show that viewpoint changes and inconsistent temporal modelling are the main obstacles to reliable action detection in untrimmed videos. It proposes to solve the first problem by generating motion features from virtual camera viewpoints that exist only during training, and to solve the second by feeding those features into a multi-scale encoder built on selective state-space sequence models. If the claim holds, detectors could maintain accuracy even when the camera moves to positions never seen in real training data and could keep coherent labels across long sequences without extra post-processing. A reader interested in practical video systems would care because current appearance-based and motion-based methods each fail on one of these two requirements.

Core claim

The authors claim that their two-stage pipeline—first extracting motion features from synthetically augmented virtual viewpoints used solely at training time, then passing them through a new view-invariant multi-scale temporal encoder that relies on selective state-space sequence modelling—produces action detections that are simultaneously more invariant to camera viewpoint and more temporally coherent than previous approaches, as measured by superior results on every split of the PKU-MMD and BABEL benchmarks.

What carries the argument

The view-invariant multi-scale temporal encoder based on selective state-space sequence modelling, which aggregates motion features across multiple simulated viewpoints and across several temporal resolutions in a single forward pass.

If this is right

Detection performance improves on every split of PKU-MMD and BABEL relative to prior state-of-the-art methods.
The system maintains coherent action labels across long untrimmed sequences because the encoder explicitly models relationships at multiple time scales.
Training can exploit unlimited synthetic viewpoint diversity without changing the inference pipeline or requiring extra real camera data.
Appearance-based and motion-based cues are combined in a way that mitigates the individual weaknesses of each family of methods.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same synthetic-viewpoint trick could be reused for other video tasks such as temporal action segmentation or dense video captioning where camera angle robustness is also required.
If the domain gap remains small, the method suggests a route to adapt detectors to entirely new environments by generating virtual views rather than collecting new real footage.
The selective state-space component might be swapped for other sequence models to test whether the performance gain comes mainly from the viewpoint augmentation or from the particular choice of encoder.

Load-bearing premise

Motion features learned from synthetically generated virtual viewpoints will transfer to real camera placements without large domain shift or new artifacts that hurt final detection accuracy.

What would settle it

Run the trained model on a held-out set of real videos whose camera angles lie outside the range of both the original training data and the synthetic augmentations; if accuracy gains over baselines vanish or if motion features show visible artifacts, the central transfer claim is false.

read the original abstract

Viewpoint change invariance and action temporal consistency are critical aspects for the effective deployment of human action detection of untrimmed videos. Existing appearance-based video detection methods often struggle with limited viewpoint diversity during training, while motion-based detection approaches frequently fail to model fine-grained temporal relationships across consecutive motion windows. This paper introduces a novel two-stage action detection approach designed to improve both view-invariance and global temporal coherence properties. In the first stage, we extract motion features from augmented virtual viewpoints, solely used at training. Then, the second stage introduces a new view-invariant, multi-scale temporal encoder based on selective state-space sequence modelling to aggregate information across viewpoints and time scales. Experiments on PKU-MMD and BABEL benchmarks demonstrate that this approach significantly outperforms state-of-the-art methods in all considered splits. Code and trained models are available at: https://icb-vision-ai.github.io/HydraView-TAD

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper pairs training-only virtual viewpoint augmentation with a selective state-space temporal encoder and reports gains on PKU-MMD and BABEL, but the transfer from synthetic views to real test data lacks direct checks.

read the letter

The main takeaway is a two-stage pipeline: motion features come from synthetically augmented virtual viewpoints used only during training, then a selective state-space model aggregates across multiple time scales for better temporal consistency in action detection. This specific pairing is not in the cited prior work and gives the paper its technical edge. Code and models are released, which is useful for anyone wanting to reproduce or build on it. The experiments claim consistent outperformance over state-of-the-art methods on the two benchmarks across all splits, which is the concrete result to evaluate. The approach targets practical problems in untrimmed video detection where camera angles vary and timing matters. On the soft spots, the central assumption is that features from synthetic viewpoints will generalize cleanly to real test data without domain shift or artifacts that the temporal encoder cannot fix. The abstract does not detail ablations that isolate the viewpoint component, nor does it show viewpoint distribution statistics or controls that would rule out the temporal encoder alone driving the gains. If those checks are missing or weak in the full paper, the view-invariance story rests on an untested transfer. This is a moderate rather than fatal gap for an empirical paper, but it needs addressing for the claims to land solidly. The work is aimed at researchers doing video action detection for surveillance, sports, or robotics who need robustness to viewpoint change. A reader already working on temporal modeling or data augmentation for detection would get the most out of it. It is worth sending to peer review because the benchmarks are standard, the code is public, and the combination is new enough to merit referee scrutiny on the experiments.

Referee Report

2 major / 2 minor

Summary. The paper proposes a two-stage framework for temporal action detection in untrimmed videos to address viewpoint invariance and temporal consistency. The first stage extracts motion features exclusively from synthetically augmented virtual viewpoints during training. The second stage introduces a view-invariant multi-scale temporal encoder based on selective state-space sequence modeling to aggregate information across viewpoints and time scales. Experiments on the PKU-MMD and BABEL benchmarks report significant outperformance over state-of-the-art methods across all considered splits, with code and models released publicly.

Significance. If the reported gains are shown to stem specifically from the proposed components rather than confounding factors, the work would meaningfully advance robust action detection by combining synthetic viewpoint augmentation with modern state-space models for temporal aggregation. The public code release supports reproducibility and is a clear strength.

major comments (2)

[Abstract and §3 (method)] Abstract (first-stage description) and method overview: The central claim of improved view-invariance rests on synthetic viewpoint augmentation during training transferring to real test data in PKU-MMD and BABEL without introducing uncompensated domain shift or artifacts. No explicit controls, such as real-vs-synthetic viewpoint feature distribution comparisons or an ablation removing the augmentation stage, are described; without these, gains on viewpoint-varying splits could arise from the temporal encoder or dataset biases alone.
[Experiments] Experiments section: The claim of significant outperformance on all splits lacks reported statistical significance tests, confidence intervals, or full ablation tables isolating the contribution of viewpoint augmentation versus the selective state-space encoder. This weakens attribution of results to the proposed view-invariance mechanism.

minor comments (2)

[Abstract] Abstract: Define acronyms such as PKU-MMD and BABEL on first use for clarity.
[Figures] Figure captions: Ensure captions explicitly describe what is shown regarding viewpoint variations or temporal consistency to aid interpretation.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive and detailed feedback. The comments highlight important aspects of experimental rigor and attribution of gains, which we address point by point below. We plan to incorporate revisions to strengthen the manuscript accordingly.

read point-by-point responses

Referee: [Abstract and §3 (method)] Abstract (first-stage description) and method overview: The central claim of improved view-invariance rests on synthetic viewpoint augmentation during training transferring to real test data in PKU-MMD and BABEL without introducing uncompensated domain shift or artifacts. No explicit controls, such as real-vs-synthetic viewpoint feature distribution comparisons or an ablation removing the augmentation stage, are described; without these, gains on viewpoint-varying splits could arise from the temporal encoder or dataset biases alone.

Authors: We agree that explicit controls would strengthen attribution of the view-invariance improvements. In the revised manuscript we will add an ablation that disables the virtual viewpoint augmentation stage while keeping the selective state-space encoder fixed, thereby isolating its contribution on the viewpoint-varying splits. The synthetic viewpoints are generated via established geometric transformations calibrated to the camera setups in PKU-MMD and BABEL; we will include a short qualitative comparison of motion-feature distributions between real and synthetic views to address potential domain-shift concerns. revision: yes
Referee: [Experiments] Experiments section: The claim of significant outperformance on all splits lacks reported statistical significance tests, confidence intervals, or full ablation tables isolating the contribution of viewpoint augmentation versus the selective state-space encoder. This weakens attribution of results to the proposed view-invariance mechanism.

Authors: We acknowledge the value of statistical reporting. The revised version will report 95% confidence intervals and paired significance tests for the main results on both benchmarks. We will also expand the ablation tables (moving key rows from the supplement into the main paper where space allows) to separately quantify the gains from viewpoint augmentation and from the state-space temporal encoder, thereby clarifying the source of the reported improvements. revision: yes

Circularity Check

0 steps flagged

Empirical pipeline evaluated on external benchmarks with no internal reductions

full rationale

The paper proposes a two-stage empirical method: synthetic viewpoint augmentation for motion feature extraction during training, followed by a selective state-space temporal encoder for view-invariance and consistency. Performance claims rest entirely on comparisons to external SOTA methods on public benchmarks PKU-MMD and BABEL. No equations, predictions, or first-principles derivations are present that reduce outputs to fitted parameters, self-definitions, or self-citation chains by construction. The approach is a standard ML pipeline whose validity is assessed via independent test sets rather than internal tautologies.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The approach rests on standard deep-learning assumptions about feature learning and transfer from synthetic augmentations; no new physical entities or ad-hoc constants are introduced in the abstract.

axioms (1)

domain assumption Neural networks trained on synthetically augmented viewpoints will learn features that generalize to real viewpoint changes.
This premise underpins the first stage and is required for the claimed invariance benefit.

pith-pipeline@v0.9.0 · 5689 in / 1256 out tokens · 55040 ms · 2026-05-22T05:42:45.623953+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

38 extracted references · 38 canonical work pages · 1 internal anchor

[1]

Improving Viewpoint-Invariance and Temporal Consistency for Action Detection

INTRODUCTION Temporal Action Detection (TAD) aims at recognizing and lo- calizing human actions in long, untrimmed video sequences. Unlike trimmed action recognition, TAD requires not only identifying the action category but also accurately determin- ing its temporal boundaries, making it a fundamental yet challenging problem for activity understanding. A...

work page internal anchor Pith review Pith/arXiv arXiv 2026
[2]

Window Encoding SWGCN shared

work page
[3]

Multi-view Multi-scale Temporal Encoding

work page
[4]

Detection Results 1 v1 n v1 2 v1 1 v2 n v2 2 v2 Fig. 2. Overview of our temporal action detection method with two viewpoints. For each input video viewpoint, an untrimmed sequence is encoded with a spatio-temporal encoder to generate features with improved view invariance. These features are then reﬁned by our multi-view and multi-scale temporal encoder (...

work page
[5]

RELATED WORK 2.1. Video-based Action Detection Early approaches to temporal action detection were largely proposal-based, drawing inspiration from object detection to generate candidate temporal segments. Although ef- fective for sparsely annotated videos, these methods were computationally expensive and poorly suited for dense per- frame predictions. To ...

work page
[6]

METHOD This section presents the designed temporal action detection method to encapsulate properties from multiple viewpoints and long temporal sequences, as illustrated in Fig. 2. Fol- lowing recent TAD methodologies [ 4, 7, 5], we start by pre- processing the input video sequence in small windows of time which are then encoded to learn relations along t...

work page
[7]

point- ing

EXPERIMENTS Experimental Setup. We train the motion encoder SWGCN with a feature dimension d = 384 . The HydraView model responsible to enforce temporal coherence and viewpoint change invariance is composed of 3 ViewMamba blocks (each one in one scale), with an output dimension of 192 in each 2D convolution, a view stride of sv = 2 and a di- lation rate o...

work page 2053
[8]

Conseil Re- gional de Bourgogne-Franche-Comte

CONCLUSION This paper introduces a novel temporal action detection framework that jointly improves view invariance and tem- poral consistency. While existing video-based approaches generally lack robustness to viewpoint variations, motion- based detection methods often fail to model temporal re- lationships across adjacent windows. To overcome these limit...

work page
[9]

Lac-latent action composition for skeleton-based ac- tion segmentation,

Di Y ang, Y aohui Wang, Antitza Dantcheva, Quan Kong, Lorenzo Garattoni, Gianpiero Francesca, and Francois Bre- mond, “Lac-latent action composition for skeleton-based ac- tion segmentation,” in ICCV, 2023

work page 2023
[10]

Duoclr: Dual-surrogate contrastive learning for skeleton-based human action segmentation,

Haitao Tian, “Duoclr: Dual-surrogate contrastive learning for skeleton-based human action segmentation,” in ICCV, 2025

work page 2025
[11]

Skeleton motion words for unsupervised skeleton-based temporal action segmentation,

Uzay G ¨okay, Federico Spurio, Dominik R Bach, and Juergen Gall, “Skeleton motion words for unsupervised skeleton-based temporal action segmentation,” in ICCV, 2025

work page 2025
[12]

Pdan: Pyramid dilated attention network for action detection,

Rui Dai, Srijan Das, Luca Minciullo, Lorenzo Garattoni, Gi- anpiero Francesca, and Franc ¸ois Bremond, “Pdan: Pyramid dilated attention network for action detection,” in WACV, 2021

work page 2021
[13]

Dual detrs for multi-label temporal action de- tection,

Y uhan Zhu, Guozhen Zhang, Jing Tan, Gangshan Wu, and Limin Wang, “Dual detrs for multi-label temporal action de- tection,” in CVPR, 2024

work page 2024
[14]

Ms-temba: Multi-scale temporal mamba for efﬁcient temporal action detection,

Arkaprava Sinha, Monish Soundar Raj, Pu Wang, Ahmed Helmy, and Srijan Das, “Ms-temba: Multi-scale temporal mamba for efﬁcient temporal action detection,” CVPR, 2026

work page 2026
[15]

Ms-tct: Multi-scale temporal con- vtransformer for action detection,

Rui Dai, Srijan Das, Kumara Kahatapitiya, Michael S Ryoo, and Franc ¸ois Br´emond, “Ms-tct: Multi-scale temporal con- vtransformer for action detection,” in CVPR, 2022

work page 2022
[16]

Toy- ota smarthome untrimmed: Real-world untrimmed videos for activity detection,

Rui Dai, Srijan Das, Saurav Sharma, Luca Minciullo, Lorenzo Garattoni, Francois Bremond, and Gianpiero Francesca, “Toy- ota smarthome untrimmed: Real-world untrimmed videos for activity detection,” TPAMI, 2023

work page 2023
[17]

An empirical study on temporal modeling for online action detec- tion,

Wen Wang, Xiaojiang Peng, Y u Qiao, and Jian Cheng, “An empirical study on temporal modeling for online action detec- tion,” CISIS, 2022

work page 2022
[18]

Mamba: Linear-time sequence mod- eling with selective state spaces,

Albert Gu and Tri Dao, “Mamba: Linear-time sequence mod- eling with selective state spaces,” in First conference on lan- guage modeling, 2024

work page 2024
[19]

Vision mamba: Efﬁcient visual representation learning with bidirectional state space model,

Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang, “Vision mamba: Efﬁcient visual representation learning with bidirectional state space model,” in ICML, 2024

work page 2024
[20]

Vmamba: Visual state space model,

Y ue Liu, Y unjie Tian, Y uzhong Zhao, Hongtian Y u, Lingxi Xie, Y aowei Wang, Qixiang Y e, Jianbin Jiao, and Y unfan Liu, “Vmamba: Visual state space model,” NeurIPS, 2024

work page 2024
[21]

Jamma: Ultra-lightweight lo- cal feature matching with joint mamba,

Xiaoyong Lu and Songlin Du, “Jamma: Ultra-lightweight lo- cal feature matching with joint mamba,” in CVPR, 2025

work page 2025
[22]

Harnessing temporal causal- ity for advanced temporal action detection,

Shuming Liu, Lin Sui, Chen-Lin Zhang, Fangzhou Mu, Chen Zhao, and Bernard Ghanem, “Harnessing temporal causal- ity for advanced temporal action detection,” arXiv preprint arXiv:2407.17792, 2024

work page arXiv 2024
[23]

Video mamba suite: State space model as a versatile alterna- tive for video understanding,

Guo Chen, Yifei Huang, Jilan Xu, Baoqi Pei, Zhe Chen, Zhiqi Li, Jiahao Wang, Kunchang Li, Tong Lu, and Limin Wang, “Video mamba suite: State space model as a versatile alterna- tive for video understanding,” IJCV, 2026

work page 2026
[24]

Usdrl: Uniﬁed skeleton-based dense represen- tation learning with multi-grained feature decorrelation,

Wanjiang Weng, Hongsong Wang, Junbo Wang, Lei He, and Guosen Xie, “Usdrl: Uniﬁed skeleton-based dense represen- tation learning with multi-grained feature decorrelation,” in AAAI, 2025

work page 2025
[25]

Stitch, contrast, and segment: Learning a human action segmentation model using trimmed skeleton videos,

Haitao Tian and Pierre Payeur, “Stitch, contrast, and segment: Learning a human action segmentation model using trimmed skeleton videos,” in AAAI, 2025, vol. 39

work page 2025
[26]

Spatial temporal graph convolutional networks for skeleton-based action recog- nition,

Sijie Y an, Y uanjun Xiong, and Dahua Lin, “Spatial temporal graph convolutional networks for skeleton-based action recog- nition,” in AAAI, 2018

work page 2018
[27]

Two- stream adaptive graph convolutional networks for skeleton- based action recognition,

Lei Shi, Yifan Zhang, Jian Cheng, and Hanqing Lu, “Two- stream adaptive graph convolutional networks for skeleton- based action recognition,” in CVPR, 2019

work page 2019
[28]

Unik: A uniﬁed framework for real-world skeleton-based action recognition,

Di Y ang, Y aohui Wang, Antitza Dantcheva, Lorenzo Garattoni, Gianpiero Francesca, and Francois Bremond, “Unik: A uniﬁed framework for real-world skeleton-based action recognition,” BMVC, 2021

work page 2021
[29]

Revealing key details to see differences: A novel prototypical perspective for skeleton-based action recog- nition,

Hongda Liu, Y unfan Liu, Min Ren, Hao Wang, Y unlong Wang, and Zhenan Sun, “Revealing key details to see differences: A novel prototypical perspective for skeleton-based action recog- nition,” in CVPR, 2025

work page 2025
[30]

Pku-mmd: A large scale benchmark for continuous multi-modal human action understanding,

Liu Chunhui, Hu Y ueyu, Li Y anghao, Song Sijie, and Liu Ji- aying, “Pku-mmd: A large scale benchmark for continuous multi-modal human action understanding,” ACM Multimedia workshop, 2017

work page 2017
[31]

BABEL: Bodies, action and behavior with english labels,

Abhinanda R. Punnakkal, Arjun Chandrasekaran, Nikos Athanasiou, Alejandra Quiros-Ramirez, and Michael J. Black, “BABEL: Bodies, action and behavior with english labels,” in CVPR, June 2021

work page 2021
[32]

AMASS: Archive of motion capture as surface shapes,

Naureen Mahmood, Nima Ghorbani, Nikolaus F. Troje, Ger- ard Pons-Moll, and Michael J. Black, “AMASS: Archive of motion capture as surface shapes,” in ICCV, Oct. 2019

work page 2019
[33]

Frame-level label reﬁnement for skeleton-based weakly-supervised action recognition,

Qing Y u and Kent Fujiwara, “Frame-level label reﬁnement for skeleton-based weakly-supervised action recognition,” AAAI, Jun. 2023

work page 2023
[34]

Temporally consistent unbal- anced optimal transport for unsupervised action segmentation,

Ming Xu and Stephen Gould, “Temporally consistent unbal- anced optimal transport for unsupervised action segmentation,” in CVPR, 2024

work page 2024
[35]

Online human action detec- tion using joint classiﬁcation-regression recurrent neural net- works,

Y anghao Li, Cuiling Lan, Junliang Xing, Wenjun Zeng, Chun- feng Y uan, and Jiaying Liu, “Online human action detec- tion using joint classiﬁcation-regression recurrent neural net- works,” in ECCV. Springer, 2016

work page 2016
[36]

Skeleton boxes: Solving skeleton based action detection with a single deep convolutional neural network,

Bo Li, Huahui Chen, Y ucheng Chen, Y uchao Dai, and Mingyi He, “Skeleton boxes: Solving skeleton based action detection with a single deep convolutional neural network,” in ICMEW. IEEE, 2017

work page 2017
[37]

Hierarchically self-supervised transformer forhuman skeleton representation learning,

Y uxiao Chen, Long Zhao, Jianbo Y uan, Y u Tian, Zhaoyang Xia, Shijie Geng, Ligong Han, and Dimitris N. Metaxas, “Hierarchically self-supervised transformer forhuman skeleton representation learning,” in ECCV, 2022

work page 2022
[38]

Temporal convolutional networks for ac- tion segmentation and detection,

Colin Lea, Michael D Flynn, Rene Vidal, Austin Reiter, and Gregory D Hager, “Temporal convolutional networks for ac- tion segmentation and detection,” in CVPR, 2017. [Supplementary Material] – Improving Viewpoint-Invariance and Temporal Consistency for Action Detection In this supplementary material, we provide additional implementation details and experi...

work page 2017

[1] [1]

Improving Viewpoint-Invariance and Temporal Consistency for Action Detection

INTRODUCTION Temporal Action Detection (TAD) aims at recognizing and lo- calizing human actions in long, untrimmed video sequences. Unlike trimmed action recognition, TAD requires not only identifying the action category but also accurately determin- ing its temporal boundaries, making it a fundamental yet challenging problem for activity understanding. A...

work page internal anchor Pith review Pith/arXiv arXiv 2026

[2] [2]

Window Encoding SWGCN shared

work page

[3] [3]

Multi-view Multi-scale Temporal Encoding

work page

[4] [4]

Detection Results 1 v1 n v1 2 v1 1 v2 n v2 2 v2 Fig. 2. Overview of our temporal action detection method with two viewpoints. For each input video viewpoint, an untrimmed sequence is encoded with a spatio-temporal encoder to generate features with improved view invariance. These features are then reﬁned by our multi-view and multi-scale temporal encoder (...

work page

[5] [5]

RELATED WORK 2.1. Video-based Action Detection Early approaches to temporal action detection were largely proposal-based, drawing inspiration from object detection to generate candidate temporal segments. Although ef- fective for sparsely annotated videos, these methods were computationally expensive and poorly suited for dense per- frame predictions. To ...

work page

[6] [6]

METHOD This section presents the designed temporal action detection method to encapsulate properties from multiple viewpoints and long temporal sequences, as illustrated in Fig. 2. Fol- lowing recent TAD methodologies [ 4, 7, 5], we start by pre- processing the input video sequence in small windows of time which are then encoded to learn relations along t...

work page

[7] [7]

point- ing

EXPERIMENTS Experimental Setup. We train the motion encoder SWGCN with a feature dimension d = 384 . The HydraView model responsible to enforce temporal coherence and viewpoint change invariance is composed of 3 ViewMamba blocks (each one in one scale), with an output dimension of 192 in each 2D convolution, a view stride of sv = 2 and a di- lation rate o...

work page 2053

[8] [8]

Conseil Re- gional de Bourgogne-Franche-Comte

CONCLUSION This paper introduces a novel temporal action detection framework that jointly improves view invariance and tem- poral consistency. While existing video-based approaches generally lack robustness to viewpoint variations, motion- based detection methods often fail to model temporal re- lationships across adjacent windows. To overcome these limit...

work page

[9] [9]

Lac-latent action composition for skeleton-based ac- tion segmentation,

Di Y ang, Y aohui Wang, Antitza Dantcheva, Quan Kong, Lorenzo Garattoni, Gianpiero Francesca, and Francois Bre- mond, “Lac-latent action composition for skeleton-based ac- tion segmentation,” in ICCV, 2023

work page 2023

[10] [10]

Duoclr: Dual-surrogate contrastive learning for skeleton-based human action segmentation,

Haitao Tian, “Duoclr: Dual-surrogate contrastive learning for skeleton-based human action segmentation,” in ICCV, 2025

work page 2025

[11] [11]

Skeleton motion words for unsupervised skeleton-based temporal action segmentation,

Uzay G ¨okay, Federico Spurio, Dominik R Bach, and Juergen Gall, “Skeleton motion words for unsupervised skeleton-based temporal action segmentation,” in ICCV, 2025

work page 2025

[12] [12]

Pdan: Pyramid dilated attention network for action detection,

Rui Dai, Srijan Das, Luca Minciullo, Lorenzo Garattoni, Gi- anpiero Francesca, and Franc ¸ois Bremond, “Pdan: Pyramid dilated attention network for action detection,” in WACV, 2021

work page 2021

[13] [13]

Dual detrs for multi-label temporal action de- tection,

Y uhan Zhu, Guozhen Zhang, Jing Tan, Gangshan Wu, and Limin Wang, “Dual detrs for multi-label temporal action de- tection,” in CVPR, 2024

work page 2024

[14] [14]

Ms-temba: Multi-scale temporal mamba for efﬁcient temporal action detection,

Arkaprava Sinha, Monish Soundar Raj, Pu Wang, Ahmed Helmy, and Srijan Das, “Ms-temba: Multi-scale temporal mamba for efﬁcient temporal action detection,” CVPR, 2026

work page 2026

[15] [15]

Ms-tct: Multi-scale temporal con- vtransformer for action detection,

Rui Dai, Srijan Das, Kumara Kahatapitiya, Michael S Ryoo, and Franc ¸ois Br´emond, “Ms-tct: Multi-scale temporal con- vtransformer for action detection,” in CVPR, 2022

work page 2022

[16] [16]

Toy- ota smarthome untrimmed: Real-world untrimmed videos for activity detection,

Rui Dai, Srijan Das, Saurav Sharma, Luca Minciullo, Lorenzo Garattoni, Francois Bremond, and Gianpiero Francesca, “Toy- ota smarthome untrimmed: Real-world untrimmed videos for activity detection,” TPAMI, 2023

work page 2023

[17] [17]

An empirical study on temporal modeling for online action detec- tion,

Wen Wang, Xiaojiang Peng, Y u Qiao, and Jian Cheng, “An empirical study on temporal modeling for online action detec- tion,” CISIS, 2022

work page 2022

[18] [18]

Mamba: Linear-time sequence mod- eling with selective state spaces,

Albert Gu and Tri Dao, “Mamba: Linear-time sequence mod- eling with selective state spaces,” in First conference on lan- guage modeling, 2024

work page 2024

[19] [19]

Vision mamba: Efﬁcient visual representation learning with bidirectional state space model,

Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang, “Vision mamba: Efﬁcient visual representation learning with bidirectional state space model,” in ICML, 2024

work page 2024

[20] [20]

Vmamba: Visual state space model,

Y ue Liu, Y unjie Tian, Y uzhong Zhao, Hongtian Y u, Lingxi Xie, Y aowei Wang, Qixiang Y e, Jianbin Jiao, and Y unfan Liu, “Vmamba: Visual state space model,” NeurIPS, 2024

work page 2024

[21] [21]

Jamma: Ultra-lightweight lo- cal feature matching with joint mamba,

Xiaoyong Lu and Songlin Du, “Jamma: Ultra-lightweight lo- cal feature matching with joint mamba,” in CVPR, 2025

work page 2025

[22] [22]

Harnessing temporal causal- ity for advanced temporal action detection,

Shuming Liu, Lin Sui, Chen-Lin Zhang, Fangzhou Mu, Chen Zhao, and Bernard Ghanem, “Harnessing temporal causal- ity for advanced temporal action detection,” arXiv preprint arXiv:2407.17792, 2024

work page arXiv 2024

[23] [23]

Video mamba suite: State space model as a versatile alterna- tive for video understanding,

Guo Chen, Yifei Huang, Jilan Xu, Baoqi Pei, Zhe Chen, Zhiqi Li, Jiahao Wang, Kunchang Li, Tong Lu, and Limin Wang, “Video mamba suite: State space model as a versatile alterna- tive for video understanding,” IJCV, 2026

work page 2026

[24] [24]

Usdrl: Uniﬁed skeleton-based dense represen- tation learning with multi-grained feature decorrelation,

Wanjiang Weng, Hongsong Wang, Junbo Wang, Lei He, and Guosen Xie, “Usdrl: Uniﬁed skeleton-based dense represen- tation learning with multi-grained feature decorrelation,” in AAAI, 2025

work page 2025

[25] [25]

Stitch, contrast, and segment: Learning a human action segmentation model using trimmed skeleton videos,

Haitao Tian and Pierre Payeur, “Stitch, contrast, and segment: Learning a human action segmentation model using trimmed skeleton videos,” in AAAI, 2025, vol. 39

work page 2025

[26] [26]

Spatial temporal graph convolutional networks for skeleton-based action recog- nition,

Sijie Y an, Y uanjun Xiong, and Dahua Lin, “Spatial temporal graph convolutional networks for skeleton-based action recog- nition,” in AAAI, 2018

work page 2018

[27] [27]

Two- stream adaptive graph convolutional networks for skeleton- based action recognition,

Lei Shi, Yifan Zhang, Jian Cheng, and Hanqing Lu, “Two- stream adaptive graph convolutional networks for skeleton- based action recognition,” in CVPR, 2019

work page 2019

[28] [28]

Unik: A uniﬁed framework for real-world skeleton-based action recognition,

Di Y ang, Y aohui Wang, Antitza Dantcheva, Lorenzo Garattoni, Gianpiero Francesca, and Francois Bremond, “Unik: A uniﬁed framework for real-world skeleton-based action recognition,” BMVC, 2021

work page 2021

[29] [29]

Revealing key details to see differences: A novel prototypical perspective for skeleton-based action recog- nition,

Hongda Liu, Y unfan Liu, Min Ren, Hao Wang, Y unlong Wang, and Zhenan Sun, “Revealing key details to see differences: A novel prototypical perspective for skeleton-based action recog- nition,” in CVPR, 2025

work page 2025

[30] [30]

Pku-mmd: A large scale benchmark for continuous multi-modal human action understanding,

Liu Chunhui, Hu Y ueyu, Li Y anghao, Song Sijie, and Liu Ji- aying, “Pku-mmd: A large scale benchmark for continuous multi-modal human action understanding,” ACM Multimedia workshop, 2017

work page 2017

[31] [31]

BABEL: Bodies, action and behavior with english labels,

Abhinanda R. Punnakkal, Arjun Chandrasekaran, Nikos Athanasiou, Alejandra Quiros-Ramirez, and Michael J. Black, “BABEL: Bodies, action and behavior with english labels,” in CVPR, June 2021

work page 2021

[32] [32]

AMASS: Archive of motion capture as surface shapes,

Naureen Mahmood, Nima Ghorbani, Nikolaus F. Troje, Ger- ard Pons-Moll, and Michael J. Black, “AMASS: Archive of motion capture as surface shapes,” in ICCV, Oct. 2019

work page 2019

[33] [33]

Frame-level label reﬁnement for skeleton-based weakly-supervised action recognition,

Qing Y u and Kent Fujiwara, “Frame-level label reﬁnement for skeleton-based weakly-supervised action recognition,” AAAI, Jun. 2023

work page 2023

[34] [34]

Temporally consistent unbal- anced optimal transport for unsupervised action segmentation,

Ming Xu and Stephen Gould, “Temporally consistent unbal- anced optimal transport for unsupervised action segmentation,” in CVPR, 2024

work page 2024

[35] [35]

Online human action detec- tion using joint classiﬁcation-regression recurrent neural net- works,

Y anghao Li, Cuiling Lan, Junliang Xing, Wenjun Zeng, Chun- feng Y uan, and Jiaying Liu, “Online human action detec- tion using joint classiﬁcation-regression recurrent neural net- works,” in ECCV. Springer, 2016

work page 2016

[36] [36]

Skeleton boxes: Solving skeleton based action detection with a single deep convolutional neural network,

Bo Li, Huahui Chen, Y ucheng Chen, Y uchao Dai, and Mingyi He, “Skeleton boxes: Solving skeleton based action detection with a single deep convolutional neural network,” in ICMEW. IEEE, 2017

work page 2017

[37] [37]

Hierarchically self-supervised transformer forhuman skeleton representation learning,

Y uxiao Chen, Long Zhao, Jianbo Y uan, Y u Tian, Zhaoyang Xia, Shijie Geng, Ligong Han, and Dimitris N. Metaxas, “Hierarchically self-supervised transformer forhuman skeleton representation learning,” in ECCV, 2022

work page 2022

[38] [38]

Temporal convolutional networks for ac- tion segmentation and detection,

Colin Lea, Michael D Flynn, Rene Vidal, Austin Reiter, and Gregory D Hager, “Temporal convolutional networks for ac- tion segmentation and detection,” in CVPR, 2017. [Supplementary Material] – Improving Viewpoint-Invariance and Temporal Consistency for Action Detection In this supplementary material, we provide additional implementation details and experi...

work page 2017