RECORD
A Circular Window-based Cascade Transformer for Online Action Detection
Computer Vision and Pattern Recognition · Submitted 30 Aug 2022
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Our proposed method achieves the state-of-the-art performances on three challenging datasets THUMOS'14, TVSeries, and HDD.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Online action detection aims at the accurate action prediction of the current frame based on long historical observations. Meanwhile, it demands real-time inference on online streaming videos. In this paper, we advocate a novel and efficient principle for online action detection. It merely updates the latest and oldest historical representations in one window but reuses the intermediate ones, which have been already computed. Based on this principle, we introduce a window-based cascade Transformer with a circular historical queue, where it conducts multi-stage attentions and cascade refinement on each window. We also explore the association between online action detection and its counterpart offline action segmentation as an auxiliary task. We find that such an extra supervision helps discriminative history clustering and acts as feature augmentation for better training the classifier and cascade refinement. Our proposed method achieves the state-of-the-art performances on three challenging datasets THUMOS'14, TVSeries, and HDD. Codes will be available after acceptance.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Online action detection aims at the accurate action prediction of the current frame based on long historical observations. | background 1.00 | 0.05 | 0.02 | asserted 1.00 |
| s1 | Meanwhile, it demands real-time inference on online streaming videos. | background 1.00 | 0.06 | 0.02 | asserted 1.00 |
| s2 | In this paper, we advocate a novel and efficient principle for online action detection. | contribution 0.99 | 0.57 | 0.06 | asserted 0.88 |
| s3 | It merely updates the latest and oldest historical representations in one window but reuses the intermediate ones, which have been already computed. | method 0.84 | 0.61 | 0.08 | asserted 1.00 |
| s4 | Based on this principle, we introduce a window-based cascade Transformer with a circular historical queue, where it conducts multi-stage attentions and cascade refinement on each window. | method 0.84 | 0.90 | 0.07 | asserted 1.00 |
| s5 | We also explore the association between online action detection and its counterpart offline action segmentation as an auxiliary task. | contribution 0.69 | 0.65 | 0.03 | asserted 0.58 |
| s6 | We find that such an extra supervision helps discriminative history clustering and acts as feature augmentation for better training the classifier and cascade refinement. | result 0.52 | 0.96 | 0.10 | asserted 1.00 |
| s7 | Our proposed method achieves the state-of-the-art performances on three challenging datasets THUMOS'14, TVSeries, and HDD. | result 1.00 | 0.96 | 0.73 | asserted 1.00 |
| s8 | Codes will be available after acceptance. | future work 0.85 | 0.19 | 0.02 | asserted 0.88 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Online action detection aims at the accurate action prediction of the current frame based on long historical observations. Meanwhile, it demands real-time inference on online streaming videos. In this paper, we advocate a novel and efficient principle for online action detection. It merely updates the latest and oldest historical representations in one window but reuses the intermediate ones, which have been already computed. Based on this principle, we introduce a window-based cascade Transformer with a circular historical queue, where it conducts multi-stage attentions and cascade refinement on each window. We also explore the association between online action detection and its counterpart offline action segmentation as an auxiliary task. We find that such an extra supervision helps discriminative history clustering and acts as feature augmentation for better training the classifier and cascade refinement. Our proposed method achieves the state-of-the-art performances on three challenging datasets THUMOS'14, TVSeries, and HDD. Codes will be available after acceptance.
Human review
How the literature uses it
3 arXiv papers cite this in 4 sentences
Papers citing it · 3 sentences without a confident label
Cites it as background 1 paper
-
With the advent of Transformer models demonstrating powerful modeling capabilities in computer vision, researchers have designed various Transformer architectures for end-to-end online action detection [ref] .
Driving behavior recognition via self-discovery learning · Driving Behavior Recognition · 0.86
What it cites · 11 sentences without a confident label
Uses the method or tool of 2 papers
-
Following the previous work [ref] , we take flow features and RGB features pre-trained on Kinetics [ref] or ActivityNet [ref] from TSN as the network inputs.
The Kinetics Human Action Video Dataset · Quantitative Results · 0.93
-
To train CWC-Trans, we use Adam [ref] with a weight decay of $5 ×10^-5$.
Adam: A Method for Stochastic Optimization · Hyperparameter Settings · 1.00
Compares with 3 papers
-
The architecture of these methods includes convolution [ref] , RNN [ref] , and Transformer [ref] .
Privileged Knowledge Distillation for Online Action Detection · Quantitative Results · 0.90
-
The architecture of these methods includes convolution [ref] , RNN [ref] , and Transformer [ref] .
LAP-Net: Adaptive Features Sampling via Learning Action Progression for Online Action Detection · Quantitative Results · 0.90
-
The architecture of these methods includes convolution [ref] , RNN [ref] , and Transformer [ref] .
RED: Reinforced Encoder-Decoder Networks for Action Anticipation · Quantitative Results · 0.90
Cites as background 12 papers
-
However, the one-stage approach [ref] localize actions without proposals, roughly divided into anchor-based approach [ref] and anchor-free approach [ref] .
ActionFormer: Localizing Moments of Actions with Transformers · Offline Action Detection · 0.96
-
There are also some attempts [ref] in modeling temporal dependencies of video using Transformer.
An Image is Worth 16x16 Words, What is a Video Worth? · Transformer · 0.95
-
There are also some attempts [ref] in modeling temporal dependencies of video using Transformer.
Activity Graph Transformer for Temporal Action Localization · Transformer · 0.95
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.