REVIEW 6 cited by
ASFormer: Transformer for Action Segmentation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Algorithms for the action segmentation task typically use temporal models to predict what action is occurring at each frame for a minute-long daily activity. Recent studies have shown the potential of Transformer in modeling the relations among elements in sequential data. However, there are several major concerns when directly applying the Transformer to the action segmentation task, such as the lack of inductive biases with small training sets, the deficit in processing long input sequence, and the limitation of the decoder architecture to utilize temporal relations among multiple action segments to refine the initial predictions. To address these concerns, we design an efficient Transformer-based model for action segmentation task, named ASFormer, with three distinctive characteristics: (i) We explicitly bring in the local connectivity inductive priors because of the high locality of features. It constrains the hypothesis space within a reliable scope, and is beneficial for the action segmentation task to learn a proper target function with small training sets. (ii) We apply a pre-defined hierarchical representation pattern that efficiently handles long input sequences. (iii) We carefully design the decoder to refine the initial predictions from the encoder. Extensive experiments on three public datasets demonstrate that effectiveness of our methods. Code is available at \url{https://github.com/ChinaYi/ASFormer}.
Forward citations
Cited by 6 Pith papers
-
Fine-Grained Action Segmentation for Renorrhaphy in Robot-Assisted Partial Nephrectomy
Introduces SIA-RAPN benchmark of 50 clinical videos with 12 fine-grained renorrhaphy action labels and evaluates four temporal segmentation models, with DiffAct leading on most metrics.
-
P-JEPA: Procedural Video Representation Learning via Joint Embedding Predictive Architecture
P-JEPA enables long-form procedural video understanding by predicting pooled masked latent vectors in a dense frame-aligned action space, achieving SOTA fine-grained action classification on EgoExo4D with an order of ...
-
Boundary-Centric Active Learning for Temporal Action Segmentation
B-ACT improves label efficiency in temporal action segmentation by selecting only boundary frames for annotation via a two-stage uncertainty-driven process that fuses neighborhood uncertainty, class ambiguity, and tem...
-
IMPACT-Scribe: Interactive Temporal Action Segmentation with Boundary Scribbles and Query Planning
IMPACT-Scribe is a correction-driven interactive system that combines uncertainty-aware boundary scribbles, local proposal modeling, cost-aware query planning, structured propagation, and correction-driven adaptation ...
-
HOI-aware Adaptive Network for Weakly-supervised Action Segmentation
AdaAct employs a HOI encoder and two-branch hypernetwork to adaptively adjust temporal encoding parameters based on video-level human-object interactions for improved weakly-supervised action segmentation.
-
Geo-RepNet: Geometry-Aware Representation Learning for Surgical Phase Recognition in Endoscopic Submucosal Dissection
Adding depth-derived geometric priors to a RepVGG backbone improves surgical phase classification on a new nine-phase ESD dataset.
Discussion (0). Sign in to comment.