REVIEW 6 cited by
Hypergraph Transformer for Skeleton-based Action Recognition
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Skeleton-based action recognition aims to recognize human actions given human joint coordinates with skeletal interconnections. By defining a graph with joints as vertices and their natural connections as edges, previous works successfully adopted Graph Convolutional networks (GCNs) to model joint co-occurrences and achieved superior performance. More recently, a limitation of GCNs is identified, i.e., the topology is fixed after training. To relax such a restriction, Self-Attention (SA) mechanism has been adopted to make the topology of GCNs adaptive to the input, resulting in the state-of-the-art hybrid models. Concurrently, attempts with plain Transformers have also been made, but they still lag behind state-of-the-art GCN-based methods due to the lack of structural prior. Unlike hybrid models, we propose a more elegant solution to incorporate the bone connectivity into Transformer via a graph distance embedding. Our embedding retains the information of skeletal structure during training, whereas GCNs merely use it for initialization. More importantly, we reveal an underlying issue of graph models in general, i.e., pairwise aggregation essentially ignores the high-order kinematic dependencies between body joints. To fill this gap, we propose a new self-attention (SA) mechanism on hypergraph, termed Hypergraph Self-Attention (HyperSA), to incorporate intrinsic higher-order relations into the model. We name the resulting model Hyperformer, and it beats state-of-the-art graph models w.r.t. accuracy and efficiency on NTU RGB+D, NTU RGB+D 120, and Northwestern-UCLA datasets.
Forward citations
Cited by 6 Pith papers
-
Partial Skeleton Visibility for Action Recognition: A Constrained Field-of-View Approach
PartialVisGraph is a hypergraph framework with learnable virtual hyperedges and a sample-adaptive transformer incorporating visibility prior, achieving reported SOTA gains up to 68.8% under simulated partial FoV on NT...
-
STAR: Skeletal Token Alignment and Rearrangement for Interaction Recognition
Skeleton-only interaction recognition that aligns skeleton and video features during training outperforms prior state-of-the-art on four benchmarks.
-
Beyond Pairwise Relations: Dynamic Manipulation Hypergraphs for Vision-Based Human Activity Recognition
Dynamic manipulation hypergraphs with time-varying multi-entity templates outperform matched pairwise graphs and static hypergraphs on fine-grained manipulation recognition.
-
Patch-Wise Hypergraph Contrastive Learning with Dual Normal Distribution Weighting for Multi-Domain Stain Transfer
STNHCL uses hypergraph modeling of patch relationships and dual Gaussian weighting of negative samples to achieve state-of-the-art multi-domain stain transfer while addressing limitations of cycle consistency.
-
CascadeFormer: A Family of Two-stage Cascading Transformers for Skeleton-based Human Action Recognition
A masked-pretrained skeleton transformer with a second fine-tuning transformer and cross-attention fusion reaches 94.66% on Penn Action, 91.16% on N-UCLA, and 81.01%/88.17% on NTU RGB+D 60 cross-subject/cross-view.
-
FLASH: Efficient Impact Fall Detection with Unified Hypergraph State-Space Model
FLASH uses single-matrix hypergraph convolutions plus Mamba to detect fall impact frames efficiently, but it is less accurate than its own dual-hypergraph predecessor.
Discussion (0). Sign in to comment.