RECORD
G-TAD: Sub-Graph Localization for Temporal Action Detection
Computer Vision and Pattern Recognition · Submitted 26 Nov 2019
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Extensive experiments show that G-TAD is capable of finding effective video context without extra supervision and achieves state-of-the-art performance on two detection benchmarks.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Temporal action detection is a fundamental yet challenging task in video understanding. Video context is a critical cue to effectively detect actions, but current works mainly focus on temporal context, while neglecting semantic context as well as other important context properties. In this work, we propose a graph convolutional network (GCN) model to adaptively incorporate multi-level semantic context into video features and cast temporal action detection as a sub-graph localization problem. Specifically, we formulate video snippets as graph nodes, snippet-snippet correlations as edges, and actions associated with context as target sub-graphs. With graph convolution as the basic operation, we design a GCN block called GCNeXt, which learns the features of each node by aggregating its context and dynamically updates the edges in the graph. To localize each sub-graph, we also design an SGAlign layer to embed each sub-graph into the Euclidean space. Extensive experiments show that G-TAD is capable of finding effective video context without extra supervision and achieves state-of-the-art performance on two detection benchmarks. On ActivityNet-1.3, it obtains an average mAP of 34.09%; on THUMOS14, it reaches 51.6% at IoU@0.5 when combined with a proposal processing method. G-TAD code is publicly available at https://github.com/frostinassiky/gtad.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Temporal action detection is a fundamental yet challenging task in video understanding. | background 1.00 | 0.04 | 0.02 | asserted 1.00 |
| s1 | Video context is a critical cue to effectively detect actions, but current works mainly focus on temporal context, while neglecting semantic context as well as other important context properties. | gap 0.99 | 0.08 | 0.03 | asserted 1.00 |
| s2 | In this work, we propose a graph convolutional network (GCN) model to adaptively incorporate multi-level semantic context into video features and cast temporal action detection as a sub-graph localization problem. | contribution 0.87 | 0.90 | 0.14 | asserted 0.98 |
| s3 | Specifically, we formulate video snippets as graph nodes, snippet-snippet correlations as edges, and actions associated with context as target sub-graphs. | method 0.99 | 0.86 | 0.03 | asserted 0.97 |
| s4 | With graph convolution as the basic operation, we design a GCN block called GCNeXt, which learns the features of each node by aggregating its context and dynamically updates the edges in the graph. | method 0.99 | 0.94 | 0.04 | asserted 1.00 |
| s5 | To localize each sub-graph, we also design an SGAlign layer to embed each sub-graph into the Euclidean space. | method 0.97 | 0.94 | 0.03 | asserted 0.99 |
| s6 | Extensive experiments show that G-TAD is capable of finding effective video context without extra supervision and achieves state-of-the-art performance on two detection benchmarks. | result 0.98 | 0.96 | 0.84 | asserted 0.96 |
| s7 | On ActivityNet-1.3, it obtains an average mAP of 34.09%; on THUMOS14, it reaches 51.6% at IoU@0.5 when combined with a proposal processing method. | result 1.00 | 0.96 | 0.12 | asserted 1.00 |
| s8 | G-TAD code is publicly available at https://github.com/frostinassiky/gtad. | contribution 0.70 | 0.45 | 0.02 | asserted 0.99 |
What was examined, and what was not
- arXiv listing abstract v2
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Temporal action detection is a fundamental yet challenging task in video understanding. Video context is a critical cue to effectively detect actions, but current works mainly focus on temporal context, while neglecting semantic context as well as other important context properties. In this work, we propose a graph convolutional network (GCN) model to adaptively incorporate multi-level semantic context into video features and cast temporal action detection as a sub-graph localization problem. Specifically, we formulate video snippets as graph nodes, snippet-snippet correlations as edges, and actions associated with context as target sub-graphs. With graph convolution as the basic operation, we design a GCN block called GCNeXt, which learns the features of each node by aggregating its context and dynamically updates the edges in the graph. To localize each sub-graph, we also design an SGAlign layer to embed each sub-graph into the Euclidean space. Extensive experiments show that G-TAD is capable of finding effective video context without extra supervision and achieves state-of-the-art performance on two detection benchmarks. On ActivityNet-1.3, it obtains an average mAP of 34.09%; on THUMOS14, it reaches 51.6% at IoU@0.5 when combined with a proposal processing method. G-TAD code is publicly available at https://github.com/frostinassiky/gtad.
Human review
How the literature uses it
1 arXiv paper cite this in 1 sentence
Papers citing it
Cites it as background 1 paper
-
In recent years, graph convolutional networks (GNN) have drawn increasing attention due to their successful applications in various tasks [ref] .
Multi-Modal Interaction Graph Convolutional Network for Temporal Language Localization in Videos · Graph Convolutional Networks · 1.00
What it cites · 2 sentences without a confident label
Uses the method or tool of 2 papers
-
For THUMOS-14, the video features are extracted from Kinetics [ref] pretrained TSN model [ref] with $σ= 5$.
The Kinetics Human Action Video Dataset · Implementation Details · 0.97
-
In this paper, we endow video context with all the above properties by casting action detection as a sub-graph localization problem based on a graph convolutional network (GCN) [ref] .
Semi-Supervised Classification with Graph Convolutional Networks · Introduction · 0.93
Cites as background 6 papers
-
In action detection, Zeng et al. [ref] consider temporal action proposals as nodes in a graph, and refine their boundaries and classification scores based on the established proposal-proposal dependencies.
Graph Convolutional Networks for Temporal Action Localization · GCN in Videos · 0.95
-
Most methods [ref] divide the task into two stages: temporal proposal generation and classification/regression of proposals.
Untrimmed Video Classification for Activity Detection: submission to ActivityNet Challenge · Video Representation · 0.91
-
Two-stream networks [ref] use 2D CNNs to extract frame features from RGB and optical flow sequences.
Towards Good Practices for Very Deep Two-Stream ConvNets · Video Representation · 0.97
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.