RECORD
Pose-aware Multi-level Feature Network for Human Object Interaction Detection
Computer Vision and Pattern Recognition · Submitted 18 Sep 2019
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Extensive experimental evaluations on public benchmarks show that our model outperforms prior methods by a considerable margin, demonstrating its efficacy in handling complex scenes.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Reasoning human object interactions is a core problem in human-centric scene understanding and detecting such relations poses a unique challenge to vision systems due to large variations in human-object configurations, multiple co-occurring relation instances and subtle visual difference between relation categories. To address those challenges, we propose a multi-level relation detection strategy that utilizes human pose cues to capture global spatial configurations of relations and as an attention mechanism to dynamically zoom into relevant regions at human part level. Specifically, we develop a multi-branch deep network to learn a pose-augmented relation representation at three semantic levels, incorporating interaction context, object features and detailed semantic part cues. As a result, our approach is capable of generating robust predictions on fine-grained human object interactions with interpretable outputs. Extensive experimental evaluations on public benchmarks show that our model outperforms prior methods by a considerable margin, demonstrating its efficacy in handling complex scenes.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Reasoning human object interactions is a core problem in human-centric scene understanding and detecting such relations poses a unique challenge to vision systems due to large variations in human-object configurations, multiple co-occurring relation instances and subtle visual difference between relation categories. | background 1.00 | 0.05 | 0.03 | asserted 1.00 |
| s1 | To address those challenges, we propose a multi-level relation detection strategy that utilizes human pose cues to capture global spatial configurations of relations and as an attention mechanism to dynamically zoom into relevant regions at human part level. | method 0.53 | 0.82 | 0.06 | asserted 0.96 |
| s2 | Specifically, we develop a multi-branch deep network to learn a pose-augmented relation representation at three semantic levels, incorporating interaction context, object features and detailed semantic part cues. | method 0.97 | 0.95 | 0.05 | asserted 0.99 |
| s3 | As a result, our approach is capable of generating robust predictions on fine-grained human object interactions with interpretable outputs. | result 0.51 | 0.85 | 0.38 | asserted 0.83 |
| s4 | Extensive experimental evaluations on public benchmarks show that our model outperforms prior methods by a considerable margin, demonstrating its efficacy in handling complex scenes. | result 0.99 | 0.96 | 0.57 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Reasoning human object interactions is a core problem in human-centric scene understanding and detecting such relations poses a unique challenge to vision systems due to large variations in human-object configurations, multiple co-occurring relation instances and subtle visual difference between relation categories. To address those challenges, we propose a multi-level relation detection strategy that utilizes human pose cues to capture global spatial configurations of relations and as an attention mechanism to dynamically zoom into relevant regions at human part level. Specifically, we develop a multi-branch deep network to learn a pose-augmented relation representation at three semantic levels, incorporating interaction context, object features and detailed semantic part cues. As a result, our approach is capable of generating robust predictions on fine-grained human object interactions with interpretable outputs. Extensive experimental evaluations on public benchmarks show that our model outperforms prior methods by a considerable margin, demonstrating its efficacy in handling complex scenes.
Human review
How the literature uses it
1 arXiv paper cite this in 4 sentences
Papers citing it · 1 sentence without a confident label
Uses its method or tool 1 paper
-
Our 2D-RN has a similar multi-stream structure, object detection and backbone following HO-RCNN [ref] , iCAN [ref] , Interactiveness [ref] and PMFNet [ref] .
Detailed 2D-3D Joint Representation for Human-Object Interaction · Results and Comparisons · 0.95
Compares with it 1 paper
-
We demonstrate our quantitative results in Tab. [ref], compared with state-of-the-art methods [ref] .
Detailed 2D-3D Joint Representation for Human-Object Interaction · Results and Comparisons · 1.00
Cites it as background 1 paper
-
Meanwhile, lots of deep learning based methods [ref] have been proposed.
Detailed 2D-3D Joint Representation for Human-Object Interaction · Related Work · 0.98
What it cites · 9 sentences without a confident label
Uses the method or tool of 1 paper
-
Spatial Configuration Map (SCM) As in [ref] , we augment human and object binary masks with an additional human pose configuration map, which provides more detailed spatial information on human parts.
Transferable Interactiveness Knowledge for Human-Object Interaction Detection · Ablation Study · 0.98
Uses data or values from 1 paper
-
Datasets We evaluate our method on two HOI benchmarks: V-COCO [ref] and HICO-DET [ref] .
Visual Semantic Role Labeling · Experimental Setting · 0.86
Compares with 2 papers
-
As shown in Tab. [ref], our baseline still performs well and surpasses most existing works except [ref] .
Transferable Interactiveness Knowledge for Human-Object Interaction Detection · Quantitative Results · 1.00
-
As shown in Tab. [ref], our baseline method achieves 48.6 mAP, outperforming all the existing approaches [ref] .
Visual Semantic Role Labeling · Quantitative Results · 1.00
Cites as background 2 papers
-
Li et al. [ref] explore interactiveness prior existing in multiple datasets and combine human pose and spatial configuration to form pose configuration map.
Transferable Interactiveness Knowledge for Human-Object Interaction Detection · Related Work · 0.86
-
Sharma et al. [ref] apply attention models realized by LSTM into action recognition task to learn important parts of video frames.
Action Recognition using Visual Attention · Related Work · 0.80
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.