Pith. sign in

REVIEW 5 major objections 4 minor 83 references

Geometric Visual Fusion Graph Neural Networks for Multi-Person Human-Object Interaction Recognition in Videos

T0 review · 5 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A bottom-up graph network that fuses geometric and visual cues per entity before modeling interactions reports state-of-the-art results on four video HOI benchmarks, with a 13.0-point F1@10 gain over the prior best on MPHOI-72.

desk verdict A solid, honest empirical paper: new architecture and new dataset, but the strongest claims on MPHOI-120 rest on single-annotator labels and an undocumented zero-shot label mapping. read the letter →

arxiv 2506.03440 v2 pith:HSDMF7K5 submitted 2025-06-03 cs.CV

Tanqiu Qiao , Ruochen Li , Frederick W. B. Li , Yoshiki Kubotani , Shigeo Morishima , Hubert P. H. Shum This is my paper
classification cs.CV
keywords human-objectinteractionrecognitionvideounderstandinggraphneuralnetworkgeometric-visualfusionmulti-personactionsegmentationMPHOI-120zero-shottransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Video human-object interaction (HOI) recognition must combine what people and objects look like with how they are arranged in space, but fusing those two kinds of cues usually washes one of them out. The paper proposes GeoVis-GNN, a bottom-up graph network that first builds a strong entity-specific representation for each person and object by adapting geometric features with graph attention and fusing them with visual features through channel attention, and only then models interactions in an interdependent entity graph. The paper argues this order matters: preserving each entity's own characteristics before reasoning about relationships yields better segmentation and labeling of interactions than top-down or mixed-fusion alternatives. To test the idea beyond two fully engaged participants, the authors introduce MPHOI-120, a dataset of three-person activities with concurrent actions and idle participants. On four benchmarks the method reports state-of-the-art F1 scores, including a 13.0-point F1@10 gain over the prior best on the two-person MPHOI-72 benchmark.

What carries the argument

The load-bearing machinery is dual-attention feature fusion followed by an interdependent entity graph. First, a graph attention layer over skeleton keypoints and object bounding-box diagonals produces structured geometric embeddings, and then a squeeze-and-excitation channel attention module concatenates the geometric and visual features of all entities, reweights their channels, and splits them back into enriched per-entity representations. Those representations enter an interdependent entity graph that models, for each target entity, the aggregated features of its neighbors with dot-product attention weights, so both explicit spatial interactions and implicit contextual dependencies are available to the recognizer. A Gumbel-Softmax module and a bidirectional GRU then produce the final segmentation and label predictions.

What would settle it

Re-annotate a random subset of MPHOI-120 frames with several independent annotators and compute inter-annotator agreement on sub-activity boundaries and labels; low agreement would show the reported F1 scores are not stable. Also train on MPHOI-72 and test zero-shot on MPHOI-120; if the transfer gain disappears or becomes negative, the claimed generalization is specific to the training dataset.

Watch

Extended reading notes

Core claim

The central claim is that visual and geometric features should be fused per entity and before interaction modeling. GeoVis-GNN first learns structured geometric embeddings with a graph attention network that weights neighboring keypoints, then uses channel attention to reweight the concatenated geometric-visual features of all entities, producing enriched entity-specific representations. An interdependent entity graph then aggregates neighbor features toward each target entity and applies dot-product attention, capturing both explicit interactions and implicit contextual dependencies, before a Gumbel-Softmax and BiGRU segment the timeline and classify sub-activities and object affordances. The paper reports that this design beats prior methods on two-person, single-person, bimanual, and concurrent partial interaction benchmarks, including a 13.0-point F1@10 gain over the prior state of the art on MPHOI-72.

Load-bearing premise

The new MPHOI-120 dataset's frame-by-frame labels come from a single annotator and the cross-dataset test assumes a clean mapping between its 17 sub-activity labels and MPHOI-72's 13 labels, so noisy or unmappable annotations would undermine the reported state-of-the-art and zero-shot gains.

Editorial extensions

If this is right

  • The per-entity fusion-first ordering should transfer to other HOI architectures: any model that currently mixes geometric and visual features in a single graph is a candidate for the same two-stage design.
  • MPHOI-120 gives the community a benchmark where participants are not all active at once, enabling evaluation on idle participants, concurrent actions, and mutual occlusions rather than only fully engaged pairs.
  • The 13.0-point F1@10 improvement on MPHOI-72 indicates that the fusion redesign, rather than additional data, drives most of the gain in two-person settings.
  • The zero-shot result, training on MPHOI-120 and testing on MPHOI-72, suggests the learned entity representations are relatively dataset-agnostic and could be applied where dense per-frame labels are unavailable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test image-based HOI detection, but the same fusion-first ordering is a natural candidate for static scenes where transformer-based fusion is common and can suffer from the same dilution of geometric cues.
  • Because the dataset was labeled by one annotator, an independent multi-annotator re-labeling of a subset would be the decisive check on whether the MPHOI-120 numbers reflect the method or the annotations.
  • The attention visualizations imply the entity graph could be used as an interpretability tool, showing which object the model attends to during an interaction rather than treating recognition as a black box.
  • If the zero-shot transfer holds on more varied target datasets, the approach could bootstrap HOI recognition in new environments without per-frame annotation; the paper leaves that demonstration to future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes GeoVis-GNN, a bottom-up graph neural network for video-based multi-person human-object interaction (HOI) recognition. The architecture first builds entity-specific representations by fusing geometric features (skeleton joints, bounding boxes) with visual features through a graph-attention embedding followed by channel attention, then models explicit and implicit interdependencies in an interdependent entity graph with attention aggregation, and finally uses a BiGRU for segmentation and label recognition. The authors also introduce MPHOI-120, a dataset of 120 videos of three-person interactions with partial engagement and concurrent actions, annotated frame-by-frame with 17 sub-activities. Experiments compare with ASSIGN, 2G-GCN, and CATS on MPHOI-120, MPHOI-72, CAD-120, and Bimanual Actions, reporting state-of-the-art F1@k results, ablations, zero-shot transfer, and qualitative attention and segmentation analyses.

Significance. If the reported results are reliable, GeoVis-GNN offers a coherent design principle, namely bottom-up entity representation before interaction modeling, and a concrete fusion mechanism that appears to give a large gain on MPHOI-72 (84.3 vs 71.3 F1@10). The paper is generally well structured, provides equations for the method, releases code and dataset, evaluates on four benchmarks, and includes ablations and attention visualizations. The main caveats are that the two largest claimed gains are on the new dataset, whose annotations are not yet shown to be reliable, and on MPHOI-72, where the comparison is primarily against the authors' own prior baselines; in addition, several reported improvements are within one standard deviation. The dataset could be a useful community resource if its annotation reliability and label mapping are clarified.

major comments (5)
  1. [Section 3.1, Section 6, Table 2] The MPHOI-120 annotations are described in Section 3.1 as produced frame-by-frame by a single trained annotator, with no inter-annotator agreement or validation protocol reported. This matters because the main new-benchmark result (Table 2) is GeoVis-GNN at 65.1±5.2 versus CATS at 62.8±2.7 F1@10, a gap smaller than the winner's per-fold standard deviation, and because Section 6 itself acknowledges that frame-level annotations in multi-person settings are costly, inconsistent, and error-prone. Label noise at a few percent could invert the comparison. Please report inter-annotator agreement on a subset (e.g., Cohen's kappa or segment-boundary agreement), describe the annotation protocol, and provide a noise-robustness analysis or a re-annotation sample.
  2. [Section 5.5, Table 7] The zero-shot experiment trains on MPHOI-120 (17 sub-activity labels) and tests on MPHOI-72 (13 sub-activity labels), but no mapping between the two label sets is provided. Without knowing whether the mapping is deterministic, bijective, or hand-curated, the reported 3.6-point F1@10 gain over CATS is not interpretable as generalization of the model; it could reflect the mapping or evaluation procedure. Please specify the mapping in full, state whether it was fixed before experiments, and ideally evaluate multiple plausible mappings to show that the conclusion is not an artifact of label alignment.
  3. [Section 5.2, Eq. (5), Table 5] For Bimanual Actions the implementation sets C2=32, C3=64 and S_u^t=0, which disables the neighbor-feature term in Eq. (5) and therefore removes the interdependent entity graph's aggregation from the model. As a result, Table 5 does not evaluate the full GeoVis-GNN, and the reported gain over 2G-GCN (85.8±2.2 vs 85.0±2.2 F1@10) is not a clean comparison of the proposed architecture. Please report the ablation with Eq. (5) enabled for this dataset, or explicitly present the Bimanual configuration as a reduced variant; otherwise the cross-scenario SOTA claim is unsupported.
  4. [Section 5.3.3, Table 4] The CAD-120 results in Table 4 show differences of at most 0.3–0.4 F1 points between GeoVis-GNN and the strongest baselines (e.g., 89.9±2.0 vs 89.6±2.1 for sub-activity F1@10), all within one standard deviation. The text nevertheless states that GeoVis-GNN beats previous networks for both labels and achieves the highest F1 scores in every configuration. Please report significance tests or confidence intervals and temper the SOTA statement accordingly; as written, the claim is overstated on this benchmark.
  5. [Section 5.2, Section 4.1.2] Several hyperparameters, including C1=128, C2=256, C3=512, λ=0.5, and the learning rate, are described as set based on empirical experimentation or preliminary validation experiments. If these choices were made using the evaluation folds of the same datasets, the comparisons to published baselines are not apples-to-apples, since the baselines did not receive the same tuning. Please clarify the validation protocol (e.g., nested cross-validation or a fixed validation split) and, if possible, report sensitivity to λ and the embedding sizes, or use fixed values across datasets.
minor comments (4)
  1. [Eq. (2)] The denominator in Eq. (2) is written as a sum over k3∈K∪{k3}, which is not well-formed; the first term in Eq. (1) also appears to double-count k1 if the sum over k2 includes k1. Please clean up the indexing.
  2. [Table 6] The 'Difference' column reports percentage gaps such as '+24.8%' without stating whether these are absolute F1@10 differences or relative improvements; please clarify the definition.
  3. [Figures 5–8 and 12] The qualitative analyses are based on a small number of selected videos; please add confusion matrices or other quantitative error analyses to support the claimed error patterns (e.g., over-segmentation, fast-action confusion).
  4. [Table 1] MPHOI-120 has only 7 subjects and 4 main activities; please report the per-activity subject counts and sub-activity label distributions so readers can assess the small-subject variability and the difficulty of the benchmark.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: GeoVis-GNN's contributions are empirical architecture and dataset claims; the reported gains are comparisons against independent published baselines, and no prediction reduces by construction to a fitted input or self-citation.

full rationale

I walked the derivation chain and found no step in which a reported 'prediction' or 'first-principles result' is equivalent to its own inputs by construction. The paper's central claim is an empirical SOTA claim, and its method is defined by standard GAT, channel-attention, graph-aggregation, and BiGRU components (Sec. 4.1-4.2); none of the equations (1)-(7) reconstruct the target F1 metric from fitted parameters. The lambda=0.5 threshold in Eq. (5) and the C1/C2/C3 dimensions in Sec. 5.2 are ordinary hyperparameter choices, not fitted quantities renamed as predictions; the paper states lambda is stable under small variations. The strongest baselines, 2G-GCN and CATS, are the authors' prior publications, but they are published, externally reproducible systems used as comparison points, and per the review rules such independent support does not raise the circularity score. I also flag the caveats that could affect trustworthiness but are not circularity: Section 3.1 states MPHOI-120 was annotated 'frame-by-frame by a single trained annotator' with no inter-annotator agreement, and Section 6 itself concedes that frame-level annotations 'become costly and inconsistent'; the Section 5.5 zero-shot study trains on 17 MPHOI-120 sub-activities and tests on 13 MPHOI-72 labels without specifying the label mapping; and Section 5.2 sets 'C2 = 32, C3 = 64 and [Su t = 0]' for Bimanual Actions, an unexplained variant that disables the interdependent graph. These are evaluation-validity and reporting gaps, not circular reductions: none of them makes the reported F1 values equal to the inputs by construction. The ablations in Tables 8-9 are internal architecture comparisons, not self-citations. I therefore conclude no material circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim depends on a few domain assumptions about feature transferability, pose/box estimation quality, and label compatibility across datasets, plus several hyperparameters (lambda, channel sizes, learning rate) that are tuned on the target datasets themselves. No new physical entities or theoretical constructs are introduced.

free parameters (4)
  • lambda (contextual fusion threshold) = 0.5
    Eq. 5; fixed after preliminary validation experiments on MPHOI-120, so the final numbers on the same dataset reflect this tuning.
  • Channel embedding dimensions C1, C2, C3 = C1=128, C2=256, C3=512; Bimanual: C2=32, C3=64
    Section 5.2: chosen based on empirical experimentation; per-dataset modification.
  • Learning rate = 0.0001
    Section 5.2, chosen for all datasets via AdamW.
  • Bimanual-specific disabling of Eq. 5 (S_u^t=0) = 0
    Section 5.2: 'we set ... and [S_u^t = 0]' for Bimanual Actions, a per-dataset architectural change.
assumptions (4)
  • domain assumption Pre-extracted visual features from Faster R-CNN trained on Visual Genome are suitable for all target HOI datasets.
    Section 4.1.1: visual features are extracted with a pretrained Faster R-CNN and not fine-tuned; downstream performance depends on transferability.
  • standard math Gumbel-Softmax with a fixed temperature yields coherent segmentation boundaries.
    Section 4.2: used as a standard relaxation; no temperature analysis is given.
  • domain assumption Skeleton and object bounding box features from Azure Kinect Body Tracking are accurate and informative for HOI recognition.
    Sections 3.1 and 4.1.1: geometric features are taken as reliable inputs; no error analysis of the pose/box estimation is provided.
  • domain assumption The label sets of MPHOI-120 (17 sub-activities) and MPHOI-72 (13 sub-activities) are compatible for the zero-shot transfer experiment.
    Section 5.5: the paper does not describe how labels are mapped or filtered, yet reports cross-dataset results.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Geometric Visual Fusion Graph Neural Networks for Multi-Person Human-Object Interaction Recognition in Videos." pith.science (2026). https://pith.science/paper/HSDMF7K5

@misc{pith2026250603440,
  author       = {Pith},
  title        = {Pith review of: Geometric Visual Fusion Graph Neural Networks for Multi-Person Human-Object Interaction Recognition in Videos},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HSDMF7K5}},
  note         = {Machine review of arXiv:2506.03440}
}
read the original abstract

Human-Object Interaction (HOI) recognition in videos requires understanding both visual patterns and geometric relationships as they evolve over time. Visual and geometric features offer complementary strengths. Visual features capture appearance context, while geometric features provide structural patterns. Effectively fusing these multimodal features without compromising their unique characteristics remains challenging. We observe that establishing robust, entity-specific representations before modeling interactions helps preserve the strengths of each modality. Therefore, we hypothesize that a bottom-up approach is crucial for effective multimodal fusion. Following this insight, we propose the Geometric Visual Fusion Graph Neural Network (GeoVis-GNN), which uses dual-attention feature fusion combined with interdependent entity graph learning. It progressively builds from entity-specific representations toward high-level interaction understanding. To advance HOI recognition to real-world scenarios, we introduce the Concurrent Partial Interaction Dataset (MPHOI-120). It captures dynamic multi-person interactions involving concurrent actions and partial engagement. This dataset helps address challenges like complex human-object dynamics and mutual occlusions. Extensive experiments demonstrate the effectiveness of our method across various HOI scenarios. These scenarios include two-person interactions, single-person activities, bimanual manipulations, and complex concurrent partial interactions. Our method achieves state-of-the-art performance.

Figures

Figures reproduced from arXiv: 2506.03440 by the authors.

Figure 1
Figure 1. Two examples (Teaching and Signing) of our collected concurrent partial interac￾tion datasets. Geometric features such as skeletons and bounding boxes are annotated. retained even when modeling complex interactions. Video-based HOI datasets have primarily focused on single-person activ￾ities or limited two-person interactions, leaving a notable gap in capturing the complexity of real-world multi-person scenarios. Ex… view at source ↗
Figure 2
Figure 2. Sample video screenshots from our new MPHOI-120 dataset, displaying concurrent partial interactions along the timelines of four multi-person HOI activities in daily life. where multiple interactions occur simultaneously, while “partial interaction" highlights moments when not all individuals are actively engaging - some may remain idle or waiting during certain moments of the activity. For example, in the Signing ac… view at source ↗
Figure 3
Figure 3. Overview of our bottom-up framework GeoVis-GNN. We first design a dual￾attention fusion for entity feature optimization, which embeds and fuses visual and geometric features in a graph attention-based mechanism and channel attention module, respectively. The enriched entity-specific representations are then inputted into the interdependent entity graph to further model explicit interactions and implicit interdepende… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: In the interdependent entity graph, we first model neighbor features [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]
Figure 5
Figure 5. Figure 5: Visualization of segmentation on MPHOI-120 for Signing activity. Red dashed boxes highlight major segmentation errors [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: Visualization of segmentation on MPHOI-72 for [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 7
Figure 7. Figure 7: Visualization of segmentation on CAD-120 for [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]
Figure 8
Figure 8. Figure 8: Visualization of segmentation on Bimanual Actions for [PITH_FULL_IMAGE:figures/full_fig_p023_8.png]
Figure 9
Figure 9. Figure 9: Example of sub-activity segmentation error on the CAD-120 dataset for [PITH_FULL_IMAGE:figures/full_fig_p024_9.png]
Figure 10
Figure 10. Figure 10: Example of sub-activity segmentation error on the MPHOI-72 dataset for [PITH_FULL_IMAGE:figures/full_fig_p024_10.png]
Figure 11
Figure 11. Figure 11: Different designs to combine geometric and visual features in channel attention [PITH_FULL_IMAGE:figures/full_fig_p026_11.png]
Figure 12
Figure 12. Figure 12: Visualization of HOI attention maps for GeoVis-GNN and 2G-GCN during a [PITH_FULL_IMAGE:figures/full_fig_p028_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

83 extracted references · 76 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in ":" * " " * FUNCTION f...

  2. [2]

    , author Wolf, C

    author Baradel, F. , author Wolf, C. , author Mille, J. , year 2017 . title Human action recognition: Pose-based attention draws focus to hands , in: booktitle Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) , pp. pages 604--613

    2017

  3. [3]

    , author Wolf, C

    author Baradel, F. , author Wolf, C. , author Mille, J. , year 2018 a. title Human activity recognition with pose-driven attention to rgb , in: booktitle British Machine Vision Conference (BMVC) , pp. pages 1--14

    2018

  4. [4]

    , author Wolf, C

    author Baradel, F. , author Wolf, C. , author Mille, J. , author Taylor, G.W. , year 2018 b. title Glimpse clouds: Human activity recognition from unstructured feature points , in: booktitle Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pp. pages 469--478

    work page 2018

  5. [5]

    , author Xie, X

    author Bhatnagar, B.L. , author Xie, X. , author Petrov, I.A. , author Sminchisescu, C. , author Theobalt, C. , author Pons-Moll, G. , year 2022 . title Behave: Dataset and method for tracking human object interactions , in: booktitle Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pp. pages 15935--15946

    work page 2022

  6. [6]

    , author Amamra, A

    author Boulahia, S.Y. , author Amamra, A. , author Madi, M.R. , author Daikh, S. , year 2021 . title Early, intermediate and late fusion strategies for robust deep learning-based multimodal action recognition . journal Machine Vision and Applications volume 32 , pages 121

    work page 2021

  7. [7]

    , author Alon, U

    author Brody, S. , author Alon, U. , author Yahav, E. , year 2021 . title How attentive are graph attention networks? journal arXiv preprint arXiv:2105.14491

    arXiv 2021

  8. [8]

    , author Liu, Y

    author Bruce, X. , author Liu, Y. , author Chan, K.C. , year 2021 . title Multimodal fusion via teacher-student network for indoor action recognition , in: booktitle Proceedings of the AAAI Conference on Artificial Intelligence , pp. pages 3199--3207

    work page 2021

Show all 83 references
  1. [9]

    , author Hidalgo, G

    author Cao, Z. , author Hidalgo, G. , author Simon, T. , author Wei, S.E. , author Sheikh, Y. , year 2018 . title Openpose: Realtime multi-person 2d pose estimation using part affinity fields . journal arXiv e-prints , pages arXiv--1812

    2018

  2. [10]

    , author Cheng, J

    author Cheng, Q. , author Cheng, J. , author Liu, Z. , author Ren, Z. , author Liu, J. , year 2024 a. title A dense-sparse complementary network for human action recognition based on rgb and skeleton modalities . journal Expert Systems with Applications volume 244 , pages 123061

    2024

  3. [11]

    , author Duan, H

    author Cheng, Y. , author Duan, H. , author Wang, C. , author Chen, Z. , year 2024 b. title Parallel disentangling network for human--object interaction detection . journal Pattern Recognition volume 146 , pages 110021

    2024

  4. [12]

    , author Gulcehre, C

    author Chung, J. , author Gulcehre, C. , author Cho, K. , author Bengio, Y. , year 2014 . title Empirical evaluation of gated recurrent neural networks on sequence modeling . journal arXiv preprint arXiv:1412.3555

    2014 arXiv

  5. [13]

    , author Losada-Guti \'e rrez, C

    author Cob-Parro, A.C. , author Losada-Guti \'e rrez, C. , author Marr \'o n-Romera, M. , author Gardel-Vicente, A. , author Bravo-Mu \ n oz, I. , year 2024 . title A new framework for deep learning video based human action recognition on the edge . journal Expert Systems with...

    2024

  6. [14]

    , author Doughty, H

    author Damen, D. , author Doughty, H. , author Farinella, G.M. , author Furnari, A. , author Ma, J. , author Kazakos, E. , author Moltisanti, D. , author Munro, J. , author Perrett, T. , author Price, W. , author Wray, M. , year 2021 . title Rescaling egocentric vision: Collec...

    2021

  7. [15]

    , author Sharma, S

    author Das, S. , author Sharma, S. , author Dai, R. , author Bremond, F. , author Thonnat, M. , year 2020 . title Vpn: Learning video-pose embedding for activities of daily living , in: booktitle European Conference on Computer Vision (ECCV) , pp. pages 72--90

    2020

  8. [16]

    , author W \"a chter, M

    author Dreher, C.R. , author W \"a chter, M. , author Asfour, T. , year 2020 . title Learning object-action relations from bimanual human demonstration using graph networks . journal IEEE Robotics and Automation Letters volume 5 , pages 187--194

    2020

  9. [17]

    , author Chen, M

    author Fan, M. , author Chen, M. , author Hu, C. , author Zhou, S. , year 2023 . title Occ\^ 2net: Robust image matching based on 3d occupancy estimation for occluded regions , in: booktitle Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pp. p...

    2023

  10. [18]

    , author Gall, J

    author Farha, Y.A. , author Gall, J. , year 2019 . title Ms-tcn: Multi-stage temporal convolutional network for action segmentation , in: booktitle Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pp. pages 3575--3584

    2019

  11. [19]

    , author Kuo, W.c

    author Fouhey, D.F. , author Kuo, W.c. , author Efros, A.A. , author Malik, J. , year 2018 . title From lifestyle vlogs to everyday interactions , in: booktitle Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pp. pages 4991--5000

    2018

  12. [20]

    , author Sun, C

    author Gao, J. , author Sun, C. , author Zhao, H. , author Shen, Y. , author Anguelov, D. , author Li, C. , author Schmid, C. , year 2020 . title Vectornet: Encoding hd maps and agent dynamics from vectorized representation , in: booktitle Proceedings of the IEEE/CVF Conferenc...

    2020

  13. [21]

    , author Ghosh, P

    author Hassan, M. , author Ghosh, P. , author Tesch, J. , author Tzionas, D. , author Black, M.J. , year 2021 . title Populating 3d scenes by learning human-scene interaction , in: booktitle Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVP...

    2021

  14. [22]

    , author Kampel, M

    author Heitzinger, T. , author Kampel, M. , year 2023 . title A fast unified system for 3d object detection and tracking , in: booktitle Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pp. pages 17044--17054

    2023

  15. [23]

    , author Shen, L

    author Hu, J. , author Shen, L. , author Sun, G. , year 2018 . title Squeeze-and-excitation networks , in: booktitle Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pp. pages 7132--7141

    2018

  16. [24]

    , author Xiao, J

    author Hu, Z. , author Xiao, J. , author Li, L. , author Liu, C. , author Ji, G. , year 2024 . title Human-centric multimodal fusion network for robust action recognition . journal Expert Systems with Applications volume 239 , pages 122314

    2024

  17. [25]

    , author Gu, S

    author Jang, E. , author Gu, S. , author Poole, B. , year 2016 . title Categorical reparameterization with gumbel-softmax . journal arXiv preprint arXiv:1611.01144

    2016 arXiv

  18. [26]

    , author Koppula, H

    author Jiang, Y. , author Koppula, H. , author Saxena, A. , year 2013 . title Hallucinated humans as the hidden context for labeling 3d scenes , in: booktitle Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pp. pages 2993--3000

    2013

  19. [27]

    , author Jung, D

    author Kim, S. , author Jung, D. , author Cho, M. , year 2023 . title Relational context learning for human-object interaction detection , in: booktitle Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. pages 2925--2934

    2023

  20. [28]

    , author Gupta, R

    author Koppula, H.S. , author Gupta, R. , author Saxena, A. , year 2013 . title Learning human activities and object affordances from rgb-d videos . journal The International Journal of Robotics Research volume 32 , pages 951--970

    2013

  21. [29]

    , author Saxena, A

    author Koppula, H.S. , author Saxena, A. , year 2016 . title Anticipating human activities using object affordances for reactive robotic response . journal IEEE Transactions on Pattern Analysis and Machine Intelligence volume 38 , pages 14--29

    2016

  22. [30]

    , author Meixner, A

    author Krebs, F. , author Meixner, A. , author Patzer, I. , author Asfour, T. , year 2021 . title The kit bimanual manipulation dataset , in: booktitle IEEE/RAS International Conference on Humanoid Robots (Humanoids) , pp. pages 0--0

    2021

  23. [31]

    , author Nguyen, T.D

    author Kresovi \'c , M. , author Nguyen, T.D. , year 2021 . title Bottom-up approaches for multi-person pose estimation and it's applications: A brief review . journal arXiv preprint arXiv:2112.11834

    2021 arXiv

  24. [32]

    , author Zhu, Y

    author Krishna, R. , author Zhu, Y. , author Groth, O. , author Johnson, J. , author Hata, K. , author Kravitz, J. , author Chen, S. , author Kalantidis, Y. , author Li, L.J. , author Shamma, D.A. , et al., year 2017 . title Visual genome: Connecting language and vision using ...

    2017

  25. [33]

    , author Sahoo, D

    author Le, H. , author Sahoo, D. , author Chen, N.F. , author Hoi, S.C. , year 2020 . title Bist: Bi-directional spatio-temporal reasoning for video-grounded dialogues . journal arXiv preprint arXiv:2010.10095

    2020 arXiv

  26. [34]

    , author Flynn, M.D

    author Lea, C. , author Flynn, M.D. , author Vidal, R. , author Reiter, A. , author Hager, G.D. , year 2017 . title Temporal convolutional networks for action segmentation and detection , in: booktitle Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog...

    2017

  27. [35]

    , author Shum, H.P.H

    author Li, L. , author Shum, H.P.H. , author Breckon, T.P. , year 2024 a. title RAPiD-Seg : Range-Aware Pointwise Distance Distribution Networks for 3D LiDAR Segmentation , in: booktitle European Conference on Computer Vision (ECCV) , publisher Springer

    2024

  28. [36]

    , author Wei, J

    author Li, L. , author Wei, J. , author Wang, W. , author Yang, Y. , year 2024 b. title Neural-logic human-object interaction detection . journal Advances in Neural Information Processing Systems volume 36

    2024

  29. [37]

    , author Katsigiannis, S

    author Li, R. , author Katsigiannis, S. , author Kim, T.K. , author Shum, H.P. , year 2025 a. title Bp-sgcn: Behavioral pseudo-label informed sparse graph convolution network for pedestrian and heterogeneous trajectory prediction . journal IEEE Transactions on Neural Networks ...

    2025

  30. [38]

    , author Katsigiannis, S

    author Li, R. , author Katsigiannis, S. , author Shum, H.P. , year 2022 . title Multiclass-sgcn: Sparse graph-based trajectory prediction with agent class embedding , in: booktitle IEEE International Conference on Image Processing (ICIP) , organization IEEE . pp. pages 2346--2350

    2022

  31. [39]

    , author Qiao, T

    author Li, R. , author Qiao, T. , author Katsigiannis, S. , author Zhu, Z. , author Shum, H.P. , year 2025 b. title Unified spatial-temporal edge-enhanced graph networks for pedestrian trajectory prediction . journal IEEE Transactions on Circuits and Systems for Video Technology

    2025

  32. [40]

    , author Du, Y

    author Li, S. , author Du, Y. , author Torralba, A. , author Sivic, J. , author Russell, B. , year 2021 . title Weakly supervised human-object interaction detection in video via contrastive spatiotemporal regions , in: booktitle Proceedings of the IEEE/CVF International Confer...

    2021

  33. [41]

    , author Yang, B

    author Liang, M. , author Yang, B. , author Hu, R. , author Chen, Y. , author Liao, R. , author Feng, S. , author Urtasun, R. , year 2020 . title Learning lane graph representations for motion forecasting , in: booktitle European Conference on Computer Vision (ECCV) , organiza...

    2020

  34. [42]

    , author Chen, Q

    author Lin, M. , author Chen, Q. , author Yan, S. , year 2013 . title Network in network . journal arXiv preprint arXiv:1312.4400

    2013 arXiv

  35. [43]

    , author Liu, Y

    author Liu, Y. , author Liu, Y. , author Jiang, C. , author Lyu, K. , author Wan, W. , author Shen, H. , author Liang, B. , author Fu, Z. , author Wang, H. , author Yi, L. , year 2022 . title Hoi4d: A 4d egocentric dataset for category-level human-object interaction , in: book...

    2022

  36. [44]

    , author Wang, Y

    author Ma, S. , author Wang, Y. , author Wang, S. , author Wei, Y. , year 2023 . title Fgahoi: Fine-grained anchors for human-object interaction detection . journal IEEE Transactions on Pattern Analysis and Machine Intelligence

    2023

  37. [45]

    , author Faez, K

    author Maraghi, V.O. , author Faez, K. , year 2019 . title Zero-shot learning on human-object interaction recognition in video , in: booktitle Iranian conference on signal processing and intelligent systems (ICSPIS) , pp. pages 1--7

    2019

  38. [46]

    , author Xiao, T

    author Materzynska, J. , author Xiao, T. , author Herzig, R. , author Xu, H. , author Wang, X. , author Darrell, T. , year 2020 . title Something-else: Compositional action recognition with spatial-temporal interaction networks , in: booktitle Proceedings of the IEEE/CVF Confe...

    2020

  39. [47]

    title Quickstart: Set up azure kinect body tracking

    author Microsoft , year 2022 . title Quickstart: Set up azure kinect body tracking . https://docs.microsoft.com/en-us/azure/kinect-dk/body-sdk-setup

    2022

  40. [48]

    , author Le, V

    author Morais, R. , author Le, V. , author Venkatesh, S. , author Tran, T. , year 2021 . title Learning asynchronous and sparse human-object interaction in videos , in: booktitle Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pp. pag...

    2021

  41. [49]

    , author Dai, A

    author Nie, Y. , author Dai, A. , author Han, X. , author Nie ner, M. , year 2022 . title Pose2room: understanding 3d scenes from human activities , in: booktitle European Conference on Computer Vision (ECCV) , organization Springer . pp. pages 425--443

    2022

  42. [50]

    , author Park, J.W

    author Park, J. , author Park, J.W. , author Lee, J.S. , year 2023 . title Viplo: Vision transformer based pose-conditioned self-loop graph for human-object interaction detection , in: booktitle Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition ...

    2023

  43. [51]

    , author Li, R

    author Qiao, T. , author Li, R. , author Li, F.W.B. , author Shum, H.P. , year 2024 . title From category to scenery: An end-to-end framework for multi-person human-object interaction recognition in videos , in: booktitle International Conference of Pattern Recognition

    2024

  44. [52]

    , author Men, Q

    author Qiao, T. , author Men, Q. , author Li, F.W.B. , author Kubotani, Y. , author Morishima, S. , author Shum, H.P.H. , year 2022 . title Geometric features informed multi-person human-object interaction recognition in videos , in: booktitle European Conference on Computer V...

    2022

  45. [53]

    , author Yang, W

    author Ren, H. , author Yang, W. , author Zhang, T. , author Zhang, Y. , year 2023 . title Proposal-based multiple instance learning for weakly-supervised temporal action localization , in: booktitle Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni...

    2023

  46. [54]

    , author He, K

    author Ren, S. , author He, K. , author Girshick, R. , author Sun, J. , year 2016 . title Faster r-cnn: towards real-time object detection with region proposal networks . journal IEEE Transactions on Pattern Analysis and Machine Intelligence volume 39 , pages 1137--1149

    2016

  47. [55]

    , author Mittal, G

    author Rizve, M.N. , author Mittal, G. , author Yu, Y. , author Hall, M. , author Sajeev, S. , author Shah, M. , author Chen, M. , year 2023 . title Pivotal: Prior-driven supervision for weakly-supervised temporal action localization , in: booktitle Proceedings of the IEEE/CVF...

    2023

  48. [56]

    , author Hicsonmez, S

    author Samet, N. , author Hicsonmez, S. , author Akbas, E. , year 2020 . title Houghnet: Integrating near and long-range evidence for bottom-up object detection , in: booktitle European Conference on Computer Vision (ECCV) , organization Springer . pp. pages 406--423

    2020

  49. [57]

    , author Saxena, A

    author Sener, O. , author Saxena, A. , year 2015 . title rcrf: Recursive belief estimation over crfs in rgb-d activity videos. , in: booktitle Robotics: Science and systems

    2015

  50. [58]

    , author Yahya, B.N

    author Setiawan, F. , author Yahya, B.N. , author Chun, S.J. , author Lee, S.L. , year 2022 . title Sequential inter-hop graph convolution neural network (sihgcn) for skeleton-based human action recognition . journal Expert Systems with Applications volume 195 , pages 116566

    2022

  51. [59]

    , author Gao, X

    author Shu, T. , author Gao, X. , author Ryoo, M.S. , author Zhu, S.C. , year 2017 . title Learning social affordance grammar from videos: Transferring human interactions to human-robot interactions , in: booktitle 2017 IEEE International Conference on Robotics and Automation ...

    2017

  52. [60]

    , author Ryoo, M.S

    author Shu, T. , author Ryoo, M.S. , author Zhu, S.C. , year 2016 . title Learning social affordance for human-robot interaction . journal arXiv preprint arXiv:1604.03692

    2016 arXiv

  53. [61]

    , author Lim, K.M

    author Tan, K.S. , author Lim, K.M. , author Lee, C.P. , author Kwek, L.C. , year 2022 . title Bidirectional long short-term memory with temporal dense sampling for human action recognition . journal Expert Systems with Applications volume 210 , pages 118484

    2022

  54. [62]

    , author Chatterjee, A

    author Tripathi, S. , author Chatterjee, A. , author Passy, J.C. , author Yi, H. , author Tzionas, D. , author Black, M.J. , year 2023 . title Deco: Dense estimation of 3d human-scene contact in the wild , in: booktitle Proceedings of the IEEE/CVF International Conference on C...

    2023

  55. [63]

    , author Sun, W

    author Tu, D. , author Sun, W. , author Min, X. , author Zhai, G. , author Shen, W. , year 2022 . title Video-based human-object interaction detection from tubelet tokens . journal Advances in Neural Information Processing Systems volume 35 , pages 23345--23357

    2022

  56. [64]

    , author Sun, W

    author Tu, D. , author Sun, W. , author Zhai, G. , author Shen, W. , year 2023 . title Agglomerative transformer for human-object interaction detection , in: booktitle Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pp. pages 21614--21624

    2023

  57. [65]

    , author Shazeer, N

    author Vaswani, A. , author Shazeer, N. , author Parmar, N. , author Uszkoreit, J. , author Jones, L. , author Gomez, A.N. , author Kaiser, L.u. , author Polosukhin, I. , year 2017 . title Attention is all you need , in: booktitle Advances in Neural Information Processing Syst...

    2017

  58. [66]

    , author Zhou, D

    author Wan, B. , author Zhou, D. , author Liu, Y. , author Li, R. , author He, X. , year 2019 . title Pose-aware multi-level feature network for human object interaction detection , in: booktitle Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , ...

    2019

  59. [67]

    , author Zhou, L

    author Wang, H. , author Zhou, L. , author Chen, Y. , author Tang, M. , author Wang, J. , year 2022 a. title Regularizing vector embedding in bottom-up human pose estimation , in: booktitle European Conference on Computer Vision , organization Springer . pp. pages 107--122

    2022

  60. [68]

    , author Zhu, G

    author Wang, N. , author Zhu, G. , author Li, H. , author Feng, M. , author Zhao, X. , author Ni, L. , author Shen, P. , author Mei, L. , author Zhang, L. , year 2023 . title Exploring spatio--temporal graph convolution for video-based human--object interaction recognition . j...

    2023

  61. [69]

    , author Zhu, G

    author Wang, N. , author Zhu, G. , author Zhang, L. , author Shen, P. , author Li, H. , author Hua, C. , year 2021 . title Spatio-temporal interaction graph parsing networks for human-object interaction recognition , in: booktitle Proceedings of the ACM International Conferenc...

    2021

  62. [70]

    , author Shen, J

    author Wang, W. , author Shen, J. , author Cheng, M.M. , author Shao, L. , year 2019 . title An iterative and cooperative top-down and bottom-up inference network for salient object detection , in: booktitle Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern...

    2019

  63. [71]

    , author Li, M

    author Wang, Y. , author Li, M. , author Cai, H. , author Chen, W.M. , author Han, S. , year 2022 b. title Lite pose: Efficient architecture design for 2d human pose estimation , in: booktitle Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , ...

    2022

  64. [72]

    , author Li, Y.L

    author Wu, X. , author Li, Y.L. , author Liu, X. , author Zhang, J. , author Wu, Y. , author Lu, C. , year 2022 . title Mining cross-person cues for body-part interactiveness learning in hoi detection , in: booktitle European Conference on Computer Vision , organization Spring...

    2022

  65. [73]

    , author Burschka, D

    author Xing, H. , author Burschka, D. , year 2022 . title Understanding spatio-temporal relations in human-object interaction using pyramid graph convolutional network , in: booktitle 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pp. pages 5...

    2022

  66. [74]

    , author Zhai, W

    author Yang, Y. , author Zhai, W. , author Luo, H. , author Cao, Y. , author Zha, Z.J. , year 2024 . title Lemon: Learning 3d human-object interaction relation from 2d images , in: booktitle Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp...

    2024

  67. [75]

    , author Wang, X

    author Ye, Q. , author Wang, X. , author Li, R. , author Zhang, Y. , year 2023 . title Human object interaction detection based on feature optimization and key human-object enhancement . journal Journal of Visual Communication and Image Representation volume 93 , pages 103824

    2023

  68. [76]

    , author Liu, H

    author You, Y. , author Liu, H. , author Wang, T. , author Li, W. , author Ding, R. , author Li, X. , year 2023 . title Co-evolution of pose and mesh for 3d human body estimation from video , in: booktitle Proceedings of the IEEE/CVF International Conference on Computer Vision...

    2023

  69. [77]

    , author Tanaka, M

    author Yu, Q. , author Tanaka, M. , author Fujiwara, K. , year 2024 . title Exploring vision transformers for 3d human motion-language models with motion patches , in: booktitle Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pp. page...

    2024

  70. [78]

    , author Nie, Q

    author Zhai, K. , author Nie, Q. , author Ouyang, B. , author Li, X. , author Yang, S. , year 2023 . title Hopfir: Hop-wise graphformer with intragroup joint refinement for 3d human pose estimation , in: booktitle Proceedings of the IEEE/CVF International Conference on Compute...

    2023

  71. [79]

    , author Pan, Y

    author Zhang, Y. , author Pan, Y. , author Yao, T. , author Huang, R. , author Mei, T. , author Chen, C.W. , year 2022 . title Exploring structure-aware transformer over interaction proposals for human-object interaction detection , in: booktitle Proceedings of the IEEE/CVF Co...

    2022

  72. [80]

    , author Wu, W

    author Zheng, C. , author Wu, W. , author Chen, C. , author Yang, T. , author Zhu, S. , author Shen, J. , author Kehtarnavaz, N. , author Shah, M. , year 2023 . title Deep learning-based human pose estimation: A survey . journal ACM Computing Surveys volume 56 , pages 1--37

    2023

  73. [81]

    , author Wang, Z

    author Zhou, J. , author Wang, Z. , author Meng, J. , author Liu, S. , author Zhang, J. , author Chen, S. , year 2022 . title Human interaction recognition with skeletal attention and shift graph convolution , in: booktitle International Joint Conference on Neural Networks (IJ...

    2022

  74. [82]

    , author Zhuo, J

    author Zhou, X. , author Zhuo, J. , author Krahenbuhl, P. , year 2019 . title Bottom-up object detection by grouping extreme and center points , in: booktitle Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pp. pages 850--859

    2019

  75. [83]

    , author Ho, E.S

    author Zhu, M. , author Ho, E.S. , author Chen, S. , author Yang, L. , author Shum, H.P. , year 2024 . title Geometric features enhanced human-object interaction detection . journal IEEE Transactions on Instrumentation and Measurement

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.