Pith. sign in

REVIEW 3 major objections 4 minor 68 references

Interacted Object Grounding in Spatio-Temporal Human-Object Interactions

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that 4D-QA grounds interacted objects in video far better than image-only baselines, using a new 1,098-class benchmark.

desk verdict GIO is a valuable new benchmark, but the 4D-QA superiority claim rests on an asymmetric baseline comparison. read the letter →

arxiv 2412.19542 v2 pith:TF2TZWNC submitted 2024-12-27 cs.CV cs.AIcs.LG

Xiaoyang Liu , Boran Wen , Xinpeng Liu , Zizheng Zhou , Hongwei Fan , Cewu Lu , Lizhuang Ma , Yulong Chen
show 1 more author
Yong-Lu Li
This is my paper · ORCID
classification cs.CVcs.AIcs.LG
keywords human-objectinteractionobjectgroundingopen-worldbenchmarkvideounderstanding4Dreconstructionquestion-answeringframeworkSAMspatio-temporalreasoning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces GIO, a large-scale third-view video benchmark for spatio-temporal human-object interaction with 1,098 interacted object classes and 290K box annotations, and defines an open-world interacted object grounding task: given a human tracklet and an interaction verb, locate the object the person is interacting with. The central claim is that current detectors and visual grounding models perform poorly on this task because they ignore temporal and spatial layout cues, and that a 4D question-answering framework (4D-QA) that selects among SAM-generated mask proposals using a reconstructed 4D human-object layout substantially outperforms them. On the GIO test set, 4D-QA reaches 23.38 mAP@0.5 and 29.71 mIoUw, against 17.53 and 20.41 for GroundingDINO, the strongest prior baseline. If this holds, it establishes that interaction-aware grounding benefits from explicit 4D spatial reasoning and provides a new benchmark to drive open-world HOI research.

What carries the argument

The central mechanism is the 4D question-answering framework (4D-QA), which turns bounding-box regression into mask selection. SAM generates fine-grained candidate object masks; a reconstructed 4D human-object layout—human SMPL meshes from PHALP aligned with ZoeDepth scene point clouds and encoded by BPS—provides the spatial prior; and a 2D then 3D transformer decoder, whose queries combine the human tracklet and an optional language interaction feature, scores each mask by cosine similarity. A GIoU distance term biases selection toward masks near the person, and depth-based clustering merges the selected masks into the final box.

What would settle it

Run the strongest baseline, GroundingDINO, with the ground-truth human tracklet and interaction verb supplied as input, and compare mAP@0.5 and mIoUw on the same GIO test split; if the gap with 4D-QA collapses, the paper's central claim of 4D superiority would be refuted.

Watch

Extended reading notes

Core claim

In the paper's own terms, the discovery is that interacted object grounding—finding the object a person is interacting with in a video, given the person's tracklet and the interaction verb—can be cast as a mask-selection question-answering problem rather than a box-regression problem, and that adding a reconstructed 4D human-object layout to the selection process yields large gains over image-based grounding. The 4D-QA pipeline feeds the video clip into SAM to produce up to 255 candidate masks per keyframe, extracts spatiotemporal context features with SlowFast, lifts the scene into 3D through human mesh recovery (PHALP), monocular depth estimation (ZoeDepth), and alignment, encodes the point clouds with BPS, and then grounds the interacted object with a two-stage 2D/3D transformer decoder whose queries combine human and verb features. The resulting mask scores, combined with a GIoU distance prior, select the interacted object. The paper argues that the 4D layout resolves occlusion and spatial ambiguities that image-only baselines cannot, and that the QA formulation makes the task robust to missing annotations in the original ST-HOI formulation.

Load-bearing premise

The comparison is fair: 4D-QA receives the true human tracklet as input, while the baselines receive only a frame and a text prompt, so the reported lead may come partly from that extra information rather than from 4D reasoning.

Editorial extensions

If this is right

  • If 4D-QA's gains replicate, interaction-aware grounding systems should treat object localization as a candidate-selection problem over dense masks rather than direct regression.
  • The GIO benchmark provides a long-tailed, open-world testbed (1,098 classes, 98 unseen at inference) that can expose where current grounding models fail on rare and temporally hidden objects.
  • The reported superiority implies that temporal and 3D cues are not optional for HOI: they resolve cases like a chair occluded by a sitting person that later becomes visible.
  • The 4D layout branch is a plug-in that could be added to other detection and grounding models, potentially improving their performance on occluded-object scenarios.
  • The QA formulation's insensitivity to missing annotations makes it a more reliable evaluation protocol than full ST-HOI triplet detection.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test of whether the reported advantage comes from 4D cues or from the extra human-tracklet input would be to feed the same tracklet to GroundingDINO; the paper does not include this condition.
  • The paper's own ablations show the 3D feature branch adds about 0.7 mAP over the 2D-only version, so the 4D reconstruction's marginal effect is modest relative to the large gap over image baselines.
  • The GIO protocol could be adapted to egocentric or hand-object datasets, extending open-world interacted object grounding beyond third-view whole-body interactions.
  • The benchmark's long-tailed object distribution, with 42.66% of instances outside the COCO class pool, could serve as a lens for studying how vision-language models handle rare-object interactions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces GIO, a new third-view video benchmark for open-world interacted-object grounding, built on AVA with 1,098 object classes and 290K frame-level human-verb-object annotations. It defines a grounding task in which, given a human actor tracklet and optional interaction semantics, a system must localize the interacted object. The authors also propose 4D-QA, which combines SAM mask proposals, SlowFast video features, a reconstructed 3D human-scene layout, and a two-stage 2D/3D transformer decoder to select the interacted mask. Experiments on GIO report that 4D-QA outperforms several image/video grounding and HOI baselines, e.g., 23.38 mAP@0.5 and 29.71 mIoUw versus 17.53 and 20.41 for GroundingDINO.

Significance. The dataset is a potentially valuable resource for ST-HOI research: it is substantially larger in object-class diversity than existing third-view HOI benchmarks, explicitly targets long-tail and open-world objects, and the authors plan to release data and code. The proposed grounding task is a sensible reformulation that avoids the annotation-missing problems of full ST-HOI detection. The 4D-QA method is an interesting combination of segment-proposal and 4D reconstruction, and the paper includes a useful ablation of its components. However, the empirical evidence for the central claim of 'significant superiority' is not yet controlled enough: the main comparison mixes differences in training data and input information with differences in method design, so the reported gap does not by itself establish that spatio-temporal/4D cues are responsible.

major comments (3)
  1. [Sec. 5.3 and Table 2] 4D-QA is trained on the GIO train split (Sec. 5.2), whereas the baselines are used off-the-shelf: Sec. 5.3 describes protocols for adapting GroundingDINO, Qwen-VL, Detic, and CG-STVG, but not fine-tuning them on GIO. The reported gap (e.g., 23.38 vs 17.53 mAP@0.5 for GroundingDINO, and 29.71 vs 20.41 mIoUw) therefore conflates the value of the proposed architecture with the value of training on the target dataset. Please report baselines fine-tuned on the GIO train set under the same task protocol (including the optional interaction semantics), or explicitly reframe the claim as superiority over zero-shot baselines and adjust the abstract and conclusion accordingly.
  2. [Sec. 5.3 and Eqs. (1)-(2)] The task defined in Sec. 3.4 provides the GT human tracklet T_h, and 4D-QA uses it in the human query Q_h via ROIAlign (Sec. 4.4), in the GIoU distance S_d (Eq. 5), and in the 3D alignment centered on the pelvis (Sec. 4.3). Among the strongest baselines, GroundingDINO, Qwen-VL, Detic, and CG-STVG are not conditioned on T_h; only PViC and Gaze receive a human box. The post-hoc distance re-ranking described in Sec. 5.4 uses the GT human box for scoring only, not for detection or attention. To support the claim that spatio-temporal/4D cues drive the improvement, please add a controlled baseline that supplies the human tracklet to these methods (e.g., a box-prompted GroundingDINO or a human-cropped variant) and report the resulting mAP and mIoUw.
  3. [Supplementary Sec. B, 'Other Details'] The three post-processing hyperparameters gamma, tau, and beta are chosen by grid search 'to identify the optimal values that maximize the model's performance,' but the manuscript does not state whether the search is performed on a held-out validation split or on the test set. If the latter, the reported numbers in Table 2 and Table 3 are optimistically biased. Please specify the split used for tuning and include a sensitivity analysis, for example results for neighboring grid values, so readers can judge the stability of the reported gains.
minor comments (4)
  1. [Abstract] The data URL in the abstract contains an apparent typo ('HAKE-A V A.' with a space); it should be given consistently as https://github.com/DirtyHarryLYL/HAKE-AVA.
  2. [Sec. 3.2] The word 'traklet' should be 'tracklet'; the description of multi-object linking would also be clearer if the rule for multiple objects in a sub-clip were stated before defining the ST-HOI tracklet label.
  3. [Sec. 5.1 and Table 2] The metric definitions are non-standard: the mAP described in Sec. 5.1 is a rank-based 'first-hit' average, not the standard detection mAP, and the table header 'mAPs' is inconsistent with the text. Please define the metric unambiguously in the main text and avoid the plural 'mAPs'.
  4. [Fig. 2] The caption contains 'Tacking Align', which appears to be a typo for 'tracking/alignment'; the figure would also benefit from a clearer indication of where the optional GroundingDINO feature enters the pipeline.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: 4D-QA is trained with supervised BCE loss, validated by within-paper ablations and an external VidHOI transfer, and its reported superiority is not an identity with its inputs.

full rationale

The claimed derivation chain is: define the GIO task (Eq. 1: predict interacted object tracklet given clip, GT human tracklet, and optional interaction semantics); generate SAM mask proposals; extract 2D, 3D, and context features; score candidate masks by Eq. 5 (cosine similarity plus GIoU distance to the human); and train with BCE loss. No prediction in this chain is defined as a fit from the target label; the object box is selected from SAM proposals by a learned decoder, and the distance term is a heuristic prior that is ablated in Table 3 (removing it degrades mAP from 23.38 to 22.07). The ablation table and the VidHOI zero-shot/fine-tuned evaluation (14.23/25.35 mAP and 22.66/29.61 mIoUw) provide checks that are independent of the GIO test-set fitting. The author-overlap citations (e.g., Li et al. 2020a for 3D HOI representation, and the HAKE-AVA data URL) motivate but do not carry the central claim: the 3D contribution is verified in-paper rather than assumed from the citation. The main confounds are evaluation-fairness/rigor issues rather than circularity: 4D-QA receives the GT human tracklet in Eq. 1-2 while GroundingDINO, Qwen-VL, Detic, and CG-STVG are not given that tracklet, and the appendix grid-searches the post-processing hyperparameters (gamma, tau, beta) to maximize performance without stating a validation split. These concerns affect how strongly the reported gap can be attributed to 4D/spatio-temporal cues, but they do not make any equation or result equivalent to its own input by construction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The benchmark and method rest on the reliability of AVA's human and action labels, on SAM's proposal coverage, on the accuracy of the PHALP and ZoeDepth 4D reconstruction, and on the post-processing thresholds found by grid search. None of these is independently verified in the paper.

free parameters (4)
  • gamma (GIoU distance weight) = not reported
    Eq. 5 mixes the mask score and the human-object GIoU distance; tuned by grid search in the appendix.
  • tau (mask acceptance threshold) = not reported
    Sec. 4.4 uses tau to decide whether a mask belongs to the target object.
  • beta (depth clustering threshold) = not reported
    Appendix B uses beta to group masks with similar depth before forming the output box.
  • positive BCE loss weight = 10
    Sec. 5.2 sets the loss coefficient for true positions to ten times that of false positions; this affects training balance.
assumptions (4)
  • domain assumption AVA human boxes and action labels are accurate enough to serve as ground truth for GIO.
    GIO inherits AVA's human tracks and action labels (Sec. 3.1); no re-annotation of those labels is reported.
  • domain assumption SAM produces a candidate mask that covers each interacted object with enough IoU (ratio > 0.9 for GT), including small and occluded objects.
    The method can only select among SAM masks (Sec. 4.2); the limitations section itself notes poor performance on small objects.
  • domain assumption The 4D reconstruction from PHALP and ZoeDepth, after alignment, is accurate enough to benefit grounding.
    Sec. 4.3 states the reconstruction precision is limited but sufficient; no quantitative evaluation of reconstruction quality is given.
  • domain assumption Annotator judgments and the WordNet-based clustering produce valid object class labels for 1,098 classes.
    Sec. 3.2 describes typo fixing and clustering but reports no inter-annotator agreement or label error rate.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Interacted Object Grounding in Spatio-Temporal Human-Object Interactions." pith.science (2026). https://pith.science/paper/TF2TZWNC

@misc{pith2026241219542,
  author       = {Pith},
  title        = {Pith review of: Interacted Object Grounding in Spatio-Temporal Human-Object Interactions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TF2TZWNC}},
  note         = {Machine review of arXiv:2412.19542}
}
read the original abstract

Spatio-temporal Human-Object Interaction (ST-HOI) understanding aims at detecting HOIs from videos, which is crucial for activity understanding. However, existing whole-body-object interaction video benchmarks overlook the truth that open-world objects are diverse, that is, they usually provide limited and predefined object classes. Therefore, we introduce a new open-world benchmark: Grounding Interacted Objects (GIO) including 1,098 interacted objects class and 290K interacted object boxes annotation. Accordingly, an object grounding task is proposed expecting vision systems to discover interacted objects. Even though today's detectors and grounding methods have succeeded greatly, they perform unsatisfactorily in localizing diverse and rare objects in GIO. This profoundly reveals the limitations of current vision systems and poses a great challenge. Thus, we explore leveraging spatio-temporal cues to address object grounding and propose a 4D question-answering framework (4D-QA) to discover interacted objects from diverse videos. Our method demonstrates significant superiority in extensive experiments compared to current baselines. Data and code will be publicly available at https://github.com/DirtyHarryLYL/HAKE-AVA.

Figures

Figures reproduced from arXiv: 2412.19542 by the authors.

Figure 1
Figure 1. In daily HOIs, we interact with diverse objects [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overview of our 4D-QA. It utilizes a 4D question-answering paradigm to effectively locate the interacted objects. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. demonstrates the above process. 4.3 Multi-Modal Feature To fully leverage the temporal and spatial continuity fea￾tures of videos, including object information, HOI details, (a) Original image. (b) SAM masks. (c) Accurate bbox. (d) GT mask&bbox [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Visualization of interacted object grounding. We [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Fine-grained performance analysis. grounding for spatiotemporal HOI understanding. In addition, we considered ST-HOI as the task design, resulting in the highest mAP of 6.8, i.e., the ST-HOI task is kind of too challenging even ignoring the annotation missing problem. …
Figure 6
Figure 6. Figure 6: (a) shows the frequency of occurrence of object [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 9
Figure 9. Figure 9: The distribution of tracklet length [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 7
Figure 7. Figure 7: The distribution of tracklet number per action. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: The distribution of normalized object size. [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 13
Figure 13. Figure 13: One frame with multiple HOIs results. 29.59 mIoUw and 22.80 mAP@0.5. Some samples contain￾ing multiple pairs of HOI are shown in [PITH_FULL_IMAGE:figures/full_fig_p013_13.png]
Figure 12
Figure 12. Figure 12: Some bad predictions of CG-STVG. Green, blue, [PITH_FULL_IMAGE:figures/full_fig_p013_12.png]
Figure 14
Figure 14. Figure 14: Data samples and their ST-HOI labels in GIO. [PITH_FULL_IMAGE:figures/full_fig_p014_14.png]
Figure 15
Figure 15. Figure 15: Visualization of 3D reconstructions from 4D-QA. [PITH_FULL_IMAGE:figures/full_fig_p015_15.png]
Figure 16
Figure 16. Figure 16: Visualization of grounding results [PITH_FULL_IMAGE:figures/full_fig_p016_16.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

68 extracted references · 59 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Bai, J.; Bai, S.; Yang, S.; Wang, S.; Tan, S.; Wang, P.; Lin, J.; Zhou, C.; and Zhou, J. 2023. Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond. arXiv:2308.12966

    arXiv 2023

  4. [4]

    Baradel, F.; Neverova, N.; Wolf, C.; Mille, J.; and Mori, G. 2018. Object level visual reasoning in videos. In ECCV

    work page 2018

  5. [5]

    F.; Birkl, R.; Wofk, D.; Wonka, P.; and M \"u ller, M

    Bhat, S. F.; Birkl, R.; Wofk, D.; Wonka, P.; and M \"u ller, M. 2023. Zoedepth: Zero-shot transfer by combining relative and metric depth. arXiv preprint arXiv:2302.12288

    arXiv 2023

  6. [6]

    Bras \'o , G.; and Leal-Taix \'e , L. 2020. Learning a neural solver for multiple object tracking. In CVPR

    work page 2020

  7. [7]

    Caba Heilbron, F.; Escorcia, V.; Ghanem, B.; and Carlos Niebles, J. 2015. Activitynet: A large-scale video benchmark for human activity understanding. In CVPR

    work page 2015

  8. [8]

    Chao, Y.-W.; Liu, Y.; Liu, X.; Zeng, H.; and Deng, J. 2018. Learning to Detect Human-Object Interactions. In WACV

    work page 2018

Show all 68 references
  1. [9]

    W.; Wang, Z.; He, Y.; Wang, J.; and Deng, J

    Chao, Y. W.; Wang, Z.; He, Y.; Wang, J.; and Deng, J. 2015. HICO: A Benchmark for Recognizing Human-Object Interactions in Images. In ICCV

    2015

  2. [10]

    Chen, Y.; Cao, Y.; Hu, H.; and Wang, L. 2020 a . Memory Enhanced Global-Local Aggregation for Video Object Detection. In CVPR

    2020

  3. [11]

    Chen, Z.; Zhong, B.; Li, G.; Zhang, S.; and Ji, R. 2020 b . Siamese Box Adaptive Network for Visual Tracking. In CVPR

    2020

  4. [12]

    Chiou, M.-J.; Liao, C.-Y.; Wang, L.-W.; Zimmermann, R.; and Feng, J. 2021. ST-HOI: A Spatial-Temporal Baseline for Human-Object Interaction Detection in Videos. In Proceedings of the 2021 Workshop on Intelligent Cross-Data Analysis and Retrieval, 9–17

    2021

  5. [13]

    Damen, D.; Doughty, H.; Maria Farinella, G.; Fidler, S.; Furnari, A.; Kazakos, E.; Moltisanti, D.; Munro, J.; Perrett, T.; Price, W.; et al. 2018. Scaling egocentric vision: The epic-kitchens dataset. In ECCV

    2018

  6. [14]

    Fan, H.; Lin, L.; Yang, F.; Chu, P.; Deng, G.; Yu, S.; Bai, H.; Xu, Y.; Liao, C.; and Ling, H. 2019. Lasot: A high-quality benchmark for large-scale single object tracking. In CVPR

    2019

  7. [15]

    Fan, Q.; Zhuo, W.; Tang, C.-K.; and Tai, Y.-W. 2020. Few-Shot Object Detection with Attention-RPN and Multi-Relation Detector. In CVPR

    2020

  8. [16]

    Feichtenhofer, C.; Fan, H.; Malik, J.; and He, K. 2019. Slowfast networks for video recognition. In ICCV

    2019

  9. [17]

    F.; Kuo, W.-c.; Efros, A

    Fouhey, D. F.; Kuo, W.-c.; Efros, A. A.; and Malik, J. 2018. From lifestyle vlogs to everyday interactions. In CVPR

    2018

  10. [18]

    Girdhar, R.; Carreira, J.; Doersch, C.; and Zisserman, A. 2019. Video action transformer network. In CVPR

    2019

  11. [19]

    Gkioxari, G.; Girshick, R.; Doll \'a r, P.; and He, K. 2018. Detecting and recognizing human-object interactions. In CVPR

    2018

  12. [20]

    Something Something

    Goyal, R.; Kahou, S. E.; Michalski, V.; Materzynska, J.; Westphal, S.; Kim, H.; Haenel, V.; Fruend, I.; Yianilos, P.; Mueller-Freitag, M.; et al. 2017. The" Something Something" Video Database for Learning and Evaluating Visual Common Sense. In ICCV

    2017

  13. [21]

    A.; Vondrick, C.; Pantofaru, C.; Li, Y.; Vijayanarasimhan, S.; Toderici, G.; Ricco, S.; Sukthankar, R.; Schmid, C.; and Malik, J

    Gu, C.; Sun, C.; Ross, D. A.; Vondrick, C.; Pantofaru, C.; Li, Y.; Vijayanarasimhan, S.; Toderici, G.; Ricco, S.; Sukthankar, R.; Schmid, C.; and Malik, J. 2018. Ava: A video dataset of spatio-temporally localized atomic visual actions. In CVPR

    2018

  14. [22]

    Gu, X.; Fan, H.; Huang, Y.; Luo, T.; and Zhang, L. 2024. Context-Guided Spatio-Temporal Video Grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18330--18339

    2024

  15. [23]

    Hu, H.; Gu, J.; Zhang, Z.; Dai, J.; and Wei, Y. 2018. Relation networks for object detection. In CVPR

    2018

  16. [24]

    Ji, J.; Krishna, R.; Fei-Fei, L.; and Niebles, J. C. 2020. Action Genome: Actions As Compositions of Spatio-Temporal Scene Graphs. In CVPR

    2020

  17. [25]

    Kato, K.; Li, Y.; and Gupta, A. 2018. Compositional learning for human object interaction. In ECCV

    2018

  18. [26]

    Kim, C.; Li, F.; Ciptadi, A.; and Rehg, J. M. 2015. Multiple hypothesis tracking revisited. In ICCV

    2015

  19. [27]

    Kim, D.; Lee, G.; Jeong, J.; and Kwak, N. 2020. Tell Me What They're Holding: Weakly-Supervised Object Detection with Transferable Knowledge from Human-Object Interaction. In AAAI

    2020

  20. [29]

    C.; Lo, W.-Y.; et al

    Kirillov, A.; Mintun, E.; Ravi, N.; Mao, H.; Rolland, C.; Gustafson, L.; Xiao, T.; Whitehead, S.; Berg, A. C.; Lo, W.-Y.; et al. 2023 b . Segment anything. arXiv preprint arXiv:2304.02643

    2023 arXiv

  21. [30]

    H.; Zhang, P.; Zhang, H.; Yang, J.; Li, C.; Zhong, Y.; Wang, L.; Yuan, L.; Zhang, L.; Hwang, J.-N.; et al

    Li, L. H.; Zhang, P.; Zhang, H.; Yang, J.; Li, C.; Zhong, Y.; Wang, L.; Yuan, L.; Zhang, L.; Hwang, J.-N.; et al. 2022 a . Grounded language-image pre-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10965--10975

    2022

  22. [31]

    Li, Y.-L.; Liu, X.; Lu, H.; Wang, S.; Liu, J.; Li, J.; and Lu, C. 2020 a . Detailed 2D-3D Joint Representation for Human-Object Interaction. In CVPR

    2020

  23. [32]

    Li, Y.-L.; Liu, X.; Wu, X.; Huang, X.; Xu, L.; and Lu, C. 2022 b . Transferable Interactiveness Knowledge for Human-Object Interaction Detection. In TPAMI

    2022

  24. [33]

    Li, Y.-L.; Liu, X.; Wu, X.; Li, Y.; and Lu, C. 2020 b . HOI Analysis: Integrating and Decomposing Human-Object Interaction. In NeurIPS

    2020

  25. [34]

    Li, Y.-L.; Xu, L.; Liu, X.; Huang, X.; Xu, Y.; Chen, M.; Ma, Z.; Wang, S.; Fang, H.-S.; and Lu, C. 2019 a . Hake: Human activity knowledge engine. arXiv preprint arXiv:1904.06539

    2019 arXiv

  26. [35]

    Li, Y.-L.; Xu, L.; Liu, X.; Huang, X.; Xu, Y.; Wang, S.; Fang, H.-S.; Ma, Z.; Chen, M.; and Lu, C. 2020 c . PaStaNet: Toward Human Activity Knowledge Engine. In CVPR

    2020

  27. [36]

    Li, Y.-L.; Zhou, S.; Huang, X.; Xu, L.; Ma, Z.; Fang, H.-S.; Wang, Y.; and Lu, C. 2019 b . Transferable interactiveness knowledge for human-object interaction detection. In CVPR

    2019

  28. [37]

    Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Dollár, P.; and Zitnick, C

    Lin, T. Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Dollár, P.; and Zitnick, C. L. 2014. Microsoft COCO: Common Objects in Context. In ECCV

    2014

  29. [38]

    Liu, C.; Jin, Y.; Xu, K.; Gong, G.; and Mu, Y. 2020. Beyond Short-Term Snippet: Video Relation Detection with Spatio-Temporal Global Context. In CVPR

    2020

  30. [39]

    Liu, S.; Zeng, Z.; Ren, T.; Li, F.; Zhang, H.; Yang, J.; Li, C.; Yang, J.; Su, H.; Zhu, J.; et al. 2023. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499

    2023 arXiv

  31. [40]

    Liu, X.; Li, Y.-L.; and Lu, C. 2022. Highlighting Object Category Immunity for the Generalization of Human-Object Interaction Detection. In AAAI 2022

    2022

  32. [41]

    Liu, X.; Li, Y.-L.; Wu, X.; Tai, Y.-W.; Lu, C.; and Tang, C.-K. 2022. Interactiveness Field in Human-Object Interactions. In CVPR

    2022

  33. [42]

    Loper, M.; Mahmood, N.; Romero, J.; Pons-Moll, G.; and Black, M. J. 2015. SMPL : A Skinned Multi-Person Linear Model. ACM Trans. Graphics (Proc. SIGGRAPH Asia)

    2015

  34. [43]

    Materzynska, J.; Xiao, T.; Herzig, R.; Xu, H.; Wang, X.; and Darrell, T. 2020. Something-Else: Compositional Action Recognition with Spatial-Temporal Interaction Networks. In CVPR

    2020

  35. [44]

    Miller, G. A. 1995. WordNet: a lexical database for English. Communications of the ACM

    1995

  36. [45]

    Ni, Z.; Valls Mascar\'o , E.; Ahn, H.; and Lee, D. 2023. Human–Object Interaction Prediction in Videos through Gaze Following. Computer Vision and Image Understanding, 233: 103741

    2023

  37. [46]

    Prokudin, S.; Lassner, C.; and Romero, J. 2019. Efficient Learning on Point Clouds With Basis Point Sets. In ECCV

    2019

  38. [47]

    Qi, S.; Wang, W.; Jia, B.; Shen, J.; and Zhu, S.-C. 2018. Learning human-object interactions by graph parsing neural networks. In ECCV

    2018

  39. [48]

    Rajasegaran, J.; Pavlakos, G.; Kanazawa, A.; and Malik, J. 2022. Tracking People by Predicting 3 D Appearance, Location & Pose. In CVPR

    2022

  40. [49]

    Redmon, J.; Divvala, S.; Girshick, R.; and Farhadi, A. 2016. You only look once: Unified, real-time object detection. In CVPR

    2016

  41. [50]

    Ren, S.; He, K.; Girshick, R.; and Sun, J. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. In NIPS

    2015

  42. [51]

    Rezatofighi, H.; Tsoi, N.; Gwak, J.; Sadeghian, A.; Reid, I.; and Savarese, S. 2019. Generalized Intersection over Union. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2019

  43. [52]

    Ristani, E.; Solera, F.; Zou, R.; Cucchiara, R.; and Tomasi, C. 2016. Performance measures and a data set for multi-target, multi-camera tracking. In ECCV

    2016

  44. [53]

    Sadeghian, A.; Alahi, A.; and Savarese, S. 2017. Tracking the untrackable: Learning to track multiple cues with long-term dependencies. In CVPR

    2017

  45. [54]

    Sadhu, A.; Chen, K.; and Nevatia, R. 2020. Video Object Grounding using Semantic Roles in Language Description. In CVPR

    2020

  46. [55]

    Shan, D.; Geng, J.; Shu, M.; and Fouhey, D. F. 2020. Understanding Human Hands in Contact at Internet Scale. In CVPR

    2020

  47. [56]

    Shang, X.; Ren, T.; Guo, J.; Zhang, H.; and Chua, T.-S. 2017. Video visual relation detection. In ACMMM

    2017

  48. [57]

    A.; Varol, G.; Wang, X.; Farhadi, A.; Laptev, I.; and Gupta, A

    Sigurdsson, G. A.; Varol, G.; Wang, X.; Farhadi, A.; Laptev, I.; and Gupta, A. 2016. Hollywood in homes: Crowdsourcing data collection for activity understanding. In ECCV

    2016

  49. [58]

    Wang, L.; Huang, B.; Zhao, Z.; Tong, Z.; He, Y.; Wang, Y.; Wang, Y.; and Qiao, Y. 2023. VideoMAE V2: Scaling Video Masked Autoencoders With Dual Masking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2023

  50. [59]

    Wang, X.; and Gupta, A. 2018. Videos as space-time region graphs. In ECCV

    2018

  51. [60]

    Weinzaepfel, P.; Martin, X.; and Schmid, C. 2016. Human action localization with sparse spatial supervision. arXiv preprint arXiv:1605.05197

    2016 arXiv

  52. [61]

    Wu, X.; Li, Y.-L.; Liu, X.; Zhang, J.; Wu, Y.; and Lu, C. 2022. Mining Cross-Person Cues for Body-Part Interactiveness Learning in HOI Detection. In ECCV

    2022

  53. [62]

    Xu, X.; Li, Y.-L.; and Lu, C. 2022. Learning to Anticipate Future with Dynamic Context Removal. In CVPR

    2022

  54. [63]

    Yang, Z.; Mahajan, D.; Ghadiyaram, D.; Nevatia, R.; and Ramanathan, V. 2019. Activity driven weakly supervised object detection. In CVPR

    2019

  55. [64]

    Yao, L.; Han, J.; Wen, Y.; Liang, X.; Xu, D.; Zhang, W.; Li, Z.; Xu, C.; and Xu, H. 2022. Detclip: Dictionary-enriched visual-concept paralleled pre-training for open-world detection. Advances in Neural Information Processing Systems, 35: 9125--9138

    2022

  56. [65]

    Yuan, Y.; Liang, X.; Wang, X.; Yeung, D.-Y.; and Gupta, A. 2017. Temporal dynamic graph LSTM for action-driven video object detection. In ICCV

    2017

  57. [66]

    Z.; Yuan, Y.; Campbell, D.; Zhong, Z.; and Gould, S

    Zhang, F. Z.; Yuan, Y.; Campbell, D.; Zhong, Z.; and Gould, S. 2023. Exploring Predicate Visual Context in Detecting Human–Object Interactions. In ICCV

    2023

  58. [67]

    M.; and Shum, H.-Y

    Zhang, H.; Li, F.; Liu, S.; Zhang, L.; Su, H.; Zhu, J.; Ni, L. M.; and Shum, H.-Y. 2022. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605

    2022 arXiv

  59. [68]

    a henb \

    Zhou, X.; Girdhar, R.; Joulin, A.; Kr \"a henb \"u hl, P.; and Misra, I. 2022. Detecting Twenty-thousand Classes using Image-level Supervision. In ECCV

    2022

  60. [69]

    Zhuo, T.; Cheng, Z.; Zhang, P.; Wong, Y.; and Kankanhalli, M. 2019. Explainable video action reasoning via prior knowledge and state transitions. In ACMMM

    2019

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.