Pith. sign in

REVIEW 4 major objections 5 minor 55 references

GenHOI: Generalizing Text-driven 4D Human-Object Interaction Synthesis for Unseen Objects

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Two-stage GenHOI animates unseen objects from text in 4D

desk verdict A genuinely useful two-stage design for text-driven 4D HOI synthesis, but the headline generalization claim is propped up by a synthetic evaluation that measures less than it appears. read the letter →

arxiv 2506.15483 v1 pith:SEZSEW2M submitted 2025-06-18 cs.CV cs.AI

Shujia Li , Haiyu Zhang , Xinyuan Chen , Yaohui Wang , Yutong Ban This is my paper
classification cs.CVcs.AI
keywords 4Dhuman-objectinteractionsynthesistext-drivenmotiongenerationdiffusionmodelsunseenobjectgeneralizationkeyframerecoverycontact-awareattentionObject-AnchorNetContactDM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the scarcity of 4D human-object interaction (HOI) datasets can be sidestepped by splitting the problem into a spatial stage and a temporal stage. First, an Object-AnchorNet recovers sparse 3D HOI keyframes—object poses relative to a human—using only 3D HOI data and the object's geometry, so it can handle objects the model has never seen. Second, a Contact-Aware Diffusion Model (ContactDM) interpolates those keyframes into a dense, temporally coherent 4D sequence, using a contact-aware encoder and attention to keep hands and objects in proper contact. The authors report that this pipeline beats prior text-driven HOI methods on the OMOMO and 3D-FUTURE datasets, including on objects not seen during training.

What carries the argument

The load-bearing mechanism is the two-stage decoupling of spatial and temporal modeling. In stage one, Object-AnchorNet (adapted from the Object Pop-up architecture [34]) takes a human point cloud, an object template point cloud, and a text prompt, and predicts the object's pose to form 3D HOI keyframes; it is trained on a union of 3D HOI datasets (BEHAVE [1], GRAB [39], Open3DHOI [44]). In stage two, ContactDM—a diffusion transformer following the CHOIS architecture [22]—interpolates the K=5 keyframes into dense 4D sequences, using a Contact-Aware Encoder (PointNet++ [35] with KNN sampling to isolate human points nearest the object) and a Contact-Aware HOI Attention (cross-attention where the keyframe features serve as keys and values). The contact signals, along with CLIP text embeddings [36] and BPS object-geometry features, condition the denoising process.

What would settle it

Run GenHOI on a motion in which the only hand-object contact occurs exactly between two of the five uniformly spaced keyframes—for example, a brief touch or a grasp-and-release within a single interval—and check whether the generated sequence misses, blurs, or penetrates that contact; a failure would falsify the K=5 uniform-sampling assumption.

Watch

Extended reading notes

Core claim

GenHOI's central claim is that high-fidelity 4D HOI synthesis for unseen objects is achievable without a large 4D HOI dataset, by decoupling spatial and temporal modeling. The spatial stage, Object-AnchorNet, learns from a mixture of 3D HOI datasets (BEHAVE, GRAB, Open3DHOI) to predict an object's pose from a human point cloud and object geometry, producing K=5 keyframes that capture the interaction's spatial layout. The temporal stage, ContactDM, is a diffusion model trained on the smaller OMOMO 4D dataset that interpolates these keyframes into 120-frame sequences, with a Contact-Aware Encoder (PointNet++ over human and object point clouds with KNN contact sampling) and a Contact-Aware HOI Attention (cross-attention) injecting contact signals into the diffusion latents. The paper reports that this design achieves state-of-the-art metrics on OMOMO and 3D-FUTURE, and that the full model matches a variant given ground-truth keyframes, indicating that the predicted keyframes are reliable enough for downstream interpolation.

Load-bearing premise

The pipeline assumes that five uniformly sampled keyframes capture all contact-critical moments of an interaction, so any grasp, release, or contact transition that happens between keyframes is lost before the diffusion stage can attempt to recover it.

Editorial extensions

If this is right

  • The reliance on large-scale 4D HOI datasets is reduced: the first stage trains on 3D HOI data, and only the second stage needs a comparatively small 4D dataset such as OMOMO.
  • Objects never seen in training—including shapes from a different dataset (3D-FUTURE)—can be animated in coherent 4D interactions, as long as their geometry is provided.
  • The full GenHOI model performs comparably to a variant that is given ground-truth keyframes, so errors in the keyframe-recovery stage do not yet bottleneck overall quality.
  • Ablations indicate that K=5 keyframes is the sweet spot: fewer lose interaction dynamics, while more accumulate keyframe prediction error.
  • Contact-aware conditioning (KNN sampling plus cross-attention) is what allows the model to maintain hand-object contact without penetration.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to test the same two-stage decoupling on multi-object or articulated-object interactions; the paper itself notes multi-object scenarios are a limitation, so the keyframe and contact machinery may need to model inter-object contacts explicitly.
  • Because the second stage is a diffusion model, keyframe conditioning could be made adaptive—for example, sampling keyframes at non-uniform times based on contact-velocity heuristics—which might recover grasp or release events that uniform K=5 sampling misses.
  • The Object-AnchorNet's point-cloud formulation could transfer to robotics: given only an object's geometry, a robot could query the model to predict how a person would grasp and manipulate it, serving as a data-driven affordance prior.
  • One way to measure the ceiling of the two-stage approach is to feed ContactDM deliberately chosen keyframes that include contact transitions (rather than uniform keyframes); if quality improves substantially, better keyframe selection becomes the next bottleneck.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes GenHOI, a two-stage framework for text-driven 4D human-object interaction (HOI) synthesis. Stage 1 uses an Object-AnchorNet (adapted from Object Pop-up) to recover sparse 3D HOI keyframes from human point clouds, object geometry, and text, trained on BEHAVE, GRAB, and Open3DHOI. Stage 2 uses a Contact-Aware Diffusion Model (ContactDM), built on the CHOIS architecture, to interpolate the keyframes into dense 4D sequences, with a Contact-Aware Encoder and Contact-Aware HOI Attention to inject contact patterns. Experiments on OMOMO (seen and unseen objects) and on 3D-FUTURE objects (via pseudo-ground-truth object motions from OMOMO) report state-of-the-art results, plus ablations on keyframe count and conditioning strategies. The central claim is that GenHOI generalizes to unseen objects while reducing reliance on large-scale 4D HOI datasets.

Significance. If substantiated, the two-stage decoupling of spatial keyframe recovery and temporal interpolation is a useful direction for alleviating the scarcity of 4D HOI data. The paper provides a concrete architecture, detailed ablations, and a project page, and the contact-aware attention design is well motivated. However, the headline claim of strong generalization to unseen objects rests on evaluation protocols that are partially synthetic and on an input setup that is not fully disclosed. The idea is promising, but the current evidence is not yet convincing at the level claimed.

major comments (4)
  1. [Sec. 4.2, Table 3] The cross-dataset evaluation on 3D-FUTURE uses pseudo-ground-truth object trajectories created by integrating 3D-FUTURE object geometries with OMOMO test-set motions (as in CHOIS). Consequently, MPJPE and Troot measure deviation from a synthetic reference whose object dynamics are drawn from the same distribution used in training, and the contact metrics (C%, Phand) are computed against this pseudo-object. Tobj and Oobj are not reported for this setting. The large improvements in Table 3 (e.g., MPJPE 10.82 vs. 16.39) therefore do not demonstrate generalization to genuinely novel object trajectories; they show interpolation quality on a synthetic distribution. Since this table is the main evidence for the abstract's claim of 'strong generalization to unseen objects', the claim is overstated. Please report object errors and, if possible, evaluate on a setting with real object motions for the 3D-FUTURE objects.
  2. [Sec. 3.2 and Sec. 4.2] The experimental protocol for GenHOI is underspecified regarding the source of the human keyframes used as input to Object-AnchorNet. The paper states that human motion sequences can be sampled from generative models, datasets, or motion capture, but it does not say which option is used in the reported experiments. If ground-truth OMOMO human motion is used to extract keyframes, then GenHOI is not evaluated end-to-end as a text-to-4D-HOI method: it receives human keyframes that the baseline methods (e.g., CHOIS) must synthesize. This would make the comparisons in Tables 1-3 inequitable and the 'text-driven' claim problematic. The authors must clearly state the input modalities at test time and, if a separate human motion generator is used, identify it and report end-to-end results.
  3. [Sec. 4.2, Table 2] On the OMOMO unseen-object split, the full GenHOI model achieves only marginal improvements over CHOIS on human metrics (MPJPE 16.07 vs. 16.50; Troot 26.92 vs. 28.78) and is worse on object metrics (Tobj 14.98 vs. 14.29; Oobj 1.20 vs. 1.04). This is the more direct cross-object generalization test, yet the claimed advantage is not clear. The paper's conclusion of 'robust generalization' relies heavily on the synthetic 3D-FUTURE evaluation, which is problematized above. The authors should temper the claim or provide additional evidence, such as results on a dataset with real object dynamics for unseen objects.
  4. [Sec. 3.2, Table 4] The choice of K=5 uniformly averaged keyframes is ablated only on the seen-object split of OMOMO. For interactions with discrete contact events (e.g., grasping, releasing), uniform keyframe averaging may miss the critical moments, and the ablation does not test robustness on unseen objects or across actions. The current evidence does not rule out that K=5 is tuned to the seen-object distribution. Please add ablations on an unseen-object split or analyze the sensitivity of the keyframe sampling strategy.
minor comments (5)
  1. [Throughout] There are multiple typos and grammatical errors: 'Addtionally' (Sec. 2.2), 'Diffuion' (Sec. 3.3), 'Interation' in Table captions, and 'temporaly' in Sec. 3.3. A careful proofread is needed.
  2. [Abstract and Sec. 4.2] The phrase 'state-of-the-art results on the publicly available OMOMO and 3D-FUTURE datasets' is misleading because 3D-FUTURE is not an HOI dataset with ground-truth 4D interactions; the evaluation uses pseudo-ground truth. Please rephrase to avoid implying that 3D-FUTURE provides direct HOI ground truth.
  3. [Sec. 4.4] The 'GenHOI w/o CA' ablation in Tables 1-3 removes the keyframe recovery stage and both contact-aware components simultaneously, so it does not isolate the contribution of the contact-aware modules. The separate ablations in Table 5 do isolate them, but the naming and presentation in the main tables are confusing.
  4. [Sec. 4.2] The differences between GenHOI and CHOIS in Table 2 are small (e.g., 0.43 cm MPJPE, 1.86 cm Troot), yet no error bars or significance tests are reported. Given the small magnitudes, statistical variability could affect the conclusions.
  5. [Sec. 3.2 and Appendix A.2] The training mixture for Object-AnchorNet includes Open3DHOI [44], but the data preprocessing for that dataset is not described. Please provide details on how the 'single aggregated subject' was constructed and how the text prompts were obtained for the 3D HOI datasets.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the pipeline is a composition of openly cited prior components and its claims are empirical.

full rationale

The paper's derivation chain is a two-stage supervised pipeline: Object-AnchorNet (Sec. 3.2) regresses object pose from human point clouds using 3D HOI datasets, and ContactDM (Sec. 3.3) is a diffusion model trained on OMOMO 4D sequences to interpolate sparse keyframes; neither stage defines its output in terms of the evaluation metric or of the final generalization claim. The 3D-FUTURE protocol (Sec. 4.2, Table 3) does construct pseudo-ground-truth by attaching 3D-FUTURE objects to OMOMO test motions, following CHOIS [22], and this legitimately weakens the strength of the 'unseen objects' generalization evidence, but it is a benchmark-validity concern, not circularity: the model is not fitted to those pseudo-GT metrics, and the reported MPJPE/Troot values are not equal to any conditioning input by construction. The K=5 keyframe choice is justified by ablation (Table 4), and the borrowings from CHOIS [22] and Object Pop-up [34] are external prior works openly cited, not self-citations. The GenHOI-GT control (Sec. 4.5) is an ablation of the first stage, not a renamed prediction. No self-citation chain is load-bearing, and no equation reduces a claimed prediction to its own input; the appropriate finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No genuinely new physical entities are introduced. The framework relies on standard components (diffusion, PointNet++, CLIP, SMPL-X) and a set of hand-selected hyperparameters, plus the assumption that static 3D HOI data transfers to unseen objects for keyframe recovery.

free parameters (4)
  • K (number of keyframes) = 5
    Chosen by hand and ablation (Table 4) to balance fidelity and error; directly controls stage 1 output cardinality and impacts all stage 2 results.
  • Mo (object point sample count) = 500
    Set in Sec 3.3; controls object geometry resolution in the Contact-Aware Encoder.
  • Mh (human point sample count) = 1000
    Set in Sec 3.3; KNN-selected human points near the object define contact context.
  • Training hyperparameters = lr 1e-4, batch 32, 50k steps, latent 1024, 4 blocks
    Reported in Sec 4.1; not tuned to a scientific constant, but central to reproducing results.
assumptions (5)
  • domain assumption Static 3D HOI datasets (BEHAVE, GRAB, Open3DHOI) provide adequate diversity to train Object-AnchorNet for unseen objects
    Sec 3.2 claims multi-source training improves generalizability; no formal guarantee.
  • domain assumption Point clouds from SMPL-X meshes and object templates contain enough geometric signal to infer object pose
    Sec 3.2 and Appendix A.2; basis of Object-AnchorNet.
  • domain assumption KNN-based nearest human points to object capture contact-relevant regions
    Sec 3.3 Contact-Aware Encoder; uniform sampling fails in Table 5, showing this choice is load-bearing.
  • domain assumption The diffusion framework [16] and architecture choices from CHOIS [22] are applicable and stable for 4D HOI interpolation
    Used without proof in Sec 3.3; standard ML practice but an unverified empirical assumption.
  • ad hoc to paper The 3D-FUTURE evaluation using OMOMO test motions as pseudo-GT yields meaningful MPJPE and Troot
    Sec 4.2 Table 3; no real 4D GT exists for 3D-FUTURE, so this protocol is constructed for this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GenHOI: Generalizing Text-driven 4D Human-Object Interaction Synthesis for Unseen Objects." pith.science (2026). https://pith.science/paper/SEZSEW2M

@misc{pith2026250615483,
  author       = {Pith},
  title        = {Pith review of: GenHOI: Generalizing Text-driven 4D Human-Object Interaction Synthesis for Unseen Objects},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SEZSEW2M}},
  note         = {Machine review of arXiv:2506.15483}
}
read the original abstract

While diffusion models and large-scale motion datasets have advanced text-driven human motion synthesis, extending these advances to 4D human-object interaction (HOI) remains challenging, mainly due to the limited availability of large-scale 4D HOI datasets. In our study, we introduce GenHOI, a novel two-stage framework aimed at achieving two key objectives: 1) generalization to unseen objects and 2) the synthesis of high-fidelity 4D HOI sequences. In the initial stage of our framework, we employ an Object-AnchorNet to reconstruct sparse 3D HOI keyframes for unseen objects, learning solely from 3D HOI datasets, thereby mitigating the dependence on large-scale 4D HOI datasets. Subsequently, we introduce a Contact-Aware Diffusion Model (ContactDM) in the second stage to seamlessly interpolate sparse 3D HOI keyframes into densely temporally coherent 4D HOI sequences. To enhance the quality of generated 4D HOI sequences, we propose a novel Contact-Aware Encoder within ContactDM to extract human-object contact patterns and a novel Contact-Aware HOI Attention to effectively integrate the contact signals into diffusion models. Experimental results show that we achieve state-of-the-art results on the publicly available OMOMO and 3D-FUTURE datasets, demonstrating strong generalization abilities to unseen objects, while enabling high-fidelity 4D HOI generation.

Figures

Figures reproduced from arXiv: 2506.15483 by the authors.

Figure 1
Figure 1. GenHOI synthesizes interaction between human and unseen object conditioned on text. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. GenHOI Overview. In Stage 1, the method recovers keyframe HOI using object geometry and human pose priors. In Stage 2, the Contact-Aware Diffusion Model (ContactDM) synthesizes the 4D HOI sequence leveraging the keyframe HOI and the encoded contact embeddings. After training, the model can generalize to unseen objects given the object geometry and the associated text prompt. x h = [j, q], x o = [o, r], (1) where J i… view at source ↗
Figure 3
Figure 3. Examples of synthetic motions for qualitative evaluation. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Results of our Object-AnchorNet. Objects (a)-(e) are from 3D-FUTURE, and (f) is from [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Results of human perceptual studies. The numbers shown in the chart represent the [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

55 extracted references · 31 canonical work pages

  1. [1]

    Behave: Dataset and method for tracking human object interactions

    Bharat Lal Bhatnagar, Xianghui Xie, Ilya A Petrov, Cristian Sminchisescu, Christian Theobalt, and Gerard Pons-Moll. Behave: Dataset and method for tracking human object interactions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15935–15946, 2022

    work page 2022

  2. [2]

    Text2hoi: Text-guided 3d motion generation for hand-object interaction

    Junuk Cha, Jihyeon Kim, Jae Shin Yoon, and Seungryul Baek. Text2hoi: Text-guided 3d motion generation for hand-object interaction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1577–1585, 2024

    work page 2024

  3. [3]

    Semgeomo: Dynamic contextual hu- man motion generation with semantic and geometric guidance.arXiv preprint arXiv:2503.01291, 2025

    Peishan Cong, Ziyi Wang, Yuexin Ma, and Xiangyu Yue. Semgeomo: Dynamic contextual hu- man motion generation with semantic and geometric guidance.arXiv preprint arXiv:2503.01291, 2025

    arXiv 2025

  4. [4]

    Human-object interaction with vision-language model guided relative movement dynamics

    Zekai Deng, Ye Shi, Kaiyang Ji, Lan Xu, Shaoli Huang, and Jingya Wang. Human-object interaction with vision-language model guided relative movement dynamics. arXiv preprint arXiv:2503.18349, 2025

    arXiv 2025

  5. [5]

    Cg-hoi: Contact-guided 3d human-object interaction genera- tion

    Christian Diller and Angela Dai. Cg-hoi: Contact-guided 3d human-object interaction genera- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19888–19901, 2024

    work page 2024

  6. [6]

    Arctic: A dataset for dexterous bimanual hand-object manipulation

    Zicong Fan, Omid Taheri, Dimitrios Tzionas, Muhammed Kocabas, Manuel Kaufmann, Michael J Black, and Otmar Hilliges. Arctic: A dataset for dexterous bimanual hand-object manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12943–12954, 2023

    2023

  7. [7]

    3d-future: 3d furniture shape with texture

    Huan Fu, Rongfei Jia, Lin Gao, Mingming Gong, Binqiang Zhao, Steve Maybank, and Dacheng Tao. 3d-future: 3d furniture shape with texture. International Journal of Computer Vision, 129:3313–3337, 2021

    2021

  8. [8]

    Coohoi: Learning cooperative human-object interaction with manipulated object dynamics

    Jiawei Gao, Ziqin Wang, Zeqi Xiao, Jingbo Wang, Tai Wang, Jinkun Cao, Xiaolin Hu, Si Liu, Jifeng Dai, and Jiangmiao Pang. Coohoi: Learning cooperative human-object interaction with manipulated object dynamics. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems , ...

    work page 2024

Show all 55 references
  1. [9]

    Auto-regressive diffusion for generating 3d human-object interactions

    Zichen Geng, Zeeshan Hayder, Wei Liu, and Ajmal Saeed Mian. Auto-regressive diffusion for generating 3d human-object interactions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 3131–3139, 2025

    2025

  2. [10]

    Imos: Intent-driven full-body motion synthesis for human-object interactions

    Anindita Ghosh, Rishabh Dabral, Vladislav Golyanik, Christian Theobalt, and Philipp Slusallek. Imos: Intent-driven full-body motion synthesis for human-object interactions. In Computer Graphics Forum, volume 42, pages 1–12. Wiley Online Library, 2023

    2023

  3. [11]

    Generating diverse and natural 3d human motions from text

    Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5152–5161, 2022

    2022

  4. [12]

    Crowdmogen: Zero-shot text-driven collective motion generation

    Xinying Guo, Mingyuan Zhang, Haozhe Xie, Chenyang Gu, and Ziwei Liu. Crowdmogen: Zero-shot text-driven collective motion generation. arXiv preprint arXiv:2407.06188, 2024

    2024 arXiv

  5. [13]

    Stochastic scene-aware motion prediction

    Mohamed Hassan, Duygu Ceylan, Ruben Villegas, Jun Saito, Jimei Yang, Yi Zhou, and Michael J Black. Stochastic scene-aware motion prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11374–11384, 2021

    2021

  6. [14]

    Resolving 3d human pose ambiguities with 3d scene constraints

    Mohamed Hassan, Vasileios Choutas, Dimitrios Tzionas, and Michael J Black. Resolving 3d human pose ambiguities with 3d scene constraints. In Proceedings of the IEEE/CVF international conference on computer vision, pages 2282–2292, 2019. 11

    2019

  7. [15]

    Nemf: Neural motion fields for kinematic animation

    Chengan He, Jun Saito, James Zachary, Holly Rushmeier, and Yi Zhou. Nemf: Neural motion fields for kinematic animation. In NeurIPS, 2022

    2022

  8. [16]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  9. [17]

    Diffusion-based generation, optimization, and planning in 3d scenes

    Siyuan Huang, Zan Wang, Puhao Li, Baoxiong Jia, Tengyu Liu, Yixin Zhu, Wei Liang, and Song-Chun Zhu. Diffusion-based generation, optimization, and planning in 3d scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16750–16761, 2023

    2023

  10. [18]

    Intercap: Joint markerless 3d tracking of humans and objects in interaction

    Yinghao Huang, Omid Taheri, Michael J Black, and Dimitrios Tzionas. Intercap: Joint markerless 3d tracking of humans and objects in interaction. In DAGM German Conference on Pattern Recognition, pages 281–299. Springer, 2022

    2022

  11. [19]

    Full-body articulated human-object interaction

    Nan Jiang, Tengyu Liu, Zhexuan Cao, Jieming Cui, Zhiyuan Zhang, Yixin Chen, He Wang, Yixin Zhu, and Siyuan Huang. Full-body articulated human-object interaction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9365–9376, 2023

    2023

  12. [20]

    David: Modeling dynamic affordance of 3d objects using pre-trained video diffusion models

    Hyeonwoo Kim, Sangwon Beak, and Hanbyul Joo. David: Modeling dynamic affordance of 3d objects using pre-trained video diffusion models. arXiv preprint arXiv:2501.08333, 2025

    2025 arXiv

  13. [21]

    Zerohsi: Zero-shot 4d human-scene interaction by video generation

    Hongjie Li, Hong-Xing Yu, Jiaman Li, and Jiajun Wu. Zerohsi: Zero-shot 4d human-scene interaction by video generation. arXiv preprint arXiv:2412.18600, 2024

    2024 arXiv

  14. [22]

    Controllable human-object interaction synthesis

    Jiaman Li, Alexander Clegg, Roozbeh Mottaghi, Jiajun Wu, Xavier Puig, and C Karen Liu. Controllable human-object interaction synthesis. In European Conference on Computer Vision, pages 54–72. Springer, 2024

    2024

  15. [23]

    Object motion guided human motion synthesis

    Jiaman Li, Jiajun Wu, and C Karen Liu. Object motion guided human motion synthesis. ACM Transactions on Graphics (TOG), 42(6):1–11, 2023

    2023

  16. [24]

    Intergen: Diffusion-based multi-human motion generation under complex interactions

    Han Liang, Wenqian Zhang, Wenxuan Li, Jingyi Yu, and Lan Xu. Intergen: Diffusion-based multi-human motion generation under complex interactions. International Journal of Computer Vision, pages 1–21, 2024

    2024

  17. [25]

    Learning basketball dribbling skills using trajectory optimization and deep reinforcement learning

    Jessica Hodgins Libin Liu. Learning basketball dribbling skills using trajectory optimization and deep reinforcement learning. ACM Transactions on Graphics, 37(4), August 2018

    2018

  18. [26]

    Motion-x: A large-scale 3d expressive whole-body human motion dataset

    Jing Lin, Ailing Zeng, Shunlin Lu, Yuanhao Cai, Ruimao Zhang, Haoqian Wang, and Lei Zhang. Motion-x: A large-scale 3d expressive whole-body human motion dataset. Advances in Neural Information Processing Systems, 36:25268–25280, 2023

    2023

  19. [28]

    Himo: A new benchmark for full-body human interacting with multiple objects

    Xintao Lv, Liang Xu, Yichao Yan, Xin Jin, Congsheng Xu, Shuwen Wu, Yifan Liu, Lincheng Li, Mengxiao Bi, Wenjun Zeng, et al. Himo: A new benchmark for full-body human interacting with multiple objects. In European Conference on Computer Vision, pages 300–318. Springer, 2024

    2024

  20. [29]

    Amass: Archive of motion capture as surface shapes

    Naureen Mahmood, Nima Ghorbani, Nikolaus F Troje, Gerard Pons-Moll, and Michael J Black. Amass: Archive of motion capture as surface shapes. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5442–5451, 2019

    2019

  21. [30]

    Expressive body capture: 3d hands, face, and body from a single image

    Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed AA Osman, Dimitrios Tzionas, and Michael J Black. Expressive body capture: 3d hands, face, and body from a single image. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognitio...

    2019

  22. [31]

    Hoi-diff: Text-driven synthesis of 3d human-object interactions using diffusion models

    Xiaogang Peng, Yiming Xie, Zizhao Wu, Varun Jampani, Deqing Sun, and Huaizu Jiang. Hoi-diff: Text-driven synthesis of 3d human-object interactions using diffusion models. arXiv preprint arXiv:2312.06553, 2023. 12

    2023 arXiv

  23. [32]

    Deepmimic: Example- guided deep reinforcement learning of physics-based character skills

    Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne. Deepmimic: Example- guided deep reinforcement learning of physics-based character skills. ACM Trans. Graph., 37(4):143:1–143:14, July 2018

    2018

  24. [33]

    Amp: Adversarial motion priors for stylized physics-based character control

    Xue Bin Peng, Ze Ma, Pieter Abbeel, Sergey Levine, and Angjoo Kanazawa. Amp: Adversarial motion priors for stylized physics-based character control. ACM Transactions on Graphics (ToG), 40(4):1–20, 2021

    2021

  25. [34]

    Ilya A Petrov, Riccardo Marin, Julian Chibane, and Gerard Pons-Moll. Object pop-up: Can we infer 3d objects and their poses from human interactions alone? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4726–4736, 2023

    2023

  26. [35]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space

    Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017

    2017

  27. [36]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pa...

    2021

  28. [37]

    Hoianimator: Generating text-prompt human-object animations using novel perceptive diffusion models

    Wenfeng Song, Xinyu Zhang, Shuai Li, Yang Gao, Aimin Hao, Xia Hou, Chenglizhao Chen, Ning Li, and Hong Qin. Hoianimator: Generating text-prompt human-object animations using novel perceptive diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat...

    2024

  29. [38]

    A survey on human interaction motion generation

    Kewei Sui, Anindita Ghosh, Inwoo Hwang, Bing Zhou, Jian Wang, and Chuan Guo. A survey on human interaction motion generation. arXiv preprint arXiv:2503.12763, 2025

    2025

  30. [39]

    Grab: A dataset of whole-body human grasping of objects

    Omid Taheri, Nima Ghorbani, Michael J Black, and Dimitrios Tzionas. Grab: A dataset of whole-body human grasping of objects. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IV 16 , pages 581–600. Springer, 2020

    2020

  31. [40]

    Human motion diffusion model

    Guy Tevet, Sigal Raab, Brian Gordon, Yoni Shafir, Daniel Cohen-or, and Amit Haim Bermano. Human motion diffusion model. In The Eleventh International Conference on Learning Repre- sentations, 2023

    2023

  32. [41]

    Physhoi: Physics-based imitation of dynamic human-object interaction

    Yinhuai Wang, Jing Lin, Ailing Zeng, Zhengyi Luo, Jian Zhang, and Lei Zhang. Physhoi: Physics-based imitation of dynamic human-object interaction. arXiv preprint arXiv:2312.04393, 2023

    2023 arXiv

  33. [42]

    Move as you say interact as you can: Language-guided human motion generation with scene affordance

    Zan Wang, Yixin Chen, Baoxiong Jia, Puhao Li, Jinlu Zhang, Jingze Zhang, Tengyu Liu, Yixin Zhu, Wei Liang, and Siyuan Huang. Move as you say interact as you can: Language-guided human motion generation with scene affordance. In Proceedings of the IEEE/CVF Conference on Compute...

    2024

  34. [43]

    Humanise: Language-conditioned human motion generation in 3d scenes

    Zan Wang, Yixin Chen, Tengyu Liu, Yixin Zhu, Wei Liang, and Siyuan Huang. Humanise: Language-conditioned human motion generation in 3d scenes. Advances in Neural Information Processing Systems, 35:14959–14971, 2022

    2022

  35. [44]

    Reconstructing in-the-wild open-vocabulary human-object interactions

    Boran Wen, Dingbang Huang, Zichen Zhang, Jiahong Zhou, Jianbin Deng, Jingyu Gong, Yulong Chen, Lizhuang Ma, and Yong-Lu Li. Reconstructing in-the-wild open-vocabulary human-object interactions. arXiv preprint arXiv:2503.15898, 2025

    2025 arXiv

  36. [45]

    Thor: Text to human-object interaction diffusion via relation intervention

    Qianyang Wu, Ye Shi, Xiaoshui Huang, Jingyi Yu, Lan Xu, and Jingya Wang. Thor: Text to human-object interaction diffusion via relation intervention. arXiv preprint arXiv:2403.11208, 2024

    2024 arXiv

  37. [46]

    Human-object interaction from human-level instructions

    Zhen Wu, Jiaman Li, and C Karen Liu. Human-object interaction from human-level instructions. arXiv preprint arXiv:2406.17840, 2024. 13

    2024 arXiv

  38. [47]

    Inter-x: Towards versatile human-human interac- tion analysis

    Liang Xu, Xintao Lv, Yichao Yan, Xin Jin, Shuwen Wu, Congsheng Xu, Yifan Liu, Yizhou Zhou, Fengyun Rao, Xingdong Sheng, et al. Inter-x: Towards versatile human-human interac- tion analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p...

    2024

  39. [48]

    Interdiff: Generating 3d human-object interactions with physics-informed diffusion

    Sirui Xu, Zhengyuan Li, Yu-Xiong Wang, and Liang-Yan Gui. Interdiff: Generating 3d human-object interactions with physics-informed diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14928–14940, 2023

    2023

  40. [49]

    Intermimic: Towards universal whole-body control for physics-based human-object interactions

    Sirui Xu, Hung Yu Ling, Yu-Xiong Wang, and Liang-Yan Gui. Intermimic: Towards universal whole-body control for physics-based human-object interactions. arXiv preprint arXiv:2502.20390, 2025

    2025

  41. [50]

    Interdreamer: Zero-shot text to 3d dynamic human-object interaction

    Sirui Xu, Ziyin Wang, Yu-Xiong Wang, and Liang-Yan Gui. Interdreamer: Zero-shot text to 3d dynamic human-object interaction. arXiv preprint arXiv:2403.19652, 2024

    2024

  42. [51]

    F-hoi: Toward fine- grained semantic-aligned 3d human-object interactions, 2024

    Jie Yang, Xuesong Niu, Nan Jiang, Ruimao Zhang, and Siyuan Huang. F-hoi: Toward fine- grained semantic-aligned 3d human-object interactions, 2024

    2024

  43. [52]

    Generating human interaction motions in scenes with text control

    Hongwei Yi, Justus Thies, Michael J Black, Xue Bin Peng, and Davis Rempe. Generating human interaction motions in scenes with text control. In European Conference on Computer Vision, pages 246–263. Springer, 2024

    2024

  44. [53]

    Chainhoi: Joint-based kinematic chain modeling for human-object interaction generation

    Ling-An Zeng, Guohong Huang, Yi-Lin Wei, Shengbo Gu, Yu-Ming Tang, Jingke Meng, and Wei-Shi Zheng. Chainhoi: Joint-based kinematic chain modeling for human-object interaction generation. arXiv preprint arXiv:2503.13130, 2025

    2025 arXiv

  45. [54]

    Core4d: A 4d human- object-human interaction dataset for collaborative object rearrangement

    Chengwen Zhang, Yun Liu, Ruofan Xing, Bingda Tang, and Li Yi. Core4d: A 4d human- object-human interaction dataset for collaborative object rearrangement. arXiv preprint arXiv:2406.19353, 2024

    2024 arXiv

  46. [55]

    Couch: Towards controllable human-chair interactions

    Xiaohan Zhang, Bharat Lal Bhatnagar, Sebastian Starke, Vladimir Guzov, and Gerard Pons- Moll. Couch: Towards controllable human-chair interactions. In European Conference on Computer Vision, pages 518–535. Springer, 2022

    2022

  47. [56]

    On the continuity of rotation representations in neural networks

    Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, and Hao Li. On the continuity of rotation representations in neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5745–5753, 2019. 14 A Technical Appendices and Supplement...

    2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.