SimGenHOI: Physically Realistic Whole-Body Humanoid-Object Interaction via Generative Modeling and Reinforcement Learning

Jiahong Xie; Jiajun Lv; Ruoyu Wang; Xingxing Zuo; Yijia Xie; Yuehao Huang; Yuhang Lin; Yukai Ma

REVIEW 4 major objections 5 minor 50 references

SimGenHOI: Physically Realistic Whole-Body Humanoid-Object Interaction via Generative Modeling and Reinforcement Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By generating humanoid-object interactions as sparse key actions and tracking them with a contact-aware reinforcement-learning policy, SimGenHOI produces motions that are controllable, physically executable in simulation, and more robust…

desk verdict A well-assembled pipeline for whole-body humanoid-object interaction that deserves peer review, but the mutual fine-tuning loop is a real co-adaptation risk and the reported numbers are modest. read the letter →

arxiv 2508.14120 v1 pith:3MH3A2N5 submitted 2025-08-18 cs.RO cs.AI

Yuhang Lin , Yijia Xie , Jiahong Xie , Yuehao Huang , Ruoyu Wang , Jiajun Lv , Yukai Ma , Xingxing Zuo This is my paper · ORCID

classification cs.ROcs.AI

keywords humanoid-objectinteractionmotiongenerationdiffusionmodelreinforcementlearningwhole-bodycontrolcontact-awarepolicykeyactionextractionlong-horizonmanipulation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SimGenHOI sets out to resolve the usual trade-off in humanoid-object interaction synthesis: generative models produce diverse and controllable motion but with artifacts such as hand penetration and foot sliding, while physics-based policies are physically consistent but narrow in behavior. The paper argues that both goals can be met by splitting the problem: a diffusion model generates a sparse set of key actions—critical poses of the hands, feet, and body—conditioned on text, object geometry, sparse waypoints, and the starting pose, and a contact-aware whole-body reinforcement-learning policy then tracks those actions inside a physics simulator, correcting penetration and sliding. A mutual fine-tuning loop closes the gap between the two: successful simulated executions are fed back to refine the generator, and the improved generator's motions are used to further train the policy. If the claim holds, users could specify a long-horizon manipulation in text and get a motion that a simulated humanoid can actually execute, not just a video-like pose sequence. The reported experiments show higher tracking success and reduced contact artifacts compared with adapted prior baselines.

What carries the argument

The load-bearing mechanism has three coupled parts. First, key-action extraction: from a dense motion sequence, the algorithm recursively inserts the frame with the largest weighted reconstruction error as a new key frame, using per-joint weights that emphasize hands and feet, until every interpolated subsegment falls below an error threshold; this converts dense clips into sparse, interpolatable control points. Second, contact guidance: the diffusion model also outputs a contact probability for hands and feet, and this signal is given to the policy as a reference cue with a reward term that encourages predicted and actual contact to match, so the policy knows when contact should occur and can be rewarded for it. Third, mutual fine-tuning: successful policy tracks from the simulator are collected as training samples for the generator, and newly generated motions are used to refine the policy, aligning the distribution of generated reference motions with motions the policy can reliably execute. The whole loop is what carries the claim that generated interaction is physically plausible.

What would settle it

Run the trained pipeline, without further training, in a different physics engine or on a real humanoid, and compare tracking success, foot sliding, and hand penetration; if the strong metrics collapse, the gains come from simulator-specific alignment rather than physical realism. Alternatively, hold out motion-capture sequences from fine-tuning and test whether policy success improves on those after the mutual loop; if only generator-sampled motions improve, the loop has not improved physical plausibility.

Watch

Extended reading notes

Core claim

The paper's claim is that physical executability and generative diversity are not opposed if generation is organized around key actions rather than dense frames. Its HOI generator, a Transformer-based diffusion model, predicts a short sequence of key actions together with object poses and hand/feet contact probabilities; intervening frames are recovered by interpolation. These reference trajectories, including a contact-guidance signal that tells the policy when to grasp and release, are handed to a whole-body policy trained by reinforcement learning in simulation. The policy tracks key joints and object pose while the contact guidance lets it discover grasping behavior through exploration. The authors further show that iteratively fine-tuning the generator on motions the policy successfully tracked, and fine-tuning the policy on the improved generator's outputs, raises tracking success. In their experiments on a human-object manipulation motion-capture dataset, the full system reaches contact and target success on all seven tested reference manipulations while reducing hand penetration and foot sliding relative to direct generation and to tracking baselines.

Load-bearing premise

The key assumption is that the motions the policy successfully tracks in the simulator are actually physically plausible, because the same simulator is used to train the policy and to evaluate the whole system, so the mutual fine-tuning loop is never validated against an independent source of physical ground truth.

Editorial extensions

If this is right

Text- and waypoint-conditioned long-horizon manipulation (lift, move, put down, push, and return an object) becomes a single pipeline: generate sparse key actions autoregressively, interpolate, track with the policy.
Sparse key actions reduce the number of denoising passes needed for long sequences, because each key action covers a longer temporal span than a dense frame.
The contact-guidance signal lets the policy learn grasping and releasing timing by exploration instead of relying on detailed reference hand poses, widening the range of manipulable objects.
The mutual fine-tuning loop improves tracking success; in the ablation, removing it lowers success from 41.67% to 37.50%, so the loop is doing real work rather than being decorative.
Passing generated motion through the physics simulator substantially improves physical metrics such as foot height, foot sliding, and hand penetration, suggesting that physics filtering alone removes many kinematic artifacts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension follows from the weakest premise: if the mutual fine-tuning loop is genuinely improving physical realism, tracking success on held-out motion-capture sequences never used in fine-tuning should improve too; if only self-generated motions become easier to track, the loop has aligned the generator to the policy rather than to physics.
The key-action extraction threshold and per-joint weights form a fidelity/complexity dial. One could adapt them per task—allocating more key frames to hand and foot phases during a grasp, fewer during transportation—to trade generation cost against policy trackability.
Because the policy is trained entirely in simulation, transferring this pipeline to a real humanoid would likely require domain randomization and system identification; the paper's contact-guidance reward structure gives a natural place to add force and torque feedback.
The interpolation between key actions effectively imposes a piecewise-linear prior on motion, and that prior likely contributes to the reported reduction in foot sliding; this could be isolated by comparing key-action interpolation against dense prediction with the same policy.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Dear [colleague],

You asked about SimGenHOI. The short version: it's a solid systems paper with a new combination of ideas, but the central claim of 'physically realistic' is undercut by a closed-loop fine-tuning procedure with no independent check, and the success rates are not as high as the abstract suggests.

What's new: they adapt sparse key-action extraction (from AWE) to humanoid-object interaction, generate key actions with a DiT diffusion model conditioned on text, object geometry, and waypoints, then track them with a contact-aware RL policy. The mutual fine-tuning loop—alternating between fine-tuning the generator on policy-tracked motions and fine-tuning the policy on generated motions—is the most novel piece.

What's good: the paper runs a comprehensive comparison against several baselines (InterDiff, MDM, OMOMO, CHOIS) including a tracking stage, and includes an ablation isolating key actions, contact guidance, and fine-tuning. Physical metrics like foot sliding and penetration improve substantially after tracking, which is real evidence the policy corrects artifacts. The writing is clear and the method is specified well enough to re-implement.

Where it's soft: the stress-test note is on target. The fine-tuning loop collects 'physically plausible' actions from policy rollouts in Isaac Gym, trains the generator on them, then retrains the policy on the generator's outputs. The success criterion (Succ_tgt within 0.5 m, interaction early termination) is defined in that same simulator. There's no independent physical anchor, so the fine-tuning gain (37.5% vs 41.7% success) could be distribution narrowing—the generator learning motions that are easy for this particular policy—rather than improved realism. The paper doesn't provide diversity or coverage metrics to rule this out. Also, all numbers are single-seed with no error bars, and the 7 reference motions in Table 3 are manually selected. The baseline comparisons may be slightly unfair: CHOIS and OMOMO need GRIP-refined hand grasps and are tracked by a different policy, while SimGen-HOI skips hand details. That doesn't invalidate the paper, but it makes 'significantly higher' hard to judge.

Net: this is a reasonable contribution to humanoid-object interaction generation and control. It's not a breakthrough, but a useful recipe. The right audience works on physics-based character control and HOI synthesis. I'd send it to peer review, but only with a strong request for code, multiple seeds, and an analysis that addresses the co-adaptation risk—a held-out policy, a second simulator, or an in-the-loop diversity measure.

Referee Report

4 major / 5 minor

Summary. SimGenHOI proposes a unified framework that couples a diffusion-transformer-based generative model, which outputs sparse "key actions" for humanoid-object interaction (HOI), with a contact-aware whole-body reinforcement-learning policy that tracks these motions in the Isaac Gym simulator. The key-action representation is interpolated into dense trajectories, and an alternating "mutual fine-tuning" procedure refines the generator on successful policy rollouts and then fine-tunes the policy on the improved generator outputs. The paper reports generation-quality metrics (e.g., MPJPE, foot sliding, hand penetration) that outperform several adapted baselines on the FullBodyManipulation dataset, and tracking results on seven reference motions. An ablation study examines the contributions of key actions, contact guidance, and fine-tuning. The authors claim that the framework generates realistic, diverse, and physically plausible long-horizon HOI and achieves significantly higher tracking success in simulation.

Significance. If the empirical claims hold, the framework would be a useful step toward integrating generative motion synthesis with physics-based control for humanoid-object interaction, particularly its key-action abstraction for long-horizon generation and its effort to close the loop between generation and control. The paper includes quantitative comparisons with several baselines and an ablation study, and it would release code. However, the validation is largely internal to the same simulator used for training, and the mutual fine-tuning loop introduces a circularity that weakens the physical-realism claim. The paper also lacks error bars, diversity metrics, and a well-defined long-horizon evaluation, which limits the strength of its central claims.

major comments (4)

[Methods, 'HOI In Physics Simulation' > 'Fine-Tuning'; Table 4] The mutual fine-tuning loop is circular with respect to the claim of improved physical realism. The generator is fine-tuned on actions successfully tracked by the same contact-aware policy, and the policy is fine-tuned on the generator's outputs; the success criterion (Succ_tgt, object within 0.5 m, and interaction early termination) is defined by that same policy and evaluated in the same Isaac Gym simulator used for training. The reported gain from fine-tuning in Table 4 is modest (41.67% vs. 37.50%), and without a diversity or coverage analysis to show that the generator does not simply narrow its output distribution to motions the current policy finds easy, the improvement could reflect distribution alignment rather than an increase in independent physical realism. I recommend an external validation: evaluate the same generated motions with a different tracking policy or in a different physics engine, and report the variance of generated joint/object trajectories to rule out distribution collapse.
[Tables 1-4; Experiments] The central claim of "significantly higher tracking success" is not supported by inferential statistics. All tables report single point estimates with no error bars, confidence intervals, or multiple seeds. For instance, in Table 4 the difference between the full system (41.67%) and "w.o. finetune" (37.50%) corresponds to about 6 of 144 sequences, and the difference with "w.o. key action" (34.72%) is about 10 sequences, which may well be within run-to-run noise. Please provide means and standard deviations over at least three random seeds for the policy training and, where feasible, for the diffusion model, and apply an appropriate significance test to the comparisons that underpin the headline claims.
[Experiments, 'Contact-Aware Interaction Tracking' and Table 3] The tracking evaluation is based on only 7 manually selected reference motions, which invites selection bias and limits the generality of the tracking comparison. Moreover, Table 3 does not clearly support the claim of superior tracking: OmniGrasp on OMOMO achieves the same 7/7 success in Succ_cont and Succ_tgt and has lower Eobj_pos (40.84 vs. 64.92) and lower Eobj_rot (0.13 vs. 0.72), while SimGen-HOI shows better Eobj_acc and Eobj_vel. The paper should either use a larger and pre-specified test set, report the criteria for selecting the 7 motions, and discuss the mixed results honestly, or temper the claim that the method outperforms prior approaches in tracking.
[Abstract and Conclusion] The paper claims that the framework "enables long-horizon manipulation tasks" and generates "diverse" humanoid-object interactions, but no experiment quantifies long-horizon performance (e.g., number of autoregressive steps, success over extended sequences, comparison of windowed vs. one-shot generation) or diversity (e.g., pairwise distances between samples generated from the same conditions). These are load-bearing claims in the abstract and conclusion, and they need dedicated evaluation. Without such results, the long-horizon and diversity claims are unsupported.

minor comments (5)

['Reward Design'] The text says "Please see the appendix E for details of the key joints," but the manuscript contains no appendix. The list of key joints is necessary to interpret Eq. (8), which states that the tracking reward is applied only to selected key joints. Please either include the appendix or specify the joints in the main text.
[Methods, 'Key Actions Extract'] The key-action extraction depends on a manually defined importance weight vector w and a stopping threshold; the paper does not state how these are chosen or whether the results are sensitive to them. A brief sensitivity analysis (e.g., varying the threshold or weights and reporting the effect on generation metrics) would strengthen the reproducibility of the approach.
[Table 2] The title of Table 2 reads "Evaluation of motions tracked by our control policy," but the rows include OMOMO GT, CHOIS, and OURS200, which appear to be generation-only or baseline inputs. The caption and text should clarify which pipeline (generation only, generation plus tracking, or baseline plus tracking) is used for each row, and the metrics should be explicitly described as computed in the Isaac Gym simulator.
[Table 3] The baseline comparison in Table 3 uses GRIP to add hand details to OMOMO and CHOIS before tracking with PHC-X, while the proposed method assigns default hand joint angles and lets the policy explore contact. This asymmetry should be discussed as a deliberate design choice, and its potential to favor the proposed method (since GRIP may not be well suited to large objects or to integration with PHC-X) should be acknowledged.
[Throughout] There are several typographical and notation issues: "seqequence" in the Methods section, the meaning of "Lin-OMOMO" versus "Pred-OMOMO" is not explained, and the formatting of C_F1, C%, and related metrics is inconsistent. A careful proofread would improve readability.

Circularity Check

1 steps flagged · score 6.0 of 10

Mutual fine-tuning closes the loop: the generator is fitted on the same policy's successful rollouts, so the reported tracking-success gain is distribution alignment, not independent physical validation.

fitted input called prediction [Methods, 'Fine-Tuning' under 'HOI In Physics Simulation'; 'Early Termination'; 'Evaluation Metrics'; Table 4]
"we evaluate the policy within a physics-aware simulator to track the generated motions and collect physically plausible actions that adhere to contact constraints. These executable and physically-plausible actions serve as high-quality samples for fine-tuning the diffusion-based generative model... Conversely, the tracking policy is further fine-tuned with the motion generated from the improved generative model."

The 'physically plausible actions' used to fine-tune the generator are defined by the same policy's tracking criteria: Interaction Early Termination triggers when object keypoint deviation exceeds 0.5 m or contact is absent for 10 frames, and the reported success metric Succ_tgt is likewise 'object has reached its final position within a distance of 0.5 meters.' Both are evaluated in Isaac Gym, the same simulator used for fine-tuning. The generator is therefore fitted to rollouts that this particular policy can already track; the reported 41.67% vs 37.50% success increase (Table 4) can be produced by the generator narrowing its output to the policy's easy-to-track distribution rather than by any independent increase in physical realism.

full rationale

The core system has substantial independent content: the diffusion model is trained on the FullBodyManipulation mocap dataset, the policy uses a pretrained motion representation (PULSE) and standard RL rewards, and the generation metrics in Table 1 are compared against external baselines and ground truth. However, the paper's mutual fine-tuning contribution is a closed co-adaptation loop. The generator is fine-tuned on rollouts that the same contact-aware policy successfully tracked, and the reported success metric Succ_tgt is defined by the same 0.5 m object-distance criterion used in that policy's Interaction Early Termination, all inside Isaac Gym, which is also the training simulator. The fine-tuning improvement in Table 4 (37.50% to 41.67%) is therefore a statistically forced consequence of distribution matching rather than an independent confirmation of physical realism. Since only this fine-tuning claim reduces by construction while the rest of the framework is externally anchored, the circularity is partial rather than total.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The load-bearing ingredients are the simulator, the SMPL-X avatar, the PULSE motion prior, the BPS object representation, the key-action/interpolation scheme, and predicted contact probabilities. These are all either standard tools or modeling choices; no new physical entities are introduced. The 'key action' is a methodological abstraction, not an invented physical entity.

free parameters (4)

Critical joint importance weights w in key action extraction = not reported (manually defined)
Eq. (5) weights joints by w when computing reconstruction error; this determines which frames become key actions and shapes the generated motion distribution.
Key action stopping threshold = not reported
The recursive key action extraction stops when reconstruction error falls below a predefined threshold; this controls the number of key actions and hence interpolation fidelity.
Policy reward weights = not fully reported
Rewards in Eqs. (8), (9), and (10) use hand-tuned weights (w_jp, w_jr, w_jv, w_jomega, w_jc, w_op, w_or, w_ov, w_omega, alpha) that balance human tracking, object tracking, and contact; these choices directly affect tracking success.
Interaction early termination thresholds = 0.5 m and 10 frames
Early termination is triggered when object keypoint deviation exceeds 0.5 m or contact is absent for 10 frames; these hand-chosen thresholds affect the success rate measurement.

assumptions (6)

domain assumption Isaac Gym simulation accurately models rigid-body dynamics and contact for humanoid-object interaction.
All physical realism and tracking success claims are evaluated in Isaac Gym with no real-world validation.
domain assumption SMPL-X with articulated hands is a sufficient humanoid avatar for whole-body interaction.
The policy and object interaction are built around SMPL-X; no comparison on other body models or hardware is provided.
domain assumption PULSE, pre-trained on AMASS, provides a motion prior that transfers to humanoid-object interaction.
The policy encoder uses PULSE; the paper does not ablate this choice.
domain assumption Basis Point Set with a 1-meter radius sphere captures task-relevant object geometry.
Object conditioning uses BPS following OMOMO; sensitivity to the representation is not tested.
domain assumption The diffusion model's predicted contact probability is a reliable signal for contact guidance.
Predicted contact probabilities feed the policy's observations and rewards; noisy predictions would degrade tracking and early termination.
domain assumption Linear interpolation between extracted key actions reconstructs trajectories well enough for policy tracking.
The method replaces dense generation with sparse key actions plus linear interpolation; the paper provides qualitative reconstruction evidence but no formal bound.

how reviews work

0 comments

Cite this review

Pith. "Pith review of SimGenHOI: Physically Realistic Whole-Body Humanoid-Object Interaction via Generative Modeling and Reinforcement Learning." pith.science (2026). https://pith.science/paper/3MH3A2N5

@misc{pith2026250814120,
  author       = {Pith},
  title        = {Pith review of: SimGenHOI: Physically Realistic Whole-Body Humanoid-Object Interaction via Generative Modeling and Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3MH3A2N5}},
  note         = {Machine review of arXiv:2508.14120}
}

read the original abstract

Generating physically realistic humanoid-object interactions (HOI) is a fundamental challenge in robotics. Existing HOI generation approaches, such as diffusion-based models, often suffer from artifacts such as implausible contacts, penetrations, and unrealistic whole-body actions, which hinder successful execution in physical environments. To address these challenges, we introduce SimGenHOI, a unified framework that combines the strengths of generative modeling and reinforcement learning to produce controllable and physically plausible HOI. Our HOI generative model, based on Diffusion Transformers (DiT), predicts a set of key actions conditioned on text prompts, object geometry, sparse object waypoints, and the initial humanoid pose. These key actions capture essential interaction dynamics and are interpolated into smooth motion trajectories, naturally supporting long-horizon generation. To ensure physical realism, we design a contact-aware whole-body control policy trained with reinforcement learning, which tracks the generated motions while correcting artifacts such as penetration and foot sliding. Furthermore, we introduce a mutual fine-tuning strategy, where the generative model and the control policy iteratively refine each other, improving both motion realism and tracking robustness. Extensive experiments demonstrate that SimGenHOI generates realistic, diverse, and physically plausible humanoid-object interactions, achieving significantly higher tracking success rates in simulation and enabling long-horizon manipulation tasks. Code will be released upon acceptance on our project page: https://xingxingzuo.github.io/simgen_hoi.

Figures

Figures reproduced from arXiv: 2508.14120 by the authors.

**Figure 2.** Our proposed framework uses a diffusion model for key action generation and reinforcement learning to train a contact-aware HOI policy in the Isaac Gym simulator. The proposed framework, as depicted in (c), mainly consists of two components, including: (a) HOI generation conditioned on text prompt, object geometry, sparse waypoints of objects, and initial humanoid pose. (b) Contact-aware humanoid-object interaction … view at source ↗

**Figure 3.** Reconstruction Error of humanoid actions for key action extraction. where µθ denotes the predicted mean and Σn is a fixed variance. Following prior work (Ho, Jain, and Abbeel 2020), we re-parameterize the model to predict the original clean data τ0 from the noisy input τn. The model is trained with the objective: L = Eτ0,n||τˆθ(xn, n, C) − τ0||1. (4) where τˆθ represents the final prediction of the model. The condi… view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Qualitative results of combining with simulation. The detailed hand-object interaction is zoomed in the bottom-right corner, while the top-right corner visualizes the OMOMO dataset (Li, Wu, and Liu 2023) grasping motion refined by GRIP (Taheri et al. 2024). representa…

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 32 canonical work pages

[1]

Bae, J.; Won, J.; Lim, D.; Min, C.-H.; and Kim, Y. M. 2023. Pmp: Learning to physically interact with environments using part-wise motion priors. In ACM SIGGRAPH 2023 Conference Proceedings, 1--10

work page 2023
[2]

Braun, J.; Christen, S.; Kocabas, M.; Aksan, E.; and Hilliges, O. 2024. Physically plausible full-body hand-object interaction synthesis. In 2024 International Conference on 3D Vision (3DV), 464--473. IEEE

work page 2024
[3]

Cen, Z.; Pi, H.; Peng, S.; Shen, Z.; Yang, M.; Zhu, S.; Bao, H.; and Zhou, X. 2024. Generating human motion in 3D scenes from text descriptions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1855--1866

work page 2024
[4]

Chen, R.; Shi, M.; Huang, S.; Tan, P.; Komura, T.; and Chen, X. 2024. Taming diffusion probabilistic models for character control. In ACM SIGGRAPH 2024 Conference Papers, 1--10

work page 2024
[5]

Chentanez, N.; M \"u ller, M.; Macklin, M.; Makoviychuk, V.; and Jeschke, S. 2018. Physics-based motion capture imitation with deep reinforcement learning. In Proceedings of the 11th ACM SIGGRAPH Conference on Motion, Interaction and Games, 1--10

work page 2018
[6]

H.; Golyanik, V.; and Theobalt, C

Dabral, R.; Mughal, M. H.; Golyanik, V.; and Theobalt, C. 2023. Mofusion: A framework for denoising-diffusion-based motion synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 9760--9770

work page 2023
[7]

Guo, C.; Zou, S.; Zuo, X.; Wang, S.; Ji, W.; Li, X.; and Cheng, L. 2022. Generating Diverse and Natural 3D Human Motions From Text. In CVPR

work page 2022
[8]

He, C.; Saito, J.; Zachary, J.; Rushmeier, H.; and Zhou, Y. 2022. Nemf: Neural motion fields for kinematic animation. NeurIPS

work page 2022

Show all 50 references

[9]

He, T.; Luo, Z.; He, X.; Xiao, W.; Zhang, C.; Zhang, W.; Kitani, K.; Liu, C.; and Shi, G. 2024. Omnih2o: Universal and dexterous human-to-humanoid whole-body teleoperation and learning. arXiv preprint arXiv:2406.08858

2024 arXiv
[10]

Ho, J.; Jain, A.; and Abbeel, P. 2020. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33: 6840--6851

2020
[11]

Huang, S.; Wang, Z.; Li, P.; Jia, B.; Liu, T.; Zhu, Y.; Liang, W.; and Zhu, S.-C. 2023. Diffusion-based generation, optimization, and planning in 3d scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 16750--16761

2023
[12]

Jiang, N.; He, Z.; Wang, Z.; Li, H.; Chen, Y.; Huang, S.; and Zhu, Y. 2024 a . Autonomous character-scene interaction synthesis from text instruction. In SIGGRAPH Asia 2024 Conference Papers, 1--11

2024
[13]

Jiang, N.; Zhang, Z.; Li, H.; Ma, X.; Wang, Z.; Chen, Y.; Liu, T.; Zhu, Y.; and Huang, S. 2024 b . Scaling up dynamic human-scene interaction modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1737--1747

2024
[14]

Kulkarni, N.; Rempe, D.; Genova, K.; Kundu, A.; Johnson, J.; Fouhey, D.; and Guibas, L. 2024. Nifty: Neural object interaction fields for guided human motion synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 947--957

2024
[15]

Li, J.; Clegg, A.; Mottaghi, R.; Wu, J.; Puig, X.; and Liu, C. K. 2024 a . Controllable human-object interaction synthesis. In ECCV

2024
[16]

Li, J.; Clegg, A.; Mottaghi, R.; Wu, J.; Puig, X.; and Liu, C. K. 2024 b . Controllable human-object interaction synthesis. In European Conference on Computer Vision, 54--72. Springer

2024
[17]

Li, J.; Wu, J.; and Liu, C. K. 2023. Object motion guided human motion synthesis. ACM Transactions on Graphics (TOG), 42(6): 1--11

2023
[18]

Liu, L.; and Hodgins, J. 2018. Learning basketball dribbling skills using trajectory optimization and deep reinforcement learning. Acm transactions on graphics (tog), 37(4): 1--14

2018
[19]

Luo, Z.; Cao, J.; Christen, S.; Winkler, A.; Kitani, K.; and Xu, W. 2024. Omnigrasp: Grasping diverse objects with simulated humanoids. Advances in Neural Information Processing Systems, 37: 2161--2184

2024
[20]

Luo, Z.; Cao, J.; Merel, J.; Winkler, A.; Huang, J.; Kitani, K.; and Xu, W. 2023 a . Universal humanoid motion representations for physics-based control. arXiv preprint arXiv:2310.04582

2023 arXiv
[21]

W.; Kitani, K.; and Xu, W

Luo, Z.; Cao, J.; Winkler, A. W.; Kitani, K.; and Xu, W. 2023 b . Perpetual Humanoid Control for Real-time Simulated Avatars. In International Conference on Computer Vision (ICCV)

2023
[22]

Luo, Z.; Yuan, Y.; and Kitani, K. M. 2022. From universal humanoid control to automatic physically valid character creation. arXiv preprint arXiv:2206.09286

2022 arXiv
[23]

F.; Pons-Moll, G.; and Black, M

Mahmood, N.; Ghorbani, N.; Troje, N. F.; Pons-Moll, G.; and Black, M. J. 2019. AMASS : Archive of motion capture as surface shapes. In ICCV

2019
[24]

Merel, J.; Tunyasuvunakool, S.; Ahuja, A.; Tassa, Y.; Hasenclever, L.; Pham, V.; Erez, T.; Wayne, G.; and Heess, N. 2020. Catch & carry: reusable neural controllers for vision-guided whole-body tasks. ACM Transactions on Graphics (TOG), 39(4): 39--1

2020
[25]

Pavlakos, G.; Choutas, V.; Ghorbani, N.; Bolkart, T.; Osman, A. A. A.; Tzionas, D.; and Black, M. J. 2019. Expressive Body Capture: 3 D Hands, Face, and Body From a Single Image. In CVPR

2019
[26]

Peebles, W.; and Xie, S. 2022. Scalable Diffusion Models with Transformers. International Conference on Computer Vision

2022
[27]

Peng, X.; Xie, Y.; Wu, Z.; Jampani, V.; Sun, D.; and Jiang, H. 2023. Hoi-diff: Text-driven synthesis of 3d human-object interactions using diffusion models. arXiv preprint arXiv:2312.06553

2023 arXiv
[28]

B.; Abbeel, P.; Levine, S.; and Van de Panne, M

Peng, X. B.; Abbeel, P.; Levine, S.; and Van de Panne, M. 2018. Deepmimic: Example-guided deep reinforcement learning of physics-based character skills. ACM Transactions On Graphics (TOG), 37(4): 1--14

2018
[29]

Prokudin, S.; Lassner, C.; and Romero, J. 2019. Efficient learning on point clouds with basis point sets. In ICCV

2019
[30]

R.; Chandrasekaran, A.; Athanasiou, N.; Quiros-Ramirez, A.; and Black, M

Punnakkal, A. R.; Chandrasekaran, A.; Athanasiou, N.; Quiros-Ramirez, A.; and Black, M. J. 2021. BABEL : Bodies, Action and Behavior with English Labels. In CVPR

2021
[31]

H.; and Cohen-Or, D

Raab, S.; Gat, I.; Sala, N.; Tevet, G.; Shalev-Arkushin, R.; Fried, O.; Bermano, A. H.; and Cohen-Or, D. 2024. Monkey see, monkey do: Harnessing self-attention in motion diffusion for zero-shot motion transfer. In SIGGRAPH Asia 2024 Conference Papers, 1--13

2024
[32]

H.; and Cohen-Or, D

Raab, S.; Leibovitch, I.; Tevet, G.; Arar, M.; Bermano, A. H.; and Cohen-Or, D. 2023. Single motion diffusion. arXiv preprint arXiv:2302.05905

2023 arXiv
[33]

W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In ICML

2021
[34]

Romero, J.; Tzionas, D.; and Black, M. J. 2022. Embodied hands: Modeling and capturing hands and bodies together. arXiv preprint arXiv:2201.02610

2022 arXiv
[35]

X.; Sharma, A.; Zhao, T

Shi, L. X.; Sharma, A.; Zhao, T. Z.; and Finn, C. 2023. Waypoint-based imitation learning for robotic manipulation. arXiv preprint arXiv:2307.14326

2023 arXiv
[36]

Shi, Y.; Wang, J.; Jiang, X.; Lin, B.; Dai, B.; and Peng, X. B. 2024. Interactive character control with auto-regressive motion diffusion models. ACM Transactions on Graphics (TOG), 43(4): 1--14

2024
[37]

Taheri, O.; Zhou, Y.; Tzionas, D.; Zhou, Y.; Ceylan, D.; Pirk, S.; and Black, M. J. 2024. Grip: Generating interaction poses using spatial cues and latent consistency. In 2024 International Conference on 3D Vision (3DV), 933--943. IEEE

2024
[38]

B.; Bermano, A

Tevet, G.; Raab, S.; Cohan, S.; Reda, D.; Luo, Z.; Peng, X. B.; Bermano, A. H.; and van de Panne, M. 2024. CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character control. arXiv preprint arXiv:2410.03441

2024 arXiv
[39]

Tevet, G.; Raab, S.; Gordon, B.; Shafir, Y.; Cohen-Or, D.; and Bermano, A. H. 2022. Human motion diffusion model. arXiv preprint arXiv:2209.14916

2022 arXiv
[40]

N.; Kaiser, .; and Polosukhin, I

Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. In Advances in Neural Information Processing Systems (NIPS)

2017
[41]

Wang, Y.; Lin, J.; Zeng, A.; Luo, Z.; Zhang, J.; and Zhang, L. 2023. Physhoi: Physics-based imitation of dynamic human-object interaction. arXiv preprint arXiv:2312.04393

2023 arXiv
[42]

Wu, Q.; Shi, Y.; Huang, X.; Yu, J.; Xu, L.; and Wang, J. 2024 a . Thor: Text to human-object interaction diffusion via relation intervention. arXiv preprint arXiv:2403.11208

2024 arXiv
[43]

Wu, Z.; Li, J.; Xu, P.; and Liu, C. K. 2024 b . Human-object interaction from human-level instructions. arXiv preprint arXiv:2406.17840

2024 arXiv
[44]

Xu, S.; Li, Z.; Wang, Y.-X.; and Gui, L.-Y. 2023. Interdiff: Generating 3d human-object interactions with physics-informed diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 14928--14940

2023
[45]

J.; Peng, X

Yi, H.; Thies, J.; Black, M. J.; Peng, X. B.; and Rempe, D. 2024. Generating human interaction motions in scenes with text control. In European Conference on Computer Vision, 246--263. Springer

2024
[46]

Zhang, M.; Cai, Z.; Pan, L.; Hong, F.; Guo, X.; Yang, L.; and Liu, Z. 2024. Motiondiffuse: Text-driven human motion generation with diffusion model. IEEE transactions on pattern analysis and machine intelligence, 46(6): 4115--4128

2024
[47]

L.; Starke, S.; Guzov, V.; and Pons-Moll, G

Zhang, X.; Bhatnagar, B. L.; Starke, S.; Guzov, V.; and Pons-Moll, G. 2022. Couch: Towards controllable human-chair interactions. In European Conference on Computer Vision, 518--535. Springer

2022
[48]

Zhou, Y.; Barnes, C.; Lu, J.; Yang, J.; and Li, H. 2019. On the continuity of rotation representations in neural networks. In CVPR

2019
[49]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...
[50]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Bae, J.; Won, J.; Lim, D.; Min, C.-H.; and Kim, Y. M. 2023. Pmp: Learning to physically interact with environments using part-wise motion priors. In ACM SIGGRAPH 2023 Conference Proceedings, 1--10

work page 2023

[2] [2]

Braun, J.; Christen, S.; Kocabas, M.; Aksan, E.; and Hilliges, O. 2024. Physically plausible full-body hand-object interaction synthesis. In 2024 International Conference on 3D Vision (3DV), 464--473. IEEE

work page 2024

[3] [3]

Cen, Z.; Pi, H.; Peng, S.; Shen, Z.; Yang, M.; Zhu, S.; Bao, H.; and Zhou, X. 2024. Generating human motion in 3D scenes from text descriptions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1855--1866

work page 2024

[4] [4]

Chen, R.; Shi, M.; Huang, S.; Tan, P.; Komura, T.; and Chen, X. 2024. Taming diffusion probabilistic models for character control. In ACM SIGGRAPH 2024 Conference Papers, 1--10

work page 2024

[5] [5]

Chentanez, N.; M \"u ller, M.; Macklin, M.; Makoviychuk, V.; and Jeschke, S. 2018. Physics-based motion capture imitation with deep reinforcement learning. In Proceedings of the 11th ACM SIGGRAPH Conference on Motion, Interaction and Games, 1--10

work page 2018

[6] [6]

H.; Golyanik, V.; and Theobalt, C

Dabral, R.; Mughal, M. H.; Golyanik, V.; and Theobalt, C. 2023. Mofusion: A framework for denoising-diffusion-based motion synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 9760--9770

work page 2023

[7] [7]

Guo, C.; Zou, S.; Zuo, X.; Wang, S.; Ji, W.; Li, X.; and Cheng, L. 2022. Generating Diverse and Natural 3D Human Motions From Text. In CVPR

work page 2022

[8] [8]

He, C.; Saito, J.; Zachary, J.; Rushmeier, H.; and Zhou, Y. 2022. Nemf: Neural motion fields for kinematic animation. NeurIPS

work page 2022