Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

SyncDiff: Synchronized Motion Diffusion for Multi-Body Human-Object Interaction Synthesis

T0 review · 5 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SyncDiff claims one diffusion model with alignment scores and explicit synchronization can generate coordinated multi-body human-object interactions.

desk verdict Solid empirical method paper; the 'maximum likelihood' label oversells a tuned weighted average, but the experiments and ablations make it worth refereeing. read the letter →

arxiv 2412.20104 v5 pith:PV5TDIAS submitted 2024-12-28 cs.CV cs.AIcs.LGcs.RO

Wenkun He , Yun Liu , Ruitao Liu , Li Yi This is my paper · ORCID
classification cs.CVcs.AIcs.LGcs.RO
keywords multi-bodyhuman-objectinteractionmotionsynthesisdiffusionmodelsynchronizationalignmentscoresfrequencydecompositionhand-object
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a single diffusion model can synthesize synchronized multi-body human-object interactions with arbitrary numbers of hands, humans, and rigid objects by modeling both individual motions and pairwise relative motions in one high-order representation. It adds two synchronization mechanisms: an alignment loss that pushes each generated relative motion to match the relative motion recomputed from the generated individual motions, and an explicit synchronization update during denoising that blends the model's predicted mean with these recomputed relative motions. The paper derives the synchronization update as maximum-likelihood sampling from a Gaussian that combines data-likelihood and alignment terms, and it splits each motion into low- and high-frequency components so that small-amplitude contact motions are not overshadowed by large-scale movements. Across five datasets, it reports better contact plausibility and higher action-recognition accuracy than prior hand-object and human-object baselines. If true, it offers a unified alternative to configuration-specific multi-body HOI synthesis pipelines.

What carries the argument

The load-bearing machinery is the high-order motion representation $x$ that concatenates, for every rigid object, its world translation and quaternion; for every articulated skeleton, its 3D joint positions; and for every ordered rigid-object pair and every skeleton-object pair, the relative motion expressed in the object's coordinate frame. The denoising network operates on the low-frequency time-domain part $x_{dc}$ and the frequency-domain high-frequency part $x_F$ separately, and the two outputs are recombined into a denoised motion. The second piece of machinery is the alignment score: for each edge of the graphical model the relative representation $\hat{x}_{b_2 \to b_1}$ is treated as a sample from a Gaussian centered at $\text{rel}(\hat{x}_{b_1}, \hat{x}_{b_2})$, which yields the alignment loss $L_{\text{align}}$. The third piece is the explicit synchronization update (Eqs. 6-8): at synchronization timesteps the new state is the weighted mean of the network's prediction and all composition-consistent reconstructions of that component from its neighbors, with noise level $\sigma' = \sqrt{\sigma^2 / (1 + 2\sigma^2\lambda)}$. The paper shows that this weighted mean is the mean of the posterior Gaussian that maximizes total data-plus-alignment likelihood under the closeness assumption.

What would settle it

Measure the average Euclidean distance between synchronized states along the sampling trajectory (states 50 denoising steps apart) and rerun inference with the same weights but with synchronization applied every step: if the distances are large, or if the per-step version does not improve the contact or semantic metrics, the maximum-likelihood interpretation of the explicit synchronization is not what carries the reported gains.

Watch

Extended reading notes

Core claim

The paper's central claim is that synchronization should be treated as a first-class object in multi-body motion diffusion, not left to implicit correlations. SyncDiff defines a graphical model whose nodes are individual body motions and whose edges are relative motions between body pairs, concatenates all of them into a single vector $x$, and trains one denoising model on that vector. During training it adds an alignment loss $L_{\text{align}}$ that supervises each generated relative motion $\hat{x}_{b_2 \to b_1}$ to agree with $\text{rel}(\hat{x}_{b_1}, \hat{x}_{b_2})$ computed from the generated individual motions. During inference, at every $s$-th denoising step, it replaces the plain DDPM step with a synchronized step: each individual and relative component becomes a precision-weighted average of the model-predicted mean and the values implied by neighboring components through composition or relative operations, with rescaled noise. The paper proves in Appendix A.2 that this update is equivalent to maximum-likelihood sampling from the Gaussian formed by combining data sample scores and alignment scores, assuming the states before and after the update are close. On top of this, the model decomposes individual and relative motions into low-frequency and high-frequency parts and supervises them separately, with high-frequency parts represented explicitly in the frequency domain, to preserve contact-level semantics. Experiments compare against DiffH2O, MACS, OMOMO, and CG-HOI on TACO, CORE4D, GRAB, OAKINK2, and BEHAVE.

Load-bearing premise

The derivation of the explicit synchronization update assumes the current noisy state and the next state are close enough that alignment quantities computed at one state also describe the other, yet synchronization is applied every 50 denoising steps, so that closeness is not guaranteed.

Editorial extensions

If this is right

  • A single SyncDiff model, trained once on a given body-count configuration, can synthesize complete multi-body interactions without requiring predefined object trajectories, grasp reference frames, or contact maps.
  • Because the relative motions are included as auxiliary outputs, every pair of bodies is explicitly tied to the individual motions, which should reduce interpenetration and contact loss compared with methods that only denoise individual trajectories.
  • The explicit synchronization update, under the paper's closeness assumption, is maximum-likelihood sampling from the Gaussian combining data-sample and alignment scores, which gives the inference procedure a stated theoretical grounding rather than a purely heuristic post-processing step.
  • Frequency decomposition separates small-amplitude semantically important movements from large-scale motion, so actions like rubbing, tapping, and pouring are not averaged away.
  • On the five datasets considered, the paper reports that SyncDiff outperforms configuration-specific baselines on contact plausibility and action-recognition accuracy, with larger gains in settings containing more bodies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the closeness assumption in Appendix A.2 does not hold for states 50 denoising steps apart, the synchronization formula is better described as a tuned precision-weighted average than as exact maximum-likelihood sampling; this would not erase the reported metric gains but would separate the empirical mechanism from the theoretical one.
  • The formula's precision weights suggest a direct extension: instead of a global $\lambda$ and fixed $s$, one could learn or schedule per-edge alignment weights, and ablation-style sweeps over $s$ already show the cost-quality trade-off.
  • The alignment-score construction only needs a compositional operation that predicts one entity from another; it could transfer to multi-agent trajectory generation or scene-graph conditioned synthesis.
  • The quadratic blow-up in pairwise relative terms with the number of objects is a natural place to prune edges that do not need synchronization, an extension the paper itself lists as future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. SyncDiff proposes a unified diffusion framework for synthesizing multi-body human-object interactions with arbitrary numbers of humans, hands, and rigid objects. It concatenates individual and pairwise relative motions into a high-order representation, trains a single transformer-based diffusion model with separate losses on low- and high-frequency components plus an alignment loss, and at inference periodically applies an explicit synchronization update that the paper claims is equivalent to maximum-likelihood sampling on a dynamic graphical model. The paper reports quantitative improvements over task-specific baselines (MACS, DiffH2O, OMOMO, CG-HOI) on TACO, CORE4D, GRAB, OAKINK2, and BEHAVE, together with ablations and a user study.

Significance. If the empirical claims hold, SyncDiff is a valuable step toward a single generative model for several HOI configurations, and the frequency-domain treatment of high-frequency interaction semantics is a useful idea with careful supplementary ablations. The paper is unusually broad in scope: five datasets, multiple ablations, runtime analysis, a user study, and supplementary experiments that decouple frequency filtering from explicit frequency-domain representation. The mathematical derivations in Appendix A.2 are transparent and the algorithm is stated in full. However, the theoretical framing currently overstates what is proven: the explicit synchronization update is an approximate, locally linearized weighted average rather than exact maximum-likelihood sampling, and the RA metric is compromised by training the evaluator on the test splits. The empirical contribution remains plausible, but the manuscript needs revision before the stated claims can be accepted.

major comments (5)
  1. [§3.6, Appendix A.2, Algorithm 1] The claim that Eqs. (6)-(8) are 'equivalent to maximum likelihood sampling ... based on both data sample scores and alignment scores' is stronger than what the proof establishes. The argument in Appendix A.2 explicitly assumes that x_t and x_{t-1} are close so that the alignment targets f_k = a_k composed with b_k can be evaluated at x_t rather than at the unknown x_{t-1}. In the actual algorithm, synchronization is performed every s=50 DDPM steps with T=1000, so consecutive synchronized states are 50 denoising steps apart, not one. Moreover, the f_k are nonlinear functions of the synchronized state through quaternion multiplication and inversion in rel and comb; fixing them at x_t turns a non-convex joint objective into independent quadratic subproblems. The resulting update is a tuned weighted average controlled by lambda_exp, not maximum-likelihood sampling from the joint posterior over x_{t-1}. Please soften the abstract, Introduction, and §3.6 claims to 'approximate' or 'local' maximum-likelihood, or provide a bound on the approximation error as a function of the synchronization interval s.
  2. [§4.2 and Appendix E.2.6] The Recognition Accuracy metric is computed with a classifier trained on the union of train, val, and test splits (§4.2). This means the test motions used for evaluation are part of the classifier's training data, so RA is not a clean measure of semantic generalization and can favor methods whose outputs resemble the ground-truth test distribution. The defense in E.2.6 correctly notes that a train-only classifier has its own bias, but it does not resolve the circularity; at minimum, please report RA from a classifier trained only on train (or train+val) and compare both versions, and avoid presenting RA as primary evidence for semantic superiority without this check.
  3. [§4.3, Tables 1-5] No error bars, confidence intervals, or significance tests are reported for any metric. Many headline comparisons ride on small margins (e.g., Table 2, GRAB unseen-object CSR: 10.5 vs 8.6; Table 3, CORE4D CRR: 6.15 vs 5.74), and single-seed runs do not support claims of 'large margin' superiority. Please report mean plus/minus standard deviation over at least three training or inference seeds and a paired significance test (or confidence intervals) for the main comparisons and ablations.
  4. [Table 2 caption and §4.3] The MACS baseline on GRAB is obtained via the authors' re-implementation rather than the released code or official results. Because MACS is a key baseline for hand-object interaction, the re-implementation must be made public together with the exact protocol (including the object-trajectory phase and hand-synthesis phase), and ideally validated against the original paper's numbers on a common split. Otherwise the comparison is not independently verifiable.
  5. [Algorithm 1 vs Eq. (6)] For m=1, Eq. (6) states that no synchronization is performed for rigid-body individual motions and the denoising formula is identical to the unsynchronized one. However, Algorithm 1's branch for m=1 (lines 16-17) applies an extra lambda_1 * mu_{o_j} term and a modified noise scale sigma' (line 31), which is not the same as Eq. (5). Since m=1 occurs in GRAB and BEHAVE, this is not a purely cosmetic discrepancy. Please reconcile the algorithm with the equation; if m=1 is actually synchronized, the text and Eq. (6) must be corrected, and the GRAB/BEHAVE results must be interpreted accordingly.
minor comments (5)
  1. [Appendix A.2, item 2] The displayed update for the articulated-skeleton individual motion writes the first data term as mu_{o_j}; this should be mu_{h_i} to match Eq. (7).
  2. [Throughout] The dataset name 'BEHA VE' appears with a spurious space in several places; it should read 'BEHAVE'.
  3. [§3.3] The cutoff L is chosen as 16, but the admissible interval is stated as [4, N/4). Please clarify the frame count N used for each dataset so the reader can verify that L=16 falls in the allowed range.
  4. [Table 2 caption] The caption says the best method is highlighted in red and the second best in blue, while the other tables and the main text say the best is in bold; please make the formatting convention consistent.
  5. [§4.5] The user study report states 150 questions and 150 participants each answering 10 questions; please make explicit how the 1500 responses are assigned so that exactly 10 responses per question are achieved.

Circularity Check

2 steps flagged · score 6.0 of 10

Appendix A.2's 'derivations' are definitional: alignment scores are posited as Gaussians centered at rel(...), so Lalign follows by algebra, and the explicit synchronization update is the closed-form minimizer of that hand-set quadratic; the maximum-likelihood claim is thus true by construction, not by external derivation.

  1. self definitional [Appendix A.2, derivation of alignment loss Lalign (Eq. 14); referenced from Section 3.5]
    "A simple solution is to let ˆxb2→b1 follow a distribution N (µ, σ2), where µ = rel(ˆxb1 , ˆxb2 ), and σ is a parameter that we can tune."

    The paper advertises a mathematical derivation of alignment scores and a proof of equivalence between Lalign and those scores. In the appendix, the alignment score is not derived from data or from the diffusion posterior; it is posited as the score of a Gaussian whose mean is exactly rel(ˆx_b1, ˆx_b2). The negative log-likelihood of that chosen Gaussian is, by algebra, proportional to ||ˆx_b2→b1 − rel(ˆx_b1, ˆx_b2)||², which is precisely each term of Lalign. Thus the 'derived' loss is equivalent to the definition of the alignment score by construction; the derivation cannot fail and supplies no independent constraint.

  2. self definitional [Appendix A.2, proof of explicit synchronization formulas (Eqs. 6-8); claimed in Section 3.6 and Abstract]
    "the negative logarithm of alignment likelihood is defined as Palign = P|V | v=1 λv∥cv − av ◦ bv∥2 2 ... maximizing total likelihood (the combination of data sample likelihood and alignment likelihoods) is equivalent to minimizing Pˆx′′ = 1 2σ2 ∥ˆx ′′ − ˆµ ′′∥2 2 + PK k=1 λk∥ˆx ′′ − ak ◦ bk∥2 2"

    The proof defines the alignment likelihood as exp(−Σλ||c−a◦b||²) and then solves the resulting quadratic minimization; the 'maximum likelihood sampling' is therefore a restatement of this manually chosen objective. The relative/combination targets f_k are evaluated at the current noisy state x_t under the stated closeness assumption, and the weight λ is set via λexp tuned per dataset (Table 19). Equations (6)-(8) are the closed-form weighted average of the diffusion prediction and the hand-set consistency targets; calling the result 'maximum likelihood inference' is true only by construction of the alignment likelihood, not by an independent derivation.

full rationale

The derivation chain is mostly self-contained: the diffusion backbone, frequency decomposition, and alignment loss are standard supervised objectives, and the empirical comparison is against external baselines with ablations and a user study. No load-bearing self-citation or imported uniqueness theorem appears. However, the paper's two headline theoretical claims—'alignment scores' and 'explicit synchronization = maximum likelihood sampling'—are definitional rather than derived. In Appendix A.2 the alignment score is posited as the score of a Gaussian centered at rel(...), so the alignment loss is its negative log-likelihood by algebra; the 'proof' of equivalence cannot fail. Likewise, the explicit synchronization update is the closed-form minimizer of a manually weighted quadratic objective, with λexp tuned per dataset, so the maximum-likelihood characterization is a restatement of the chosen objective, not an external result. These steps make the theoretical claim circular in the narrow sense, though the empirical contribution (contact/semantic metrics, ablations, user study) remains independently supported and is not invalidated. The stated one-step closeness assumption (x_t ≈ x_{t−1}) is an approximation that further weakens the max-likelihood reading, but that is a correctness risk rather than circularity. Score 6 reflects partial circularity: central theory reduces by construction, while the empirical method stands on its own.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central method rests on several design choices that are partially ad hoc: the alignment-score Gaussian model, the tuned synchronization weights, and the frequency cutoff. No new physical entities or external constraints are postulated. The most load-bearing axiom is the closeness assumption between diffusion states 50 steps apart, which is used to justify the maximum-likelihood framing of explicit synchronization.

free parameters (5)
  • λexp = 0.3
    Empirical weight controlling the strength of alignment likelihood relative to data likelihood in explicit synchronization; tuned per dataset, not derived.
  • λalign = 0.3 for TACO/CORE4D/OAKINK2, 0.15 for GRAB/BEHAVE
    Weight of the alignment loss in the total training loss; chosen by hand.
  • Synchronization interval s = 50
    Selected on TACO Split 1 as a speed-quality tradeoff; affects the validity of the 'x_t close to x_{t-1}' assumption.
  • Frequency cutoff L = 16
    Chosen from a reconstruction-error table; balances motion fidelity against removing high-frequency noise.
  • λac = 0.3 to 2.5 depending on dataset
    Weight for the high-frequency reconstruction loss; tuned per dataset.
assumptions (5)
  • domain assumption Individual and relative motions are represented in world/object coordinate systems and concatenated; articulated skeletons are MANO/SMPL-X reconstructible from joints and shape β.
    Section 3.1-3.2. This limits the method to settings where at least one rigid body provides a coordinate frame, excluding pure multi-human interaction.
  • domain assumption A single diffusion model can learn the joint distribution over the high-order concatenated representation, including redundant relative motions.
    Section 3.4. This is an architectural assumption about model capacity and learnability.
  • ad hoc to paper The alignment score for a relative motion can be modeled as a Gaussian centered at rel(x_b1, x_b2).
    Section A.2. This is a design choice that makes the L2 alignment loss follow by definition; it is not derived from an external principle.
  • ad hoc to paper Denoising states at consecutive synchronization steps are close enough to reuse quantities from x_t when updating x_{t-1}.
    Appendix A.2, explicitly stated as 'we make an assumption that x_t and x_{t-1} are close'.
  • domain assumption The frequency cutoff L discards only noise, not semantic high-frequency content.
    Section 3.3 and B.1. The choice of L is based on reconstruction error, not on a formal separation of semantic and non-semantic frequencies.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SyncDiff: Synchronized Motion Diffusion for Multi-Body Human-Object Interaction Synthesis." pith.science (2026). https://pith.science/paper/PV5TDIAS

@misc{pith2026241220104,
  author       = {Pith},
  title        = {Pith review of: SyncDiff: Synchronized Motion Diffusion for Multi-Body Human-Object Interaction Synthesis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PV5TDIAS}},
  note         = {Machine review of arXiv:2412.20104}
}
read the original abstract

Synthesizing realistic human-object interaction motions is a critical problem in VR/AR and human animation. Unlike the commonly studied scenarios involving a single human or hand interacting with one object, we address a more generic multi-body setting with arbitrary numbers of humans, hands, and objects. This complexity introduces significant challenges in synchronizing motions due to the high correlations and mutual influences among bodies. To address these challenges, we introduce SyncDiff, a novel method for multi-body interaction synthesis using a synchronized motion diffusion strategy. SyncDiff employs a single diffusion model to capture the joint distribution of multi-body motions. To enhance motion fidelity, we propose a frequency-domain motion decomposition scheme. Additionally, we introduce a new set of alignment scores to emphasize the synchronization of different body motions. SyncDiff jointly optimizes both data sample likelihood and alignment likelihood through an explicit synchronization strategy. Extensive experiments across four datasets with various multi-body configurations demonstrate the superiority of SyncDiff over existing state-of-the-art motion synthesis methods.

Figures

Figures reproduced from arXiv: 2412.20104 by the authors.

Figure 1
Figure 1. SyncDiff is a unified framework synthesizing synchronized multi-body interaction motions with any number of hands, humans, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of SyncDiff. The light blue boxes show the inference process with explicit synchronization steps performed every s step. For denoising steps irrelevant to explicit synchronization (those marked as “(s − 1) times”), the noise level is set to σt. For the calculation from xˆ to µˆ, no noise is added. For the calculation of xˆt−1 based on xˆt and µˆt in explicit synchronization step, the noise level is σ ′ t . … view at source ↗
Figure 3
Figure 3. Qualitative results from GRAB [81] dataset [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Qualitative results from TACO [48] dataset. Invalid ac￾tion indicates the poses cannot complete the operation effectively. In [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 6
Figure 6. Figure 6: Qualitative results from OAKINK2 [108] dataset. The task requires precise contact between objects, where the bottle cap needs to align perfectly with the bottle, and there needs to be a tendency for it to be twisted down in a clockwise spiral. while our method mitigate…
Figure 5
Figure 5. Figure 5: Qualitative results from TACO [48] dataset. Human-Object Interaction. For human-object interac￾tion synthesis, we compare our method to OMOMO [42] and CG-HOI [20]. For OMOMO, we first use their condi￾tional diffusion models to generate object trajectories, and then use…
Figure 8
Figure 8. Figure 8: Qualitative results from BEHAVE [2] dataset. Baseline methods suffer from unreasonable grasp poses due to unsynchro￾nized synthesis of body transformations. chronization mechanisms and all three designs, denoted as “w/o Lalign, exp sync” and “w/o all”, respectively. Mo…
Figure 9
Figure 9. Figure 9: User study results on different dataset splits. [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: Qualitative results from TACO [48] dataset. Peri￾odic relative motions are required between two objects. The color changes from deep to light, representing time passage. After re￾moving the decomposition mechanism, the spatula tends to get stuck in a small area on the…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FaceEditTalker: Controllable Talking Head Generation with Facial Attribute Editing

    cs.CV 2025-05 conditional novelty 5.0 of 10

    A single framework can edit predefined facial attributes in audio-synchronized talking head videos while preserving identity and lip-sync quality.

Reference graph

Works this paper leans on

146 extracted references · 56 canonical work pages · cited by 1 Pith paper

  1. [1]

    Spectral synchronization of multiple views in se(3)

    Federica Arrigoni, Beatrice Rossi, and Andrea Fusiello. Spectral synchronization of multiple views in se(3). SIAM Journal on Imaging Sciences, 9(4):1963–1990, 2016. 3

    1963

  2. [2]

    Petrov, Cris- tian Sminchisescu, Christian Theobalt, and Gerard Pons- Moll

    Bharat Lal Bhatnagar, Xianghui Xie, Ilya A. Petrov, Cris- tian Sminchisescu, Christian Theobalt, and Gerard Pons- Moll. Behave: Dataset and method for tracking human object interactions. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 15914–15925, 2022. 2, 6, 8

    2022

  3. [3]

    Contactdb: Analyzing and predicting grasp contact via thermal imaging

    Samarth Brahmbhatt, Cusuh Ham, Charles C Kemp, and James Hays. Contactdb: Analyzing and predicting grasp contact via thermal imaging. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8709–8719, 2019

    2019

  4. [4]

    Contactpose: A dataset of grasps with object contact and hand pose

    Samarth Brahmbhatt, Chengcheng Tang, Christopher D Twigg, Charles C Kemp, and James Hays. Contactpose: A dataset of grasps with object contact and hand pose. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIII 16, pages 361–378. Springer, 2020. 2

    2020

  5. [5]

    Text2hoi: Text-guided 3d motion generation for hand-object interaction

    Junuk Cha, Jihyeon Kim, Jae Shin Yoon, and Seungryul Baek. Text2hoi: Text-guided 3d motion generation for hand-object interaction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1577–1585, 2024. 2

    2024

  6. [6]

    Dexycb: A benchmark for capturing hand grasping of objects

    Yu-Wei Chao, Wei Yang, Yu Xiang, Pavlo Molchanov, Ankur Handa, Jonathan Tremblay, Yashraj S Narang, Karl Van Wyk, Umar Iqbal, Stan Birchfield, et al. Dexycb: A benchmark for capturing hand grasping of objects. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9044–9053, 2021. 2

    2021

  7. [7]

    Nalin Chaturvedi, Amit Sanyal, and N. H. Mcclamroch. Rigid-body attitude control. IEEE Control Systems, 31:30– 51, 2011. 3

    2011

  8. [8]

    Karen Liu

    Sirui Chen, Albert Wu, and C. Karen Liu. Synthesizing dexterous nonprehensile pregrasp for ungraspable objects. In Special Interest Group on Computer Graphics and In- teractive Techniques Conference Conference Proceedings, page 1–10. ACM, 2023. 2

Show all 146 references
  1. [9]

    Distributed adaptive attitude control for networked underactuated flexible space- craft

    Ti Chen, Jinjun Shan, and Hao Wen. Distributed adaptive attitude control for networked underactuated flexible space- craft. IEEE Transactions on Aerospace and Electronic Sys- tems, 55(1):215–225, 2019. 3

    2019

  2. [10]

    Ilvr: Conditioning method for denoising diffusion probabilistic models

    Jooyoung Choi, Sungwon Kim, Yonghyun Jeong, Youngjune Gwon, and Sungroh Yoon. Ilvr: Conditioning method for denoising diffusion probabilistic models. arXiv preprint arXiv:2108.02938, 2021. 2

    2021 arXiv

  3. [11]

    D-grasp: Physically plausible dynamic grasp synthesis for hand- object interactions

    Sammy Christen, Muhammed Kocabas, Emre Aksan, Jemin Hwangbo, Jie Song, and Otmar Hilliges. D-grasp: Physically plausible dynamic grasp synthesis for hand- object interactions. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 20577–2...

    2022

  4. [12]

    Learning human-to-robot handovers from point clouds

    Sammy Christen, Wei Yang, Claudia P ´erez-D’Arpino, Ot- mar Hilliges, Dieter Fox, and Yu-Wei Chao. Learning human-to-robot handovers from point clouds. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9654–9664, 2023. 2

    2023

  5. [13]

    Diffh2o: Diffusion-based synthesis of hand-object interactions from textual descriptions

    Sammy Christen, Shreyas Hampali, Fadime Sener, Edoardo Remelli, Tomas Hodan, Eric Sauser, Shugao Ma, and Bugra Tekin. Diffh2o: Diffusion-based synthesis of hand-object interactions from textual descriptions. In SIGGRAPH Asia 2024 Conference Papers, pages 1–11, 2024. 1, 2, 3, 6...

    2024

  6. [14]

    Semgeomo: Dynamic contextual human motion generation with semantic and geometric guidance, 2025

    Peishan Cong, Ziyi Wang, Yuexin Ma, and Xiangyu Yue. Semgeomo: Dynamic contextual human motion generation with semantic and geometric guidance, 2025. 2

    2025

  7. [15]

    Col- lage: Collaborative human-agent interaction generation us- ing hierarchical latent diffusion and language models.arXiv preprint arXiv:2409.20502, 2024

    Divyanshu Daiya, Damon Conover, and Aniket Bera. Col- lage: Collaborative human-agent interaction generation us- ing hierarchical latent diffusion and language models.arXiv preprint arXiv:2409.20502, 2024. 2

    2024 arXiv

  8. [16]

    Chen, and Fr´edo Durand

    Abe Davis, Justin G. Chen, and Fr´edo Durand. Image-space modal bases for plausible manipulation of objects in video. ACM Transactions on Graphics (SIGGRAPH) , 34(6):1–7,

  9. [17]

    Visual vibration analysis

    Myers Abraham Davis. Visual vibration analysis. PhD the- sis, Massachusetts Institute of Technology, 2016. 3

    2016

  10. [18]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural informa- tion processing systems, 34:8780–8794, 2021. 2

    2021

  11. [19]

    Wind projection basis for real-time animation of trees

    Julien Diener, Mathieu Rodriguez, Lionel Baboud, and Li- onel Reveret. Wind projection basis for real-time animation of trees. In Computer Graphics Forum, pages 533–540. Wi- ley Online Library, 2009. 3

    2009

  12. [20]

    Cg-hoi: Contact-guided 3d human-object interaction generation

    Christian Diller and Angela Dai. Cg-hoi: Contact-guided 3d human-object interaction generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19888–19901, 2024. 2, 3, 7, 8

    2024

  13. [21]

    Arctic: A dataset for dexterous bimanual hand- object manipulation

    Zicong Fan, Omid Taheri, Dimitrios Tzionas, Muhammed Kocabas, Manuel Kaufmann, Michael J Black, and Otmar Hilliges. Arctic: A dataset for dexterous bimanual hand- object manipulation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages...

    2023

  14. [22]

    Eigenactor: Variant body-object interac- tion generation evolved from invariant action basis reason- ing, 2025

    Xuehao Gao, Yang Yang, Shaoyi Du, Yang Wu, Yebin Liu, and Guo-Jun Qi. Eigenactor: Variant body-object interac- tion generation evolved from invariant action basis reason- ing, 2025. 2

    2025

  15. [23]

    Physics-based dexterous manipulations with esti- mated hand poses and residual reinforcement learning

    Guillermo Garcia-Hernando, Edward Johns, and Tae-Kyun Kim. Physics-based dexterous manipulations with esti- mated hand poses and residual reinforcement learning. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 9561–9568. IEEE, 2020. 2

    2020

  16. [24]

    Imos: Intent- driven full-body motion synthesis for human-object inter- actions

    Anindita Ghosh, Rishabh Dabral, Vladislav Golyanik, Christian Theobalt, and Philipp Slusallek. Imos: Intent- driven full-body motion synthesis for human-object inter- actions. In Computer Graphics Forum, pages 1–12. Wiley Online Library, 2023. 2, 6, 8

    2023

  17. [25]

    Soft actor-critic algorithms and applications

    Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel, et al. Soft actor-critic algorithms and applications. arXiv preprint arXiv:1812.05905, 2018. 2

    2018 arXiv

  18. [26]

    Honnotate: A method for 3d annotation of hand and object poses

    Shreyas Hampali, Mahdi Rad, Markus Oberweger, and Vin- cent Lepetit. Honnotate: A method for 3d annotation of hand and object poses. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 3196–3206, 2020. 2

    2020

  19. [27]

    Stochas- tic scene-aware motion prediction

    Mohamed Hassan, Duygu Ceylan, Ruben Villegas, Jun Saito, Jimei Yang, Yi Zhou, and Michael J Black. Stochas- tic scene-aware motion prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 11374–11384, 2021. 2

    2021

  20. [28]

    Takeshi Hatanaka, Yuji Igarashi, Masayuki Fujita, and Mark W. Spong. Passivity-based pose synchronization in three dimensions. IEEE Transactions on Automatic Con- trol, 57(2):360–375, 2012. 3

    2012

  21. [29]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 2, 3

    2022 arXiv

  22. [30]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural informa- tion processing systems, 33:6840–6851, 2020. 2, 5, 16, 17

    2020

  23. [31]

    Ima- gen video: High definition video generation with diffusion models

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Ima- gen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303, 2022. 2

    arXiv

  24. [32]

    Video dif- fusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video dif- fusion models. Advances in Neural Information Processing Systems, 35:8633–8646, 2022. 2, 16

    2022

  25. [33]

    Yuji Igarashi, Takeshi Hatanaka, Masayuki Fujita, and Mark W. Spong. Passivity-based attitude synchronization in se(3). IEEE Transactions on Control Systems Technol- ogy, 17(5):1119–1134, 2009. 3

    2009

  26. [34]

    Controllable hand grasp generation for hoi and efficient evaluation meth- ods, 2025

    Ishant, Rongliang Wu, and Joo Hwee Lim. Controllable hand grasp generation for hoi and efficient evaluation meth- ods, 2025. 2

    2025

  27. [35]

    Planning with diffusion for flexible behav- ior synthesis

    Michael Janner, Yilun Du, Joshua B Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behav- ior synthesis. arXiv preprint arXiv:2205.09991, 2022. 2

    2022 arXiv

  28. [36]

    Event- triggered attitude consensus with absolute and relative atti- tude measurements

    Xin Jin, Yang Shi, Yang Tang, and Xiaotai Wu. Event- triggered attitude consensus with absolute and relative atti- tude measurements. Automatica, 122:109245, 2020. 3

    2020

  29. [37]

    Guided motion diffusion for controllable human motion synthesis

    Korrawe Karunratanakul, Konpat Preechakul, Supasorn Suwajanakorn, and Siyu Tang. Guided motion diffusion for controllable human motion synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 2151–2162, 2023. 2, 26

    2023

  30. [38]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 2

    2013 arXiv

  31. [39]

    The kit bimanual manipulation dataset

    Franziska Krebs, Andre Meixner, Isabel Patzer, and Tamim Asfour. The kit bimanual manipulation dataset. In 2020 IEEE-RAS 20th International Conference on Humanoid Robots (Humanoids), pages 499–506. IEEE, 2021. 2

    2020

  32. [40]

    H2o: Two hands manipulating objects for first person interaction recognition

    Taein Kwon, Bugra Tekin, Jan St ¨uhmer, Federica Bogo, and Marc Pollefeys. H2o: Two hands manipulating objects for first person interaction recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 10138–10148, 2021. 2

    2021

  33. [41]

    Controllable human-object interaction synthesis

    Jiaman Li, Alexander Clegg, Roozbeh Mottaghi, Jiajun Wu, Xavier Puig, and C Karen Liu. Controllable human-object interaction synthesis. arXiv preprint arXiv:2312.03913 ,

    arXiv

  34. [42]

    Object motion guided human motion synthesis

    Jiaman Li, Jiajun Wu, and C Karen Liu. Object motion guided human motion synthesis. ACM Transactions on Graphics (TOG), 42(6):1–11, 2023. 1, 2, 3, 6, 7, 8

    2023

  35. [43]

    Generative image dynamics

    Zhengqi Li, Richard Tucker, Noah Snavely, and Aleksander Holynski. Generative image dynamics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24142–24153, 2024. 3

    2024

  36. [44]

    Intergen: Diffusion-based multi-human motion generation under complex interactions

    Han Liang, Wenqian Zhang, Wenxuan Li, Jingyi Yu, and Lan Xu. Intergen: Diffusion-based multi-human motion generation under complex interactions. International Jour- nal of Computer Vision, 132(9):3463–3483, 2024. 2, 26

    2024

  37. [45]

    Dexrepnet: Learn- ing dexterous robotic grasping network with geometric and spatial hand-object representations

    Qingtao Liu, Yu Cui, Zhengnan Sun, Haoming Li, Gaofeng Li, Lin Shao, Jiming Chen, and Qi Ye. Dexrepnet: Learn- ing dexterous robotic grasping network with geometric and spatial hand-object representations. arXiv preprint arXiv:2303.09806, 2023. 2

    2023 arXiv

  38. [46]

    Contactgen: Generative contact modeling for grasp generation

    Shaowei Liu, Yang Zhou, Jimei Yang, Saurabh Gupta, and Shenlong Wang. Contactgen: Generative contact modeling for grasp generation. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision, pages 20609– 20620, 2023. 2

    2023

  39. [47]

    Hoi4d: A 4d egocentric dataset for category-level human-object interaction

    Yunze Liu, Yun Liu, Che Jiang, Kangbo Lyu, Weikang Wan, Hao Shen, Boqiang Liang, Zhoujie Fu, He Wang, and Li Yi. Hoi4d: A 4d egocentric dataset for category-level human-object interaction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , p...

    2022

  40. [48]

    Taco: Benchmarking general- izable bimanual tool-action-object understanding

    Yun Liu, Haolin Yang, Xu Si, Ling Liu, Zipeng Li, Yuxiang Zhang, Yebin Liu, and Li Yi. Taco: Benchmarking general- izable bimanual tool-action-object understanding. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21740–21751, 2024....

    2024

  41. [49]

    Controlled synchro- nization of heterogeneous robotic manipulators in the task space

    Yen-Chen Liu and Nikhil Chopra. Controlled synchro- nization of heterogeneous robotic manipulators in the task space. IEEE Transactions on Robotics , 28(1):268–275,

  42. [50]

    Repaint: In- painting using denoising diffusion probabilistic models

    Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: In- painting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 11461–11471, 2022. 2

    2022

  43. [51]

    Isaac gym: High performance gpu-based physics simulation for robot learning

    Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, et al. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108.10470, 2021. 2

    2021 arXiv

  44. [52]

    Learning dexter- ous grasping with object-centric visual affordances

    Priyanka Mandikal and Kristen Grauman. Learning dexter- ous grasping with object-centric visual affordances. In2021 IEEE international conference on robotics and automation (ICRA), pages 6169–6176. IEEE, 2021. 2

    2021

  45. [53]

    Synchronization on riemannian mani- folds: Multiply connected implies multistable

    Johan Markdahl. Synchronization on riemannian mani- folds: Multiply connected implies multistable. IEEE Trans- actions on Automatic Control, 66(9):4311–4318, 2020. 3

    2020

  46. [54]

    High-dimensional kuramoto models on stiefel manifolds synchronize complex networks almost globally

    Johan Markdahl, Johan Thunberg, and Jorge Goncalves. High-dimensional kuramoto models on stiefel manifolds synchronize complex networks almost globally. Automat- ica, 113:108736, 2020. 3

    2020

  47. [55]

    Distributed finite- time attitude containment control for multiple rigid bodies

    Ziyang Meng, Wei Ren, and Zheng You. Distributed finite- time attitude containment control for multiple rigid bodies. Automatica, 46(12):2092–2099, 2010. 3

    2010

  48. [56]

    Generating continual human motion in diverse 3d scenes

    Aymen Mir, Xavier Puig, Angjoo Kanazawa, and Gerard Pons-Moll. Generating continual human motion in diverse 3d scenes. In 2024 International Conference on 3D Vision (3DV), pages 903–913. IEEE, 2024. 2

    2024

  49. [57]

    Expressive body capture: 3d hands, face, and body from a single image

    Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed AA Osman, Dimitrios Tzionas, and Michael J Black. Expressive body capture: 3d hands, face, and body from a single image. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognitio...

    2019

  50. [58]

    Hoi-diff: Text-driven syn- thesis of 3d human-object interactions using diffusion mod- els

    Xiaogang Peng, Yiming Xie, Zizhao Wu, Varun Jampani, Deqing Sun, and Huaizu Jiang. Hoi-diff: Text-driven syn- thesis of 3d human-object interactions using diffusion mod- els. arXiv preprint arXiv:2312.06553, 2023. 2

    2023 arXiv

  51. [59]

    Efficient learning on point clouds with basis point sets

    Sergey Prokudin, Christoph Lassner, and Javier Romero. Efficient learning on point clouds with basis point sets. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4332–4341, 2019. 4, 23, 24

    2019

  52. [60]

    Pointnet: Deep learning on point sets for 3d classification and segmentation

    Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 652– 660, 2017. 23

    2017

  53. [61]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space.Advances in neural information processing systems, 30, 2017

    Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space.Advances in neural information processing systems, 30, 2017. 23

    2017

  54. [62]

    Human grasp generation for rigid and deformable objects with decom- posed vq-vae, 2025

    Mengshi Qi, Zhe Zhao, and Huadong Ma. Human grasp generation for rigid and deformable objects with decom- posed vq-vae, 2025. 2

    2025

  55. [63]

    Dexmv: Im- itation learning for dexterous manipulation from human videos

    Yuzhe Qin, Yueh-Hua Wu, Shaowei Liu, Hanwen Jiang, Ruihan Yang, Yang Fu, and Xiaolong Wang. Dexmv: Im- itation learning for dexterous manipulation from human videos. In European Conference on Computer Vision, pages 570–587. Springer, 2022. 2

    2022

  56. [64]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In International conference on machine learning...

    2021

  57. [65]

    Learning complex dexterous manipulation with deep reinforcement learning and demonstrations

    Aravind Rajeswaran, Vikash Kumar, Abhishek Gupta, Giu- lia Vezzani, John Schulman, Emanuel Todorov, and Sergey Levine. Learning complex dexterous manipulation with deep reinforcement learning and demonstrations. arXiv preprint arXiv:1709.10087, 2017. 2

    2017 arXiv

  58. [66]

    Distributed leaderless consensus algorithms for networked euler–lagrange systems

    Wei Ren. Distributed leaderless consensus algorithms for networked euler–lagrange systems. International Journal of Control, 82:2137–2149, 2009. 3

    2009

  59. [67]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2, 4, 16

    2022

  60. [68]

    Javier Romero, Dimitrios Tzionas, and Michael J. Black. Embodied hands: modeling and capturing hands and bod- ies together. ACM Transactions on Graphics, 36(6):1–17,

  61. [69]

    Autonomous rigid body attitude synchronization

    Alain Sarlette, Rodolphe Sepulchre, and Naomi Ehrich Leonard. Autonomous rigid body attitude synchronization. In 2007 46th IEEE Conference on Decision and Control , pages 2566–2571, 2007. 3

    2007

  62. [70]

    Proximal policy optimization algo- rithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Rad- ford, and Oleg Klimov. Proximal policy optimization algo- rithms. arXiv preprint arXiv:1707.06347, 2017. 2

    2017 arXiv

  63. [71]

    Human motion diffusion as a generative prior

    Yonatan Shafir, Guy Tevet, Roy Kapon, and Amit H Bermano. Human motion diffusion as a generative prior. arXiv preprint arXiv:2303.01418, 2023. 2, 26

    2023 arXiv

  64. [72]

    Towards open domain text-driven synthesis ofmulti-person motions

    Mengyi Shan, Lu Dong, Yutao Han, Yuan Yao, Tao Liu, Ifeoma Nwogu, Guo Jun Qi, and Mitch Hill. Towards open domain text-driven synthesis ofmulti-person motions. In European Conference on Computer Vision, 2025. 26

    2025

  65. [73]

    Macs: Mass conditioned 3d hand and object motion synthe- sis

    Soshi Shimada, Franziska Mueller, Jan Bednarik, Bardia Doosti, Bernd Bickel, Danhang Tang, Vladislav Golyanik, Jonathan Taylor, Christian Theobalt, and Thabo Beeler. Macs: Mass conditioned 3d hand and object motion synthe- sis. In 2024 International Conference on 3D Vision (3D...

    2024

  66. [74]

    Stochastic motion— motion under the influence of wind

    Mikio Shinya and Alain Fournier. Stochastic motion— motion under the influence of wind. Computer Graphics Forum, 11(3), 1992. 3

    1992

  67. [75]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International confer- ence on machine learning, pages 2256–2265. PMLR, 2015. 16, 17

    2015

  68. [76]

    Learn- ing structured output representation using deep conditional generative models

    Kihyuk Sohn, Honglak Lee, and Xinchen Yan. Learn- ing structured output representation using deep conditional generative models. Advances in neural information pro- cessing systems, 28, 2015. 2

    2015

  69. [77]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 2

    2010 arXiv

  70. [78]

    Generative modeling by estimating gradients of the data distribution

    Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019. 16

    2019

  71. [79]

    Multi-scale stochastic modelling of complex nat- ural phenomena

    Jos Stam. Multi-scale stochastic modelling of complex nat- ural phenomena. PhD thesis, 1995. 3

    1995

  72. [80]

    Stochastic dynamics: Simulating the effects of turbulence on flexible structures

    Jos Stam. Stochastic dynamics: Simulating the effects of turbulence on flexible structures. Computer Graphics Fo- rum, 16(3), 1997. 3

    1997

  73. [81]

    Grab: A dataset of whole-body human grasping of objects

    Omid Taheri, Nima Ghorbani, Michael J Black, and Dim- itrios Tzionas. Grab: A dataset of whole-body human grasping of objects. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IV 16, pages 581–600. Springer, 2020. 2, 5, 6, 8

    2020

  74. [82]

    Goal: Generating 4d whole-body motion for hand-object grasping

    Omid Taheri, Vasileios Choutas, Michael J Black, and Dim- itrios Tzionas. Goal: Generating 4d whole-body motion for hand-object grasping. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 13263–13273, 2022. 2

    2022

  75. [83]

    Role-aware interaction generation from textual description

    Mikihiro Tanaka and Kent Fujiwara. Role-aware interaction generation from textual description. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 15953–15963, 2023. 2

    2023

  76. [84]

    Human motion diffusion model

    Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Daniel Cohen-Or, and Amit H Bermano. Human motion diffusion model. arXiv preprint arXiv:2209.14916 , 2022. 1, 2

    2022 arXiv

  77. [85]

    Con- sensus and formation control on se (3) for switching topolo- gies

    Johan Thunberg, Jorge Goncalves, and Xiaoming Hu. Con- sensus and formation control on se (3) for switching topolo- gies. Automatica, 66:109–121, 2016. 3

    2016

  78. [86]

    A lifting method for analyzing distributed synchronization on the unit sphere

    Johan Thunberg, Johan Markdahl, Florian Bernard, and Jorge Goncalves. A lifting method for analyzing distributed synchronization on the unit sphere. Automatica, 96:253– 258, 2018. 3

    2018

  79. [87]

    Dynamic controllers for column synchronization of rotation matrices: a qr-factorization approach

    Johan Thunberg, Johan Markdahl, and Jorge Goncalves. Dynamic controllers for column synchronization of rotation matrices: a qr-factorization approach. Automatica, 93:20– 25, 2018. 3

    2018

  80. [88]

    Towards semantic 3d hand-object interaction generation via functional text guidance, 2025

    Yongqi Tian, Xueyu Sun, Haoyuan He, Linji Hao, Ning Ding, and Caigui Jiang. Towards semantic 3d hand-object interaction generation via functional text guidance, 2025. 2

    2025

  81. [89]

    Mujoco: A physics engine for model-based control

    Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ international conference on intelligent robots and systems, pages 5026–5033. IEEE, 2012. 2

    2012

  82. [90]

    Unidexgrasp++: Im- proving dexterous grasping policy learning via geometry- aware curriculum and iterative generalist-specialist learn- ing

    Weikang Wan, Haoran Geng, Yun Liu, Zikang Shan, Yaodong Yang, Li Yi, and He Wang. Unidexgrasp++: Im- proving dexterous grasping policy learning via geometry- aware curriculum and iterative generalist-specialist learn- ing. In Proceedings of the IEEE/CVF International Con- fere...

    2023

  83. [91]

    Flocking of networked uncertain eu- ler–lagrange systems on directed graphs

    Hanlei Wang. Flocking of networked uncertain eu- ler–lagrange systems on directed graphs. Automatica, 49 (9):2774–2779, 2013. 3

    2013

  84. [92]

    Consensus of networked mechanical sys- tems with communication delays: A unified framework

    Hanlei Wang. Consensus of networked mechanical sys- tems with communication delays: A unified framework. IEEE Transactions on Automatic Control , 59(6):1571– 1576, 2014. 3

    2014

  85. [93]

    Synthesizing long-term 3d human mo- tion and interaction in 3d scenes

    Jiashun Wang, Huazhe Xu, Jingwei Xu, Sifei Liu, and Xiaolong Wang. Synthesizing long-term 3d human mo- tion and interaction in 3d scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9401–9411, 2021. 2

    2021

  86. [94]

    Scene- aware generative network for human motion synthesis

    Jingbo Wang, Sijie Yan, Bo Dai, and Dahua Lin. Scene- aware generative network for human motion synthesis. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 12206–12215, 2021. 2

    2021

  87. [95]

    Region- grasp: A novel task for contact region controllable hand grasp generation, 2024

    Yilin Wang, Chuan Guo, Li Cheng, and Hai Jiang. Region- grasp: A novel task for contact region controllable hand grasp generation, 2024. 2

    2024

  88. [96]

    Genh2r: Learning generalizable human- to-robot handover via scalable simulation demonstration and imitation

    Zifan Wang, Junyu Chen, Ziqing Chen, Pengwei Xie, Rui Chen, and Li Yi. Genh2r: Learning generalizable human- to-robot handover via scalable simulation demonstration and imitation. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 16...

    2024

  89. [97]

    Thor: Text to human-object inter- action diffusion via relation intervention

    Qianyang Wu, Ye Shi, Xiaoshui Huang, Jingyi Yu, Lan Xu, and Jingya Wang. Thor: Text to human-object inter- action diffusion via relation intervention. arXiv preprint arXiv:2403.11208, 2024. 2

    2024 arXiv

  90. [98]

    Human-object interaction from human-level instructions

    Zhen Wu, Jiaman Li, and C Karen Liu. Human-object interaction from human-level instructions. arXiv preprint arXiv:2406.17840, 2024. 2

    2024 arXiv

  91. [99]

    Bayesian diffusion models for 3d shape reconstruction

    Haiyang Xu, Yu Lei, Zeyuan Chen, Xiang Zhang, Yue Zhao, Yilin Wang, and Zhuowen Tu. Bayesian diffusion models for 3d shape reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10628–10638, 2024. 2, 3

    2024

  92. [100]

    Inter-x: Towards versatile human-human interac- tion analysis

    Liang Xu, Xintao Lv, Yichao Yan, Xin Jin, Shuwen Wu, Congsheng Xu, Yifan Liu, Yizhou Zhou, Fengyun Rao, Xingdong Sheng, Yunhui Liu, Wenjun Zeng, and Xiaokang Yang. Inter-x: Towards versatile human-human interac- tion analysis. In Proceedings of the IEEE/CVF conference on compu...

    2024

  93. [101]

    Regen- net: Towards human action-reaction synthesis

    Liang Xu, Yizhou Zhou, Yichao Yan, Xin Jin, Wenhan Zhu, Fengyun Rao, Xiaokang Yang, and Wenjun Zeng. Regen- net: Towards human action-reaction synthesis. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1759–1769, 2024. 2

    2024

  94. [102]

    Interdiff: Generating 3d human-object interactions with physics-informed diffusion

    Sirui Xu, Zhengyuan Li, Yu-Xiong Wang, and Liang-Yan Gui. Interdiff: Generating 3d human-object interactions with physics-informed diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 14928–14940, 2023. 2

    2023

  95. [103]

    Intermimic: Towards universal whole-body control for physics-based human-object interactions, 2025

    Sirui Xu, Hung Yu Ling, Yu-Xiong Wang, and Liang-Yan Gui. Intermimic: Towards universal whole-body control for physics-based human-object interactions, 2025. 2

    2025

  96. [104]

    Cpf: Learning a contact potential field to model the hand-object interaction

    Lixin Yang, Xinyu Zhan, Kailin Li, Wenqiang Xu, Jiefeng Li, and Cewu Lu. Cpf: Learning a contact potential field to model the hand-object interaction. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 11097–11106, 2021. 2

    2021

  97. [105]

    Oakink: A large-scale knowledge repos- itory for understanding hand-object interaction

    Lixin Yang, Kailin Li, Xinyu Zhan, Fei Wu, Anran Xu, Liu Liu, and Cewu Lu. Oakink: A large-scale knowledge repos- itory for understanding hand-object interaction. InProceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 20953–20962, 2022. 2

    2022

  98. [106]

    Generating human interaction motions in scenes with text control

    Hongwei Yi, Justus Thies, Michael J Black, Xue Bin Peng, and Davis Rempe. Generating human interaction motions in scenes with text control. In European Conference on Computer Vision, pages 246–263. Springer, 2025. 2

    2025

  99. [107]

    Physdiff: Physics-guided human motion diffusion model

    Ye Yuan, Jiaming Song, Umar Iqbal, Arash Vahdat, and Jan Kautz. Physdiff: Physics-guided human motion diffusion model. In Proceedings of the IEEE/CVF international con- ference on computer vision, pages 16010–16021, 2023. 2

    2023

  100. [108]

    Oakink2: A dataset of bimanual hands-object manipulation in complex task completion

    Xinyu Zhan, Lixin Yang, Yifei Zhao, Kangrui Mao, Han- lin Xu, Zenan Lin, Kailin Li, and Cewu Lu. Oakink2: A dataset of bimanual hands-object manipulation in complex task completion. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages ...

    2024

  101. [109]

    Core4d: A 4d human-object-human interac- tion dataset for collaborative object rearrangement

    Chengwen Zhang, Yun Liu, Ruofan Xing, Bingda Tang, and Li Yi. Core4d: A 4d human-object-human interac- tion dataset for collaborative object rearrangement. arXiv preprint arXiv:2406.19353, 2024. 2, 5, 7, 8

    2024 arXiv

  102. [110]

    Manipnet: neural manipulation synthesis with a hand-object spatial representation

    He Zhang, Yuting Ye, Takaaki Shiratori, and Taku Ko- mura. Manipnet: neural manipulation synthesis with a hand-object spatial representation. ACM Transactions on Graphics (ToG), 40(4):1–14, 2021. 2

    2021

  103. [111]

    Ar- tigrasp: Physically plausible synthesis of bi-manual dexter- ous grasping and articulation

    Hui Zhang, Sammy Christen, Zicong Fan, Luocheng Zheng, Jemin Hwangbo, Jie Song, and Otmar Hilliges. Ar- tigrasp: Physically plausible synthesis of bi-manual dexter- ous grasping and articulation. In 2024 International Con- ference on 3D Vision (3DV) , pages 235–246. IEEE, 2024. 2

    2024

  104. [112]

    Graspxl: Generating grasping motions for diverse objects at scale

    Hui Zhang, Sammy Christen, Zicong Fan, Otmar Hilliges, and Jie Song. Graspxl: Generating grasping motions for diverse objects at scale. In European Conference on Com- puter Vision, pages 386–403. Springer, 2025. 2

    2025

  105. [113]

    Manidext: Hand-object manipulation synthesis via continuous cor- respondence embeddings and residual-guided diffusion

    Jiajun Zhang, Yuxiang Zhang, Liang An, Mengcheng Li, Hongwen Zhang, Zonghai Hu, and Yebin Liu. Manidext: Hand-object manipulation synthesis via continuous cor- respondence embeddings and residual-guided diffusion. arXiv preprint arXiv:2409.09300, 2024. 2

    2024 arXiv

  106. [114]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 2

    2023

  107. [115]

    Motiondif- fuse: Text-driven human motion generation with diffusion model

    Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondif- fuse: Text-driven human motion generation with diffusion model. arXiv preprint arXiv:2208.15001, 2022. 2

    2022 arXiv

  108. [116]

    Bimart: A unified ap- proach for the synthesis of 3d bimanual interaction with ar- ticulated objects

    Wanyue Zhang, Rishabh Dabral, Vladislav Golyanik, Vasileios Choutas, Eduardo Alvarado, Thabo Beeler, Marc Habermann, and Christian Theobalt. Bimart: A unified ap- proach for the synthesis of 3d bimanual interaction with ar- ticulated objects. arXiv preprint arXiv:2412.05066, 2024. 2

    2024 arXiv

  109. [117]

    Couch: Towards controllable human-chair interactions

    Xiaohan Zhang, Bharat Lal Bhatnagar, Sebastian Starke, Vladimir Guzov, and Gerard Pons-Moll. Couch: Towards controllable human-chair interactions. InEuropean Confer- ence on Computer Vision, pages 518–535. Springer, 2022. 2

    2022

  110. [118]

    Ood-hoi: Text-driven 3d whole-body human-object interactions generation be- yond training domains, 2024

    Yixuan Zhang, Hui Yang, Chuanchen Luo, Junran Peng, Yuxi Wang, and Zhaoxiang Zhang. Ood-hoi: Text-driven 3d whole-body human-object interactions generation be- yond training domains, 2024. 2

    2024

  111. [119]

    Motion-x++: A large-scale multi- modal 3d whole-body human motion dataset, 2025

    Yuhong Zhang, Jing Lin, Ailing Zeng, Guanlin Wu, Shun- lin Lu, Yurong Fu, Yuanhao Cai, Ruimao Zhang, Haoqian Wang, and Lei Zhang. Motion-x++: A large-scale multi- modal 3d whole-body human motion dataset, 2025. 2

    2025

  112. [120]

    Compositional human-scene interaction synthesis with semantic control

    Kaifeng Zhao, Shaofei Wang, Yan Zhang, Thabo Beeler, and Siyu Tang. Compositional human-scene interaction synthesis with semantic control. In European Conference on Computer Vision, pages 311–327. Springer, 2022. 2

    2022

  113. [121]

    Synthesizing diverse human motions in 3d indoor scenes

    Kaifeng Zhao, Yan Zhang, Shaofei Wang, Thabo Beeler, and Siyu Tang. Synthesizing diverse human motions in 3d indoor scenes. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision, pages 14738– 14749, 2023. 2

    2023

  114. [122]

    Cams: Canonicalized manipulation spaces for category-level functional hand-object manipulation syn- thesis

    Juntian Zheng, Qingyuan Zheng, Lixing Fang, Yun Liu, and Li Yi. Cams: Canonicalized manipulation spaces for category-level functional hand-object manipulation syn- thesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 585–594,

  115. [123]

    Smoodi: Stylized motion diffusion model

    Lei Zhong, Yiming Xie, Varun Jampani, Deqing Sun, and Huaizu Jiang. Smoodi: Stylized motion diffusion model. In European Conference on Computer Vision, pages 405–421. Springer, 2025. 2, 3

    2025

  116. [124]

    Learning diverse bimanual dexterous manipula- tion skills from human demonstrations

    Bohan Zhou, Haoqi Yuan, Yuhui Fu, and Zongqing Lu. Learning diverse bimanual dexterous manipula- tion skills from human demonstrations. arXiv preprint arXiv:2410.02477, 2024. 2

    2024 arXiv

  117. [125]

    Toch: Spatio-temporal object-to-hand correspondence for motion refinement

    Keyang Zhou, Bharat Lal Bhatnagar, Jan Eric Lenssen, and Gerard Pons-Moll. Toch: Spatio-temporal object-to-hand correspondence for motion refinement. In European Con- ference on Computer Vision , pages 1–19. Springer, 2022. 2

    2022

  118. [126]

    On the continuity of rotation representations in neural networks

    Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, and Hao Li. On the continuity of rotation representations in neural networks. IEEE, 2019. 26

    2019

  119. [127]

    Velocity-free leader–follower cooperative attitude tracking of multiple rigid bodies on so(3)

    Yao Zou and Ziyang Meng. Velocity-free leader–follower cooperative attitude tracking of multiple rigid bodies on so(3). IEEE Transactions on Cybernetics , 49(12):4078– 4089, 2019. 3 Appendix A. Details of Our Two Synchronization Mech- anisms A.1. Diffusion Model Basics Diffusi...

    2019

  120. [128]

    Illustrate how to define a set of alignment scores fea- turing synchronization analogous to the commonly used data sample scores in diffusion models, and derive the cor- responding loss term Lalign in Section 3.5

  121. [129]

    Before we start, let’s derive some commonly used for- mulas in diffusion models that we will need in our proof, which the readers might not be familiar with

    Prove that the explicit synchronization formulas in Section 3.5 are equivalent to maximum total likelihood sam- pling on the newly computed Gaussian distribution, where data sample scores and alignment scores are jointly consid- ered. Before we start, let’s derive some commonl...

  122. [130]

    Their mean values are referred to asµt and ˆµt, and the standard variance σt is a predefined constant in DDPM [30]

    is given by DKL(N (µ1, σ2 1)||N (µ2, σ2 2)) = log σ2 σ1 + σ2 1 + ∥µ1 − µ2∥2 2 2σ2 2 − 1 2 (12) In our derivation, q(xt−1 | xt, x0) means the ground- truth reverse process distribution, whilepθ(xt−1 | xt) is our predicted distribution in stepwise denoising. Their mean values ar...

  123. [131]

    (14) Next, let’s prove the equivalence between the explicit synchronization formulas and maximum total likelihood sampling in inference

    Summing up for all such pairs (b1, b2), and removing the coefficients, we can derive the alignment loss Lalign = X j1,j2∈[1,m],j1̸=j2 ∥ˆxoj2 →oj1 − rel(ˆxoj1 , ˆxoj2 )∥2 2 + X i∈[1,n],j∈[1,m] ∥ˆxhi→oj − rel(ˆxoj , ˆxhi )∥2 2. (14) Next, let’s prove the equivalence between the ...

  124. [132]

    λ0 = 1 2σ2 , λ1 = λ2 = · · ·= λm−1 = λ m−1

    For individual motions of rigid body oj(j ∈ [1, m]) (Here we assume that m > 1, otherwise there is no need for explicit synchronization on this part), relevant pairs of (ak, bk) consist of (ˆxoj′ , ˆxoj →oj′ )(j ′ ̸= j). λ0 = 1 2σ2 , λ1 = λ2 = · · ·= λm−1 = λ m−1. Here λ is an...

  125. [133]

    λ0 = 1 2σ2 , λ1 = λ2 = · · ·= λm = λ m

    For individual motions of articulated skele- ton hi(i ∈ [1, n]), relevant pairs of (ak, bk) consist of (ˆxoj , ˆxhi→oj )(j ∈ [1, m]). λ0 = 1 2σ2 , λ1 = λ2 = · · ·= λm = λ m. Therefore, ˆµ ′ hi = 1 1 2σ2 + λ  λ0 ˆµhi + X j∈[1,m] λ mcomb ˆxoj , ˆxhi→oj   = 1 1 + 2σ2λ ˆµoj + ...

  126. [134]

    w/o exp sync

    For relative motions, there is only one relevant pair of (ak, bk), where ak and bk are both individual motions, which can obtain the relative motion through relative com- position. Here λ0 = 1 2σ2 , λ1 = λ. Therefore, ˆµ ′ oj →oj′ = 1 1 2σ2 + λ λ0 ˆµoj →oj′ + λ1rel ˆxoj′ , ˆxo...

  127. [135]

    Lack of Articulation-Aware Modeling. Our method models articulated objects (such as those in OAKINK2 [108]) as part-wise rigid body individuals di- rectly and coordinates their motions without leveraging their intrinsic articulations. Integrating these articulations into multi...

  128. [136]

    As body number increases, the time consumption for the cal- culation of alignment loss and explicit synchronization step grows quadratically

    High-cost of Explicit Synchronization Step. As body number increases, the time consumption for the cal- culation of alignment loss and explicit synchronization step grows quadratically. Note that for multi-body HOI synthe- sis, not all pairwise relationships are necessary. A p...

  129. [137]

    Unlike methods that utilize true physical simulations, our approach cannot guarantee physical truthfulness

    Lack of Physically Accurate Guarantees. Unlike methods that utilize true physical simulations, our approach cannot guarantee physical truthfulness. In many cases, mi- nor errors can be observed in the supplementary videos, but these small discrepancies may be sufficient to cau...

  130. [138]

    Limitations in Pure Multi-human Interaction Syn- thesis. Since the relative representations need to be gener- ated in the coordinate systems of rigid bodies, whose mo- tions can be represented by translations and rotations, our method may not be directly adapted for pure multi...

  131. [139]

    one model for all datasets

    Does SyncDiff lose the flexibility of “one model for all datasets”? Due to the scarcity of mocap datasets, as well as the need to adopt models into multiple scenarios, sometimes it is necessary to merge data from multiple datasets for train- ing. However, due to the fixed numb...

  132. [140]

    Why doesn’t SyncDiff use the 6 DoF rotation rep- resentation that is widely employed in the current mo- tion synthesis methods? To represent the rotation R ∈ SO(3) of an object, the 6 DoF representation proposed in [126] concatenates the first two columns of the 3 × 3 rotation...

  133. [141]

    synchronization

    Is the explicit synchronization during inference in SyncDiff inspired by the guidance strategy in Guided Motion Diffusion (GMD) [37]? As is stated in Section 2.2, several prominent works in- ject external priors or constraints into synthesized results by performing linear fusi...

  134. [142]

    a set of joints whose motion can be reconstructed based on joint information and shape parameter β

    Can the articulated skeletons in SyncDiff be other body parts besides hands and the full body? In Section 3.1, our definition of articulated skeletons is “a set of joints whose motion can be reconstructed based on joint information and shape parameter β”. To extend SyncDiff to...

  135. [143]

    For ex- ample, if we want to perform HOI motion synthesis for feet, we need to collect high-quality data such as kicking a ball, putting on shoes, ice skating, etc

    Collect relevant high-precision mocap data. For ex- ample, if we want to perform HOI motion synthesis for feet, we need to collect high-quality data such as kicking a ball, putting on shoes, ice skating, etc

  136. [144]

    Design a data representation for motion reconstruction based on joint information and shape parameter β, similar to MANO for hands or SMPL-X for human bodies

  137. [145]

    Does SyncDiff’s ignorance of affordance result in the item being manipulated incorrectly? For example, the cup was not picked up from the handle position? SyncDiff does not consider affordance because the five datasets we use do not provide affordance for rigid objects at all....

  138. [146]

    In fact, experiments reveal that this approach causes ground-truth test splits to underperform some base- lines

    Is the metric RA (Recognition Accuracy) fair? Since the discriminator is trained on the combination of train, val, and test folders, is there any risk of overfit- ting? Training the classifier solely on the training split would introduce a more serious issue: motion synthesis ...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.