Pith. sign in

REVIEW 5 major objections 6 minor 74 references

IKMo: Image-Keyframed Motion Generation with Trajectory-Pose Conditioned Motion Diffusion Model

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that decoupling trajectory and keyframe-pose control, rather than fusing them, improves every controlled-motion metric and enables image-keyframed generation.

desk verdict Useful control method for motion generation, but the paper's central claim that decoupling drives the gains is not actually isolated in the experiments. read the letter →

arxiv 2505.21146 v1 pith:BWG2SV6H submitted 2025-05-27 cs.GR cs.CV

Yang Zhao , Yan Zhang , Xubo Yang This is my paper
classification cs.GRcs.CV
keywords humanmotiongenerationdiffusionmodelkeyframeposecontroltrajectoryimage-keyframedmultimodallargelanguagemodelscontrollableNet
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that when a motion diffusion model is given both a trajectory and keyframe poses, the two constraints should be handled through separate pathways rather than merged into one condition. It claims that this decoupling, implemented with a gradient-based Motion Optimization stage and a Motion ControlNet with parallel trajectory and pose encoders, yields state-of-the-art results under trajectory-plus-keyframe control on both HumanML3D and KIT-ML, improving trajectory following, keyframe pose matching, and motion realism together. The same model is wrapped in a user-facing pipeline, IKMo, in which multimodal language-model agents turn user images and text into 3D keyframe poses, a motion description, and a trajectory. A ten-participant user study supports the claim that image keyframes make generated motion closer to what users expect. If the claims hold, the payoff is practical: animators and general users can specify motion style and spatial path from images and simple text, without manually posing a skeleton or drawing 3D curves.

What carries the argument

The mechanism that carries the argument is decoupled trajectory-pose conditioning with two parallel control pathways. In Motion Optimization, the noised motion $x_t$ is updated by gradient descent on $\alpha L_{\mathrm{traj}}(x_t, c_{\mathrm{traj}}) + (1-\alpha) L_{\mathrm{pose}}(x_t, c_{\mathrm{pose}})$, with $\alpha = L_{\mathrm{traj}}/(L_{\mathrm{traj}} + L_{\mathrm{pose}})$ balancing the two error scales; before the pose loss is computed, the constraint pose is translated so its root aligns with the trajectory constraint or with the current motion's root projection. In Motion Control, a Trajectory Encoder and a Pose Encoder process the two conditions in parallel, and their fused features are injected through a Motion ControlNet, defined as a trainable copy of the transformer encoder of the diffusion backbone, into the denoising network. The image front end is a multi-agent MLLM system whose Motion Design Agent recovers SMPL poses via TokenHMR, maps them to a canonical 22-joint HumanML3D-compatible format, and whose Trajectory Planning Agent generates parameterized curves; training with small random global rotations on sampled poses is what makes poses extracted from arbitrary photos compatible with HumanML3D motion data.

What would settle it

Run a head-to-head ablation that replaces the parallel Trajectory and Pose encoders with a single fused encoder while keeping Motion Optimization, data, and training budget identical: if the fused version matches IKMo's Trajectory Error and Pose Dist on HumanML3D, the central claim that decoupling causes the gains would be refuted.

Watch

Extended reading notes

Core claim

IKMo's central claim is that trajectory constraints and keyframe-pose constraints have different semantics—trajectory is the absolute spatial path of the root joint, while a keyframe pose is a relative full-body configuration—so a diffusion model that fuses them into one representation before conditioning sacrifices both control accuracy and pose fidelity. The paper therefore keeps the two signals decoupled through two stages: Motion Optimization, which perturbs the noised motion by gradient descent on a weighted sum of a trajectory loss and an alignment-aware pose loss, and Motion Control, in which separate Trajectory and Pose encoders feed features into a Motion ControlNet attached to the denoising backbone. On HumanML3D and KIT-ML, the method reports the best numbers on every metric under the pelvis+keyframes setting, cutting Trajectory Error to about 5% of OmniControl's and CondMDI's values, Location Error to 4–11% of theirs, and Average Error to 15–18% of theirs, with a new Pose Dist metric also favoring IKMo. For image input, the paper claims the first framework to use human images as keyframe cues: a multi-agent MLLM system extracts 3D poses with TokenHMR, maps them to a HumanML3D-compatible 22-joint format, plans parameterized trajectories, and generates a text prompt, and the user study indicates this image-keyframed pipeline produces motions closer to user expectations than randomly sampled keyframe poses.

Load-bearing premise

The image-keyframed pipeline assumes that 3D poses recovered from user photographs by TokenHMR and mapped to the HumanML3D coordinate system are compatible with training-time motion poses up to small random rotations; if real photos bring scale, skeleton, or camera-projection mismatches beyond those rotations, the visual-input front end degrades.

Editorial extensions

If this is right

  • Under the pelvis+keyframes setting, the paper reports state-of-the-art values on every metric on both HumanML3D and KIT-ML, with Trajectory Error at roughly 5% of OmniControl's and CondMDI's, Location Error at 4–11% of theirs, and Average Error at 15–18% of theirs.
  • Image keyframes extracted from user photos or video frames can specify motion style and timing, so animation control no longer requires manually designing 3D trajectories or posing a full skeleton.
  • Training with small random pose rotations makes the model robust to the arbitrary global orientation of image-derived poses, preserving FID and R-Precision when rotations are applied at evaluation.
  • The ablations show the Motion ControlNet is the main driver of keyframe pose fidelity, with Pose Dist degrading 7.72 times without it, while Motion Optimization cuts trajectory, location, and average errors by over 90%.
  • The MLLM agent pre-processing yields motions rated more similar to the reference video than a pose-sampling baseline in the ten-participant user study.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The decoupling principle should transfer to other mixed control signals in generative animation, such as absolute scene or contact positions combined with relative body poses, where joint fusion would again entangle coordinate frames; a direct test would swap trajectory and keyframe inputs for scene and contact inputs while keeping the two-stage parallel design.
  • The scale of the reported gains suggests that prior joint-conditioning models lose capacity to representational conflict rather than to insufficient data, a prediction that could be checked by scaling up the baselines' encoders and seeing whether the gap persists.
  • Because the multi-agent front end already outputs structured poses, trajectories, and text, a natural extension is coordinating several characters or a moving camera from the same structured specification.
  • Evaluating the image front end on real photographs with known mocap ground truth would isolate pose-extraction error from generation error and test whether the small-rotation compatibility assumption holds outside HumanML3D.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes IKMo, a motion diffusion framework for simultaneous trajectory and keyframe-pose control. The core idea is to decouple trajectory and pose conditioning: a test-time Motion Optimization stage gradient-perturbs the noised motion using a dynamic combination of L_traj and L_pose, and a Motion ControlNet injects features from separate Trajectory and Pose encoders. On top of this, an MLLM-based multi-agent system converts user images and texts into a motion description, keyframe poses, and a trajectory. Experiments on HumanML3D and KIT-ML report state-of-the-art control metrics, ablations show that both stages contribute, and a 10-participant user study is used to support the image-keyframed pipeline.

Significance. If the central claim were firmly established, the paper would be a useful step toward flexible motion control from heterogeneous constraints, and the image-keyframed interface is practically appealing. The paper is transparent about its implementation details and ablates both proposed modules, and the supplementary description of the MLLM agents is helpful. However, the main conceptual claim—that decoupling trajectory and pose processing is what drives the reported improvements—is not isolated by any experiment, and the headline control metrics largely coincide with the inference-time optimization objective. The user study is underpowered and compares against a strawman baseline rather than the actual SOTA keyframe method. The paper is best read as a system paper with a promising pipeline; the scientific claim about decoupling needs additional evidence.

major comments (5)
  1. [§4.3, §5.2, Table 4] The central claim that decoupling trajectory and pose processing is responsible for the performance gains is not isolated by any experiment. Relative to OmniControl and CondMDI, IKMo changes several factors at once: parallel Trajectory/Pose encoders with a fused ControlNet (§4.3), a test-time gradient-perturbation stage with dynamic weighting (Eqs. 9–10), and random-rotation pose augmentation. Table 4 ablates Motion Optimization and Motion ControlNet, but no variant replaces the parallel encoders with a coupled or fused encoder while keeping the optimization module fixed. The large margins in Tables 1–2 therefore cannot be attributed specifically to decoupling; please add such an ablation (for example, a single encoder over concatenated trajectory and pose features, with the same optimization and augmentation) and report all metrics for that variant.
  2. [§5.1, §B.3, Eqs. 5–10] The control metrics Traj.err, Loc.err, Avg.err, and the new Pose Dist are L2 distances between generated positions and the constraint values, which are essentially the same quantities that Eqs. 5, 6, and 9 minimize at inference through the Motion Optimization module. Consequently, the improvement from enabling Motion Optimization in Table 4 (e.g., Traj.err from 0.3006 to 0.0246) is partly by construction, and it is unclear how much of the reported margin over baselines reflects genuine control quality rather than the choice of evaluation metric matching the optimized objective. Please report, at least for the ablation variants, the metrics that are not direct targets of the inference-time loss (FID, R-precision, Foot skating, Diversity), and consider reporting the constraint error before versus after the gradient perturbation.
  3. [Appendix C.1, Table 1] The claim in Section 5.2 that OmniControl 'experiences a significant drop in performance when keyframe conditions are added' is based on comparing the pelvis-only OmniControl result taken from the original paper with the pelvis+keyframes result re-evaluated under the authors' unified setting; Appendix C.1 further states that the better of the two re-evaluations was reported. This mixes experimental settings and selects the favorable baseline number, so it does not support the conclusion that a single control mechanism is inherently suboptimal. All baselines and conditions should be evaluated under one identical protocol, or the causal interpretation of this cross-setting comparison should be removed.
  4. [§5.5, Table 6] The user study does not compare IKMo against the actual keyframe-based SOTA method CondMDI; the 'CondMDI-style baseline' is the authors' own motion diffusion model with randomly sampled keyframe poses from HumanML3D. With only 10 participants and two videos, this design cannot support the abstract's claim that the MLLM-based agent pre-processing 'makes generated motion more in line with users' expectation.' A comparison against the actual CondMDI interface, or at least a content-matched baseline using keyframes derived from the same images, is needed, together with a larger sample and appropriate significance testing.
  5. [§4.3, §A.2, Limitations] The image-to-motion front end assumes that TokenHMR poses mapped into the HumanML3D convention (Eq. 11) are compatible with training-time poses up to small random rotations. The Limitations paragraph appropriately acknowledges that intermediate pose extraction may introduce errors, but no quantitative test of the full image-to-motion pipeline on real user photos (including scale and skeleton mismatches) is provided. The rotation augmentation addresses only global rotation, not the other failure modes; either add such an evaluation or narrow the claims about robustness of the IKMo pipeline.
minor comments (6)
  1. [Abstract] The abstract contains grammatical errors: 'We conducts' should be 'We conduct', 'users' exception' should be 'users' expectation', and the phrasing 'meets more in line' is ungrammatical.
  2. [Tables 1 and 5, Section 5.2] The labels 'Ours (on rotations)' and 'Ours (w/ rotations)' are used inconsistently; please adopt a single naming convention throughout.
  3. [Appendix B.3] The definition of Pose Dist should specify which joints are included in the average (all 22 joints or only controlled joints) and whether the pelvis-centered transform is applied identically to generated and reference poses; the current wording is ambiguous.
  4. [Table 3] CondMDI is not evaluated under the new Pose Dist metric, so the claim of 'best performance across all metrics' is not fully supported for the pose-similarity metric; if CondMDI's representation is incompatible, this should be stated explicitly in the main text.
  5. [§2.2] The statement that CondMDI 'suffers from larger trajectory control errors and is similarly difficult to manually configure for effective guidance' is asserted without a citation; it should be supported by the quantitative results or softened.
  6. [Table 5] The no-rotation model has better FID and lower control errors on the standard evaluation (0.177 FID, Traj.err 0.0176) than the rotation-augmented model (0.239 FID, Traj.err 0.0246), yet the rotation-augmented version is used in all subsequent comparisons; the paper should explicitly discuss this trade-off.

Circularity Check

1 steps flagged · score 6.0 of 10

Control-accuracy predictions reduce to the inference-time objective: Traj.err./Loc.err./Avg.err./Pose Dist are L2 (or thresholded L2) versions of L_traj/L_pose that Eq. (9) explicitly minimizes, so the SOTA margins on those metrics are not independent evidence for the decoupled design.

  1. fitted input called prediction [Section 4.3, Eq. (9) with Eqs. (5)-(6); Section 5.1 and B.2-B.3; Table 4]
    "In the first stage, Motion Optimization, we jointly utilize the pose analysis function and trajectory analysis function to construct a gradient perturbation term that updates the noised motion 𝑥𝑡. The perturbation is defined as: 𝑥𝑡 = 𝑥𝑡 − 𝜏∇𝑥𝑡(𝛼·𝐿traj(𝑥𝑡,𝑐traj)+(1−𝛼)·𝐿pose(𝑥𝑡,𝑐pose)) (9). ... To further quantify the accuracy of pose control, we introduce a new metric, Pose Dist, which evaluates the average Euclidean distance between pelvis-centered generated and reference poses at controlled frames."

    The reported control metrics are the same L2 quantities that Eq. (9) minimizes at test time. L_traj (Eq. 5) is the L2 distance between generated root positions and trajectory constraints; L_pose (Eq. 6) is the L2 distance between aligned generated and reference keyframe poses. The metrics in Tables 1-4 are defined in B.2-B.3 as keyframe location error and mean Euclidean distance between generated/reference positions (Traj.err., Loc.err., Avg.err.) and pelvis-centered L2 pose distance (Pose Dist), i.e., the same or thresholded versions of those losses. Thus the claimed SOTA margins on these metrics, and the 90% drop when Motion Optimization is removed (Table 4), largely verify that the test-time optimizer minimized its own objective.

full rationale

The only construction-level circularity is the control-metric evaluation: Traj.err., Loc.err., Avg.err., and Pose Dist are L2 distances between the generated motion and the same constraint signals that Eq. (9) explicitly gradient-descents at inference. Reporting these as evidence of superior controllability is partly a check that Motion Optimization optimized its own objective, not a test of the decoupled-encoder claim. The paper's independent metrics (FID, R-Precision, Diversity, user study) and its qualitative comparisons are not circular. There are no load-bearing self-citations, imported uniqueness theorems, or citation-smuggled ansatze. The limitation paragraph honestly acknowledges imperfect keyframe matching and intermediate pose-extraction errors, and the absent coupled-variant ablation is a controlled-comparison/correctness concern rather than a circularity. Because the central controllability numbers reduce by construction while independent realism/text-alignment content remains, the appropriate circularity score is 6.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central architecture rests on the assumption that trajectory and pose should be decoupled, on the inference-time gradient heuristic borrowed from OmniControl, and on the compatibility of image-derived poses with the HumanML3D coordinate frame. The only invented ledger entry is the Pose Dist metric. No physical entities are introduced.

free parameters (3)
  • guidance strength tau
    Step size for the inference-time gradient perturbation in Eq. 9. The value is not reported and is chosen by hand; it controls the tradeoff between trajectory and pose fidelity.
  • small random rotation scale
    Magnitude of global rotations applied to sampled keyframe poses during training (Section 4.3). The scale is unspecified and chosen ad hoc to make the model robust to camera-induced rotations.
  • control sparsity levels = 1, 2, 5, 49, 196
    Chosen evaluation densities inherited from OmniControl (Appendix B.5). Metrics are averaged over these levels, so the reported control-error numbers depend on this hand-selected protocol.
assumptions (4)
  • domain assumption Decoupling trajectory and pose control signals is beneficial, while unified representation distorts pose structure and reduces control precision.
    Central design premise argued in Sections 2.2 and 4. It is plausible but not independently proven; the ablation only removes whole modules, not the decoupling itself.
  • domain assumption A pelvis-centered pose representation plus small random global rotations during training makes image-derived 3D poses usable by the model.
    Assumed in Sections 4.3 and A.2. If real extracted poses have scale, skeleton, or projection mismatches beyond these rotations, the image-keyframed front end fails.
  • domain assumption Gradient perturbation of the noised motion x_t using L_traj and L_pose improves the final clean motion.
    Inference-time guidance heuristic taken from OmniControl and extended to pose. It is not derived from diffusion theory and is validated only empirically.
  • domain assumption The baseline comparisons are fair despite differences in evaluation protocols and re-implementation.
    Section C.1 states that original OmniControl numbers are used for the pelvis-only rows and that CondMDI was adapted to the authors' pipeline. Without released evaluation code, this fairness cannot be checked.
invented entities (1)
  • Pose Dist metric
    purpose: New evaluation metric that measures pelvis-centered average Euclidean distance between generated and reference keyframe poses.
    Introduced in Sections 5.1 and B.3. It is defined by the authors, has no external validation, and is used to compare against OmniControl while CondMDI is excluded due to representation mismatch.

how reviews work

0 comments
Cite this review

Pith. "Pith review of IKMo: Image-Keyframed Motion Generation with Trajectory-Pose Conditioned Motion Diffusion Model." pith.science (2026). https://pith.science/paper/BWG2SV6H

@misc{pith2026250521146,
  author       = {Pith},
  title        = {Pith review of: IKMo: Image-Keyframed Motion Generation with Trajectory-Pose Conditioned Motion Diffusion Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BWG2SV6H}},
  note         = {Machine review of arXiv:2505.21146}
}
read the original abstract

Existing human motion generation methods with trajectory and pose inputs operate global processing on both modalities, leading to suboptimal outputs. In this paper, we propose IKMo, an image-keyframed motion generation method based on the diffusion model with trajectory and pose being decoupled. The trajectory and pose inputs go through a two-stage conditioning framework. In the first stage, the dedicated optimization module is applied to refine inputs. In the second stage, trajectory and pose are encoded via a Trajectory Encoder and a Pose Encoder in parallel. Then, motion with high spatial and semantic fidelity is guided by a motion ControlNet, which processes the fused trajectory and pose data. Experiment results based on HumanML3D and KIT-ML datasets demonstrate that the proposed method outperforms state-of-the-art on all metrics under trajectory-keyframe constraints. In addition, MLLM-based agents are implemented to pre-process model inputs. Given texts and keyframe images from users, the agents extract motion descriptions, keyframe poses, and trajectories as the optimized inputs into the motion generation model. We conducts a user study with 10 participants. The experiment results prove that the MLLM-based agents pre-processing makes generated motion more in line with users' expectation. We believe that the proposed method improves both the fidelity and controllability of motion generation by the diffusion model.

Figures

Figures reproduced from arXiv: 2505.21146 by the authors.

Figure 1
Figure 1. We propose a human motion diffusion model that realizes simultaneous control via trajectory and keyframe pose [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. (a) The overall pipeline of IKMo. Given an input image and textual requirement, our MLLM-based multi-agent system outputs a motion configuration consisting of a motion description, keyframe poses, and trajectory coordinates. This configuration is then fed into our Conditioned Motion Diffusion Model to generate the final human motion. (b) Details of the Conditioned Motion Diffusion Model. The model predicts a clean m… view at source ↗
Figure 3
Figure 3. Qualitative Results. All input images are generated by Doubao. Colored entity frames represent keyframes, while gray [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Qualitative results using video/text inputs. Both [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Ablation results. All input images are generated by [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

74 extracted references · 36 canonical work pages

  1. [1]

    Zhongang Cai, Jianping Jiang, Zhongfei Qing, Xinying Guo, Mingyuan Zhang, Zhengyu Lin, Haiyi Mei, Chen Wei, Ruisi Wang, Wanqi Yin, et al. 2024. Digital life project: Autonomous 3d characters with social intelligence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 582–592

    work page 2024

  2. [2]

    Pablo Cervantes, Yusuke Sekikawa, Ikuro Sato, and Koichi Shinoda. 2022. Implicit neural representations for variable length human motion generation. InEuropean Conference on Computer Vision . Springer, 356–372

    work page 2022

  3. [3]

    Caroline Chan, Shiry Ginosar, Tinghui Zhou, and Alexei A Efros. 2019. Everybody dance now. In Proceedings of the IEEE/CVF international conference on computer vision. 5933–5942

    work page 2019

  4. [4]

    Bohong Chen, Yumeng Li, Yao-Xiang Ding, Tianjia Shao, and Kun Zhou. 2024. Enabling synergistic full-body control in prompt-based co-speech motion gen- eration. In Proceedings of the 32nd ACM International Conference on Multimedia . 6774–6783

    work page 2024

  5. [5]

    Ling-Hao Chen, Shunlin Lu, Ailing Zeng, Hao Zhang, Benyou Wang, Ruimao Zhang, and Lei Zhang. 2024. Motionllm: Understanding human behaviors from human motions and videos. arXiv preprint arXiv:2405.20340 (2024)

    arXiv 2024

  6. [6]

    Kiran Chhatre, Nikos Athanasiou, Giorgio Becherini, Christopher Peters, Michael J Black, Timo Bolkart, et al . 2024. Emotional speech-driven 3d body animation via disentangled latent diffusion. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition . 1942–1953

    2024

  7. [7]

    Baptiste Chopin, Hao Tang, and Mohamed Daoudi. 2024. Bipartite graph diffusion model for human interaction generation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision . 5333–5342

    work page 2024

  8. [8]

    Setareh Cohan, Guy Tevet, Daniele Reda, Xue Bin Peng, and Michiel van de Panne

    work page

Show all 74 references
  1. [9]

    Sisi Dai, Wenhao Li, Haowen Sun, Haibin Huang, Chongyang Ma, Hui Huang, Kai Xu, and Ruizhen Hu. 2024. Interfusion: Text-driven generation of 3d human- object interaction. In European Conference on Computer Vision . Springer, 18–35

    2024

  2. [10]

    Bruno Degardin, Joao Neves, Vasco Lopes, Joao Brito, Ehsan Yaghoubi, and Hugo Proença. 2022. Generative adversarial graph convolutional networks for human action synthesis. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 1150–1159

    2022

  3. [11]

    Christian Diller and Angela Dai. 2024. Cg-hoi: Contact-guided 3d human-object interaction generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 19888–19901

    2024

  4. [12]

    Sai Kumar Dwivedi, Yu Sun, Priyanka Patel, Yao Feng, and Michael J Black. 2024. Tokenhmr: Advancing human mesh recovery with a tokenized pose representa- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1323–1333

    2024

  5. [13]

    Jiawei Gao, Ziqin Wang, Zeqi Xiao, Jingbo Wang, Tai Wang, Jinkun Cao, Xiaolin Hu, Si Liu, Jifeng Dai, and Jiangmiao Pang. 2024. Coohoi: Learning cooperative human-object interaction with manipulated object dynamics. Advances in Neural Information Processing Systems 37 (2024), ...

    2024

  6. [14]

    Chuan Guo, Yuxuan Mu, Muhammad Gohar Javed, Sen Wang, and Li Cheng. 2024. Momask: Generative masked modeling of 3d human motions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 1900–1910

    2024

  7. [15]

    Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng

  8. [16]

    Chuan Guo, Xinxin Zuo, Sen Wang, and Li Cheng. 2022. Tm2t: Stochastic and tokenized modeling for the reciprocal generation of 3d human motions and texts. In European Conference on Computer Vision . Springer, 580–597

    2022

  9. [17]

    Chuan Guo, Xinxin Zuo, Sen Wang, Shihao Zou, Qingyao Sun, Annan Deng, Minglun Gong, and Li Cheng. 2020. Action2motion: Conditioned generation of 3d human motions. In Proceedings of the 28th ACM International Conference on Multimedia. 2021–2029

    2020

  10. [18]

    Mohamed Hassan, Partha Ghosh, Joachim Tesch, Dimitrios Tzionas, and Michael J Black. 2021. Populating 3D scenes by learning human-scene interaction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14708–14718

    2021

  11. [19]

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. Advances in neural information processing systems 33 (2020), 6840–6851

    2020

  12. [20]

    Li Hu. 2024. Animate anyone: Consistent and controllable image-to-video syn- thesis for character animation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 8153–8163

    2024

  13. [21]

    Siyuan Huang, Zan Wang, Puhao Li, Baoxiong Jia, Tengyu Liu, Yixin Zhu, Wei Liang, and Song-Chun Zhu. 2023. Diffusion-based generation, optimization, and planning in 3d scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 16750–16761

    2023

  14. [22]

    Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, and Tao Chen. 2023. Mo- tiongpt: Human motion as a foreign language. Advances in Neural Information Processing Systems 36 (2023), 20067–20079

    2023

  15. [23]

    Nan Jiang, Zimo He, Zi Wang, Hongjie Li, Yixin Chen, Siyuan Huang, and Yixin Zhu. 2024. Autonomous character-scene interaction synthesis from text instruc- tion. In SIGGRAPH Asia 2024 Conference Papers . 1–11

    2024

  16. [24]

    Korrawe Karunratanakul, Konpat Preechakul, Supasorn Suwajanakorn, and Siyu Tang. 2023. Guided motion diffusion for controllable human motion synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 2151– 2162

    2023

  17. [25]

    Manuel Kaufmann, Emre Aksan, Jie Song, Fabrizio Pece, Remo Ziegler, and Otmar Hilliges. 2020. Convolutional autoencoders for human motion infilling. In 2020 International Conference on 3D Vision (3DV) . IEEE, 918–927

    2020

  18. [26]

    Jihoon Kim, Jiseob Kim, and Sungjoon Choi. 2023. Flame: Free-form language- based motion synthesis & editing. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 8255–8263

    2023

  19. [27]

    Buyu Li, Yongchi Zhao, Shi Zhelun, and Lu Sheng. 2022. Danceformer: Music con- ditioned 3d dance generation with parametric motion transformer. InProceedings of the AAAI Conference on Artificial Intelligence , Vol. 36. 1272–1279

    2022

  20. [28]

    Han Liang, Wenqian Zhang, Wenxuan Li, Jingyi Yu, and Lan Xu. 2024. Intergen: Diffusion-based multi-human motion generation under complex interactions. International Journal of Computer Vision 132, 9 (2024), 3463–3483

    2024

  21. [29]

    Donggeun Lim, Cheongi Jeong, and Young Min Kim. 2023. Mammos: Mapping multiple human motion with scene understanding and natural interactions. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 4278– 4287

    2023

  22. [30]

    Haiyang Liu, Zihao Zhu, Giorgio Becherini, Yichen Peng, Mingyang Su, You Zhou, Xuefei Zhe, Naoya Iwamoto, Bo Zheng, and Michael J Black. 2024. EMAGE: Towards unified holistic co-speech gesture generation via expressive masked audio gesture modeling. In Proceedings of the IEEE/...

    2024

  23. [31]

    Xinpeng Liu, Haowen Hou, Yanchao Yang, Yong-Lu Li, and Cewu Lu. 2024. Revisit human-scene interaction via space occupancy. In European Conference on Computer Vision. Springer, 1–19

    2024

  24. [32]

    Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black. 2015. SMPL: a skinned multi-person linear model. ACM Trans- actions on Graphics (TOG) 34, 6 (2015), 1–16

    2015

  25. [33]

    Ilya Loshchilov and Frank Hutter. 2017. Decoupled Weight Decay Regularization. In International Conference on Learning Representations

    2017

  26. [34]

    Qiujing Lu, Yipeng Zhang, Mingjian Lu, and Vwani Roychowdhury. 2022. Action- conditioned on-demand motion generation. In Proceedings of the 30th ACM Inter- national Conference on Multimedia . 2249–2257

    2022

  27. [35]

    Wan-Duo Kurt Ma, John P Lewis, and W Bastiaan Kleijn. 2024. Trailblazer: Trajectory control for diffusion-based video generation. In SIGGRAPH Asia 2024 Conference Papers. 1–11

    2024

  28. [36]

    Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Xiu Li, and Qifeng Chen. 2024. Follow your pose: Pose-guided text-to-video generation using pose-free videos. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 38. 4117–4125

    2024

  29. [37]

    Naureen Mahmood, Nima Ghorbani, Nikolaus F Troje, Gerard Pons-Moll, and Michael J Black. 2019. AMASS: Archive of motion capture as surface shapes. In Proceedings of the IEEE/CVF international conference on computer vision . 5442– 5451

    2019

  30. [38]

    Mathis Petrovich, Michael J Black, and Gül Varol. 2021. Action-conditioned 3D human motion synthesis with transformer VAE. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 10985–10995

    2021

  31. [39]

    Mathis Petrovich, Michael J Black, and Gül Varol. 2022. Temos: Generating diverse human motions from textual descriptions. In European Conference on Computer Vision. Springer, 480–497

    2022

  32. [40]

    Mathis Petrovich, Michael J Black, and Gül Varol. 2023. Tmr: Text-to-motion retrieval using contrastive 3d human motion synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 9488–9497

    2023

  33. [41]

    Matthias Plappert, Christian Mandery, and Tamim Asfour. 2016. The kit motion- language dataset. Big data 4, 4 (2016), 236–252

    2016

  34. [42]

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen

  35. [43]

    Davis Rempe, Zhengyi Luo, Xue Bin Peng, Ye Yuan, Kris Kitani, Karsten Kreis, Sanja Fidler, and Or Litany. 2023. Trace and pace: Controllable pedestrian anima- tion via guided trajectory diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni...

    2023

  36. [44]

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. 2022. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural infor...

    2022

  37. [45]

    arXiv preprint arXiv:2204.06125 1, 2 (2022), 3

    Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 1, 2 (2022), 3

    2022 arXiv

  38. [46]

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli

  39. [47]

    Mikihiro Tanaka and Kent Fujiwara. 2023. Role-aware interaction generation from textual description. In Proceedings of the IEEE/CVF international conference on computer vision. 15999–16009

    2023

  40. [48]

    Yoni Shafir, Guy Tevet, Roy Kapon, and Amit Haim Bermano. 2024. Human Motion Diffusion as a Generative Prior. In ICLR

    2024

  41. [49]

    Guy Tevet, Brian Gordon, Amir Hertz, Amit H Bermano, and Daniel Cohen-Or

  42. [50]

    Guy Tevet, Sigal Raab, Brian Gordon, Yoni Shafir, Daniel Cohen-or, and Amit Haim Bermano. 2023. Human Motion Diffusion Model. In The Eleventh International Conference on Learning Representations

    2023

  43. [51]

    Jonathan Tseng, Rodrigo Castellon, and Karen Liu. 2023. Edge: Editable dance generation from music. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 448–458

    2023

  44. [52]

    Maham Tanveer, Yang Zhou, Simon Niklaus, Ali Mahdavi Amiri, Hao Zhang, Krishna Kumar Singh, and Nanxuan Zhao. 2024. MotionBridge: Dynamic Video Inbetweening with Flexible Controls. arXiv preprint arXiv:2412.13190 (2024)

    2024 arXiv

  45. [53]

    Tao Wang, Zhihua Wu, Qiaozhi He, Jiaming Chu, Ling Qian, Yu Cheng, Junliang Xing, Jian Zhao, and Lei Jin. 2025. StickMotion: Generating 3D Human Motions by Drawing a Stickman. arXiv preprint arXiv:2503.04829 (2025)

    2025 arXiv

  46. [54]

    In European Conference on Computer Vision

    Motionclip: Exposing human motion generation to clip space. In European Conference on Computer Vision . Springer, 358–374

  47. [55]

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. 2024. Motionctrl: A unified and flexible motion controller for video generation. InACM SIGGRAPH 2024 Conference Papers. 1–11

    2024

  48. [56]

    Zizhao Wu, Qin Wang, Xinyang Zheng, Jianglei Ye, Ping Yang, Yunhai Wang, and Yigang Wang. 2024. Doodle Your Motion: Sketch-Guided Human Motion Generation. IEEE Transactions on Visualization and Computer Graphics (2024)

    2024

  49. [57]

    Jiawei Wang, Yuchen Zhang, Jiaxin Zou, Yan Zeng, Guoqiang Wei, Liping Yuan, and Hang Li. 2024. Boximator: Generating rich and controllable motions for video synthesis. arXiv preprint arXiv:2402.01566 (2024)

    2024 arXiv

  50. [58]

    Yiming Xie, Varun Jampani, Lei Zhong, Deqing Sun, and Huaizu Jiang. 2023. Omnicontrol: Control any joint at any time for human motion generation. arXiv preprint arXiv:2310.08580 (2023)

    2023 arXiv

  51. [59]

    Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Jiuniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jingren Zhou. 2023. Videocomposer: Composi- tional video synthesis with motion controllability.Advances in Neural Information Processing Systems 36 (2023), 7594–7611

    2023

  52. [60]

    Sirui Xu, Zhengyuan Li, Yu-Xiong Wang, and Liang-Yan Gui. 2023. Interdiff: Generating 3d human-object interactions with physics-informed diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 14928– 14940

    2023

  53. [61]

    Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. 2024. Magicanimate: Temporally consistent human image animation using diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern ...

    2024

  54. [62]

    Zeqi Xiao, Tai Wang, Jingbo Wang, Jinkun Cao, Wenwei Zhang, Bo Dai, Dahua Lin, and Jiangmiao Pang. 2024. Unified Human-Scene Interaction via Prompted Chain-of-Contacts. In ICLR

    2024

  55. [63]

    Shengming Yin, Chenfei Wu, Jian Liang, Jie Shi, Houqiang Li, Gong Ming, and Nan Duan. 2023. Dragnuwa: Fine-grained control in video generation by integrating text, image, and trajectory. arXiv preprint arXiv:2308.08089 (2023)

    2023 arXiv

  56. [64]

    Jinbo Xing, Hanyuan Liu, Menghan Xia, Yong Zhang, Xintao Wang, Ying Shan, and Tien-Tsin Wong. 2024. Tooncrafter: Generative cartoon interpolation. ACM Transactions on Graphics (TOG) 43, 6 (2024), 1–11

    2024

  57. [65]

    Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. 2024. Motiondiffuse: Text-driven human motion genera- tion with diffusion model. IEEE transactions on pattern analysis and machine intelligence 46, 6 (2024), 4115–4128

    2024

  58. [66]

    Mingyuan Zhang, Xinying Guo, Liang Pan, Zhongang Cai, Fangzhou Hong, Huirong Li, Lei Yang, and Ziwei Liu. 2023. Remodiffuse: Retrieval-augmented motion diffusion model. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 364–373

    2023

  59. [67]

    Hongwei Yi, Hualin Liang, Yifei Liu, Qiong Cao, Yandong Wen, Timo Bolkart, Dacheng Tao, and Michael J Black. 2023. Generating holistic 3d human motion from speech. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 469–480

    2023

  60. [68]

    Yuang Zhang, Jiaxi Gu, Li-Wen Wang, Han Wang, Junqi Cheng, Yuefeng Zhu, and Fangyuan Zou. 2024. Mimicmotion: High-quality human motion video generation with confidence-aware pose guidance. arXiv preprint arXiv:2406.19680 (2024)

    2024 arXiv

  61. [69]

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. 2023. Adding conditional con- trol to text-to-image diffusion models. InProceedings of the IEEE/CVF international conference on computer vision . 3836–3847

    2023

  62. [72]

    Xiaohan Zhang, Bharat Lal Bhatnagar, Sebastian Starke, Vladimir Guzov, and Gerard Pons-Moll. 2022. Couch: Towards controllable human-chair interactions. In European Conference on Computer Vision . Springer, 518–535

    2022

  63. [74]

    Kaifeng Zhao, Yan Zhang, Shaofei Wang, Thabo Beeler, and Siyu Tang. 2023. Synthesizing diverse human motions in 3d indoor scenes. In Proceedings of the IEEE/CVF international conference on computer vision . 14738–14749. A MULTI-AGENT SYSTEM FOR STRUCTURED MOTION SPECIFICATION ...

    2023

  64. [2015]

    In International conference on machine learning

    Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning . pmlr, 2256–2265

  65. [2022]

    InProceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Generating diverse and natural 3d human motions from text. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition . 5152–5161

  66. [2024]

    In ACM SIGGRAPH 2024 Conference Papers

    Flexible motion in-betweening with diffusion models. In ACM SIGGRAPH 2024 Conference Papers. 1–9

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.