Pith. sign in

REVIEW 4 major objections 4 minor 5 cited by

MotionLab: Unified Human Motion Generation and Editing via the Motion-Condition-Motion Paradigm

T0 review · 4 major / 4 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper's central claim is that a single model, MotionLab, can unify all six major human motion generation and editing tasks and outperform its own task-specific specialists on every one.

desk verdict A strong unified-motion architecture with a neat paradigm, but the paper's own supplementary table contradicts its headline superiority claim and needs a fix before publication. read the letter →

arxiv 2502.02358 v5 pith:2SLV6H2P submitted 2025-02-04 cs.CV

Ziyan Guo , Zeyu Hu , De Wen Soh , Na Zhao This is my paper · ORCID
classification cs.CV
keywords humanmotiongenerationeditingMotion-Condition-MotionparadigmrectifiedflowTransformercurriculumlearningmulti-taskLab
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MotionLab's thesis is that human motion generation and editing are one problem, not many: given a source motion (possibly none), a condition (text, trajectory, keyframes, or style), and a target motion, learn the map between them. The paper builds a single rectified-flow model around this Motion-Condition-Motion schema and trains it on six tasks with a two-stage curriculum. It reports that the unified model beats its own task-specialist versions on every task and shows strong results against published baselines on the same benchmarks, which matters because one network could then replace a zoo of separate motion models. The takeaway claim is that multi-task knowledge sharing, not task-specific architecture, is what buys both quality and speed.

What carries the argument

The load-bearing object is the Motion-Condition-Motion triple (source motion, condition, target motion), which rewrites every task as a conditional transportation problem. The machinery that solves it: rectified flows, which learn a straight velocity field from a noisy target back to the true target; the MotionFlow Transformer (MFT), a joint-attention transformer with a dedicated modality path per input, so source motion, target motion, trajectory, style, and text all interact in one attention pass; Aligned ROPE, a 1D rotary positional encoding applied to each modality so that tokens at the same frame of source and target stay time-aligned; Task Instruction Modulation, which feeds a CLIP text embedding describing the task (e.g., "edit source motion by given text") as a modulation signal; and Motion Curriculum Learning, which first pre-trains on masked reconstruction and then fine-tunes tasks from easy to hard. The shared mechanism is that spatially concrete conditions (trajectory, keyframes, source motion) can partially represent abstract conditions (text, style), so the curriculum transfers spatial knowledge to the abstract modalities.

What would settle it

Run the identical MotionLab training with the fine-tuning task order reversed (style transfer and trajectory-based editing first, text-based generation last) or randomly permuted. If a different order also makes the unified model beat its own specialist models on the main metrics, the curriculum's specific ordering is not doing the claimed work; if the reversed order collapses text-generation FID while preserving editing gains, the difficulty assumption is supported.

Watch

Extended reading notes

Core claim

The paper's claim is that the main human motion tasks — text-based generation, trajectory-based generation, motion in-between, text-based editing, trajectory-based editing, and style transfer — reduce to one schema, Motion-Condition-Motion: every task specifies a source motion (possibly empty for generation), a condition (text, trajectory, keyframes, or style motion), and a target motion, and the model learns a single rectified-flow velocity field that transports a noisy target into the correct target given the source and condition. On the benchmarks, MotionLab reports text-generation FID of 0.167 (specialist: 0.209), trajectory-generation average error of 0.0334 (specialist: 0.0398), text-editing R@1 of 56.34 (specialist: 41.44), trajectory-editing R@1 of 72.65 (specialist: 59.86), in-betweening average error of 0.0283 (specialist: 0.0371), and style-transfer CRA/SRA of 44.62/69.21 (specialist: 43.53/67.55). The ablations attribute the dominant share of these gains to Motion Curriculum Learning: without it, text-generation FID rises from 0.167 to 1.956 and editing retrieval drops by about half, while the full unified model beats its own specialist models on every task.

Load-bearing premise

The load-bearing premise is that the curriculum's difficulty ordering — fewer modalities before more, editing before generation, spatial conditions before text and style — is the right one; the paper justifies it by intuition and the ablations do not test an alternative ordering.

Editorial extensions

If this is right

  • A single MotionLab network covers text-based generation, trajectory-based generation, motion in-between, text-based editing, trajectory-based editing, and style transfer, so a practitioner no longer needs to train and run six separate models.
  • Data-scarce editing tasks inherit spatial knowledge from data-rich generation tasks; the editing retrieval numbers beat specialist models trained only on the editing data.
  • The Aligned ROPE component is what keeps source and target frames synchronized; removing it roughly triples in-betweening error (0.0756 vs 0.0283) and sharply drops editing retrieval.
  • With 50 inference steps and classifier-free guidance, text generation takes about 0.07 seconds per sample, faster than every diffusion-based baseline except the real-time MotionLCM.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's curriculum claim could be stress-tested by reversing the fine-tuning order; if a different order still beats the specialists, the explanatory mechanism would be multi-task reweighting rather than the specific easy-to-hard order.
  • Because the schema treats any input as a potential source motion and any output as a target motion, the same design could transfer to other time-aligned sequence-editing domains, such as facial-expression editing or gesture editing driven by language, where source and target must stay frame-aligned.
  • Task Instruction Modulation depends on CLIP's semantics; tasks whose instructions fall outside CLIP's text distribution would need a new conditioning vector, which is both a simple extension path and a likely failure mode for unusual editing commands.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes the Motion-Condition-Motion paradigm, in which every human motion generation or editing task is expressed as a mapping from source motion to target motion under a condition. Based on this paradigm, the authors introduce MotionLab, a unified framework built on rectified flows and a multimodal transformer (MotionFlow Transformer) with Aligned Rotational Position Encoding, task instruction modulation, and a curriculum learning schedule. The central claim is that this single model matches or surpasses task-specific specialists and prior state-of-the-art methods across text-based generation/editing, trajectory-based generation/editing, motion in-betweening, and style transfer, while also being efficient at inference. The main-text quantitative results (Tables 3–5) support several of these claims, and the supplementary provides additional comparisons for in-betweening and style transfer. However, the supplementary in-between table directly contradicts the paper's statement that MotionLab outperforms CondMDI on all settings, and the lack of variance estimates leaves some comparisons uncertain.

Significance. If the empirical results can be made internally consistent, the paper makes a valuable contribution to unified human motion modeling. The Motion-Condition-Motion formulation is conceptually clean and potentially scalable, and the proposed architectural components—especially Aligned ROPE and task instruction modulation—are well motivated. The paper also provides extensive ablations (Tables 6, 9–12) and reports inference-time and memory costs, which are welcome. The main weakness is that the evidence for the headline claim is not yet consistent: the supplementary Table 8 shows that MotionLab's 1-frame in-between FID is about 4.9 times worse than CondMDI's, and several comparisons lack multiple-seed statistics. These issues must be resolved before the claim of universal superiority can be accepted.

major comments (4)
  1. [Supplementary Material, Section 11, Table 8] The text in Section 11 states that "our framework outperforms CondMDI on all settings," but Table 8 shows the opposite for the 1-frame in-between setting: MotionLab's FID is 0.7547 versus CondMDI's 0.1551, R-precision top-3 is 0.6681 versus 0.6787, and Diversity is 8.9058 versus 9.5807 (ground-truth Diversity is 9.503). This is an internal contradiction in the reported evidence, and it is load-bearing because the paper's headline is that MotionLab surpasses specialists on all evaluated tasks. The authors should either correct the numbers, replace the metric with one for which the comparison is favorable, or explicitly qualify the claim to exclude or explain the 1-frame case. As written, the empirical support for the central claim is incomplete.
  2. [Section 5.3 and Section 12, Table 11] The success of Motion Curriculum Learning rests on the easy-to-hard ordering specified in Section 5.3, but no alternative ordering is tested. The ablations in Table 11 remove masked pre-training, remove ordering, or use random selection, but they do not reverse or permute the stages. The reader cannot tell whether the improvement comes from the specific ordering (e.g., spatial conditions before text and style) or from the multi-task training schedule itself. Please add an ablation with at least one reversed order (for example, text-based generation first, or style transfer before trajectory-based editing) to demonstrate that the claimed transfer of spatial knowledge depends on the proposed order rather than on the mere fact of multi-task fine-tuning.
  3. [Section 6, Tables 3–5; Section 12, Table 11] No standard deviations or multiple seeds are reported for any quantitative result. The sampling procedure is stochastic (CFG with learned velocity fields), and some comparisons are close (e.g., Table 3, R@3: MotionLab 0.810 vs MotionCLR 0.831; Diversity 9.593 vs 9.607). Without multiple runs, it is unclear whether the reported differences are meaningful. Please report mean and standard deviation over at least three seeds for the central metrics, or state explicitly that the evaluation protocol is deterministic and explain why.
  4. [Section 6.3, Table 6; Section 12, Table 14] The CFG strengths in Table 14 of the supplementary were "obtained" by ablations on the evaluation benchmarks, and the curriculum-learning variant uses FID from the last evaluation to set sampling probabilities. This means the reported numbers are partially the result of fitting hyperparameters to the test benchmarks. While this is common in the field, the paper should acknowledge it in the main text, report the validation split used for tuning, and indicate the sensitivity of the main results to the chosen CFG strengths. This also bears on the comparison with baselines, which may not have been tuned in the same way.
minor comments (4)
  1. [Figure 1] The grouped bars in Figure 1 are difficult to interpret because the values for different metrics are on very different scales and the legend for the three groups (Previous SOTA, Specialists, Ours) is sparse; please clarify in the caption which metric each panel refers to and consider normalizing the axes.
  2. [Section 4, Table 2] The term "condition" is used broadly to include text, trajectory, keyframe poses, style motion, and even the empty set; please state explicitly that a condition may be a concatenation of multiple signals and that the source motion can be empty for generation tasks.
  3. [Section 6, Evaluation Metrics] The "Average Error" metric for trajectory-based generation and motion in-between is described only qualitatively; please specify which joints are included, how keyframe errors are aggregated, and whether the error is computed in the HumanML3D feature space or in joint coordinates.
  4. [Section 6.3, Table 6] The row label "w/o motion curriculum learning" is described in the text as "directly learns all tasks based on their FID compared to the last evaluation," which is more precisely a random/adaptive task-selection strategy; consider renaming the row to "random task selection" to avoid confusion with removing the curriculum ordering.

Circularity Check

0 steps flagged · score 0.0 of 10

MotionLab: no circularity found; the central results are empirical evaluations on external benchmarks, and the one self-citation ([25]) is not load-bearing. The Section 11 vs Table 8 CondMDI FID discrepancy is a correctness caveat, not a circular derivation.

full rationale

The paper's derivation chain is empirical: it casts tasks into a source/condition/target taxonomy (Table 2), trains a rectified-flow velocity field with Eq. 6, and evaluates the sampled motions on benchmarks. The flow objective regresses v_theta(t, x_t) to x_1 - x_0; this standard regression target is not reused as an evaluation metric. The reported FID, R-precision, average error, and SRA/CRA are computed from generated samples, so they are not defined in terms of the model's own fit. The only self-citation ([25], TSTMotion) appears in a list of trajectory-based editing prior works and carries no argumentative weight. I explicitly flag two caveats that affect the strength of the empirical claims but do not constitute circularity: (i) Section 5.3 sets task re-training probabilities from 'the FID of the last evaluation' and Section 15 reports CFG strengths 'obtained' through ablations on the benchmark, so the reported numbers are not fully benchmark-independent; (ii) Section 11 says 'our framework outperforms CondMDI on all settings' while Table 8 shows Ours 1-frame FID 0.7547 versus CondMDI 0.1551, an internal inconsistency. Neither is a case where a prediction equals its input by construction or a fitted parameter is renamed as a prediction, so the circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the efficacy of the curriculum ordering, the sufficiency of CLIP instruction embeddings, and the suitability of rectified flows. The per-task CFG strengths are fitted to the benchmarks, making the reported numbers dependent on these tuned values.

free parameters (4)
  • CFG hyperparameters (lambda_C, lambda_S) = Varies per task, e.g., 5.75 for text generation, 2.25 for editing
    Tuned on the evaluation benchmarks as described in Section 15 of the supplementary material.
  • Curriculum learning probability allocation = 5% unconditional, 5% reconstruction, 45% previous tasks, 45% new task
    Chosen by hand in Section 5.3; not ablated.
  • Inference timesteps N = 50
    Used during inference; Figure 6 suggests balance between quality and efficiency but no auto-tuning.
  • Pre-training and fine-tuning epochs = 1000 epochs pre-training, 200 epochs per task in fine-tuning
    Set as described in Section 5.3.
assumptions (4)
  • domain assumption Task difficulty ordering: fewer modalities is simpler; editing is easier than generation; more specific conditions are simpler.
    Used in Section 5.3 to design the curriculum; not independently validated or compared to alternative orderings.
  • domain assumption Spatial conditions (source motion, trajectory) can represent abstract conditions (text, style) and enable knowledge transfer.
    Invoked in Section 5.3 to justify the curriculum and the claim that multi-task learning improves abstract-condition tasks.
  • standard math Rectified flows with linear interpolation are a suitable generative model for human motion.
    Adopted from prior work [37,38]; standard in the field, so not a new assumption.
  • domain assumption CLIP text embeddings of hand-written instruction phrases can differentiate tasks and conditions.
    Assumed in Section 5.2; the ablation (Table 9) provides evidence but only for the specific phrases used.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MotionLab: Unified Human Motion Generation and Editing via the Motion-Condition-Motion Paradigm." pith.science (2026). https://pith.science/paper/2SLV6H2P

@misc{pith2026250202358,
  author       = {Pith},
  title        = {Pith review of: MotionLab: Unified Human Motion Generation and Editing via the Motion-Condition-Motion Paradigm},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2SLV6H2P}},
  note         = {Machine review of arXiv:2502.02358}
}
read the original abstract

Human motion generation and editing are key components of computer vision. However, current approaches in this field tend to offer isolated solutions tailored to specific tasks, which can be inefficient and impractical for real-world applications. While some efforts have aimed to unify motion-related tasks, these methods simply use different modalities as conditions to guide motion generation. Consequently, they lack editing capabilities, fine-grained control, and fail to facilitate knowledge sharing across tasks. To address these limitations and provide a versatile, unified framework capable of handling both human motion generation and editing, we introduce a novel paradigm: \textbf{Motion-Condition-Motion}, which enables the unified formulation of diverse tasks with three concepts: source motion, condition, and target motion. Based on this paradigm, we propose a unified framework, \textbf{MotionLab}, which incorporates rectified flows to learn the mapping from source motion to target motion, guided by the specified conditions. In MotionLab, we introduce the 1) MotionFlow Transformer to enhance conditional generation and editing without task-specific modules; 2) Aligned Rotational Position Encoding to guarantee the time synchronization between source motion and target motion; 3) Task Specified Instruction Modulation; and 4) Motion Curriculum Learning for effective multi-task learning and knowledge sharing across tasks. Notably, our MotionLab demonstrates promising generalization capabilities and inference efficiency across multiple benchmarks for human motion. Our code and additional video results are available at: https://diouo.github.io/motionlab.github.io/.

Figures

Figures reproduced from arXiv: 2502.02358 by the authors.

Figure 1
Figure 1. Demonstration of our MotionLab’s versatility, performance and efficiency. Ours specialists refer to the proposed framework [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of our MotionLab and the detail of its MotionFlow Transformer (MFT). [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Qualitative results on the text-based motion generation. For clarity, as time progresses, motions transit from light to dark colors. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Qualitative results on text-based motion editing. The transparent motion is source motion, and the other is the generated motion. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Qualitative results on the trajectory-based motion generation. The red balls are the trajectory of the pelvis, right hand and foot. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Impact of timesteps during inference on MotionLab. The closer to the lower left corner, the more powerful the model. Additionally, we evaluate the impact of timesteps during inference on our MotionLab and compare its performance with baseline methods in terms of genera…
Figure 7
Figure 7. Figure 7: Demonstration of the difference between diffusion mod [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Comparison of the motion style transfer with MCM [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Ablation results of MotionLab on the motion in-between (with text). Beige motion is use 1D-learnable position encoding, purple [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PMG: Progressive Motion Generation via Sparse Anchor Postures Curriculum Learning

    cs.CV 2025-04 conditional novelty 6.0 of 10

    ProMoGen generates human motion conditioned on both a trajectory and sparse anchor postures via a diffusion transformer trained with a dense-to-sparse curriculum.

  2. Visual Prompting for One-shot Controllable Video Editing without Inversion

    cs.CV 2025-04 conditional novelty 6.0 of 10

    A one-shot video editing method that uses a 2x2 visual prompt grid, modified consistency sampling, and Stein Variational Gradient Descent to propagate first-frame edits without DDIM inversion.

  3. MAPF-World: Action World Model for Multi-Agent Path Finding

    cs.AI 2025-08 unverdicted novelty 5.0 of 10

    MAPF-World, an autoregressive action world model that predicts future states and actions, is claimed to beat state-of-the-art learnable MAPF solvers while using 96.5% fewer parameters and 92% less data.

  4. Motion Generation: A Survey of Generative Approaches and Benchmarks

    cs.CV 2025-07 unverdicted novelty 3.0 of 10

    A structured survey that categorizes recent motion generation methods by underlying generative approach and compiles datasets, metrics, and statistical trends.

  5. Generative AI for Character Animation: A Comprehensive Survey of Techniques, Applications, and Future Directions

    cs.CV 2025-04 conditional novelty 3.0 of 10

    A comprehensive survey that unifies generative AI techniques for character animation across facial, gesture, motion, and 3D asset generation, with a shared taxonomy and resource list.

Reference graph

Works this paper leans on

83 extracted references · 47 canonical work pages · cited by 5 Pith papers

  1. [1]

    Unpaired motion style transfer from video to animation

    Kfir Aberman, Yijia Weng, Dani Lischinski, Daniel Cohen- Or, and Baoquan Chen. Unpaired motion style transfer from video to animation. ACM Transactions on Graphics (TOG), 39(4):64–1, 2020. 1, 3

    2020

  2. [2]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    arXiv

  3. [3]

    Listen, denoise, action! audio-driven motion synthesis with diffusion models

    Simon Alexanderson, Rajmund Nagy, Jonas Beskow, and Gustav Eje Henter. Listen, denoise, action! audio-driven motion synthesis with diffusion models. ACM Transactions on Graphics (TOG), 42(4):1–20, 2023. 3

    2023

  4. [4]

    Teach: Temporal action composition for 3d hu- mans

    Nikos Athanasiou, Mathis Petrovich, Michael J Black, and G¨ul Varol. Teach: Temporal action composition for 3d hu- mans. In 2022 International Conference on 3D Vision (3DV), pages 414–423. IEEE, 2022. 1

    2022

  5. [5]

    Sinc: Spatial composition of 3d human motions for simultaneous action generation

    Nikos Athanasiou, Mathis Petrovich, Michael J Black, and G¨ul Varol. Sinc: Spatial composition of 3d human motions for simultaneous action generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 9984–9995, 2023. 1

    work page 2023

  6. [6]

    Motionfix: Text-driven 3d human motion editing

    Nikos Athanasiou, Alp ´ar Cseke, Markos Diomataris, Michael J Black, and G ¨ul Varol. Motionfix: Text-driven 3d human motion editing. In SIGGRAPH Asia 2024 Conference Papers, pages 1–11, 2024. 1, 2, 3, 6, 7

    work page 2024

  7. [7]

    Curriculum learning

    Yoshua Bengio, J ´erˆome Louradour, Ronan Collobert, and Ja- son Weston. Curriculum learning. InProceedings of the 26th annual international conference on machine learning, pages 41–48, 2009. 5

    work page 2009

  8. [8]

    Motionclr: Motion generation and training-free edit- ing via understanding attention mechanisms

    Ling-Hao Chen, Wenxun Dai, Xuan Ju, Shunlin Lu, and Lei Zhang. Motionclr: Motion generation and training-free edit- ing via understanding attention mechanisms. arXiv preprint arXiv:2410.18977, 2024. 2, 3, 6

    arXiv 2024

Show all 83 references
  1. [9]

    Executing your commands via motion diffusion in latent space

    Xin Chen, Biao Jiang, Wen Liu, Zilong Huang, Bin Fu, Tao Chen, and Gang Yu. Executing your commands via motion diffusion in latent space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 18000–18010, 2023. 2, 3, 4, 6

    2023

  2. [10]

    Diffusion posterior sam- pling for general noisy inverse problems

    Hyungjin Chung, Jeongsol Kim, Michael T Mccann, Marc L Klasky, and Jong Chul Ye. Diffusion posterior sam- pling for general noisy inverse problems. arXiv preprint arXiv:2209.14687, 2022. 2

    2022 arXiv

  3. [11]

    Flexible motion in-betweening with diffusion models

    Setareh Cohan, Guy Tevet, Daniele Reda, Xue Bin Peng, and Michiel van de Panne. Flexible motion in-betweening with diffusion models. In ACM SIGGRAPH 2024 Conference Pa- pers, pages 1–9, 2024. 1, 2, 3, 6

    2024

  4. [12]

    Motionlcm: Real-time controllable motion generation via latent consistency model

    Wenxun Dai, Ling-Hao Chen, Jingbo Wang, Jinpeng Liu, Bo Dai, and Yansong Tang. Motionlcm: Real-time controllable motion generation via latent consistency model. InEuropean Conference on Computer Vision , pages 390–408. Springer,

  5. [13]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural informa- tion processing systems, 34:8780–8794, 2021. 1

    2021

  6. [14]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Ab- hishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 ,

    arXiv

  7. [15]

    Scaling recti- fied flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. In Forty-first International Conference on Mach...

    2024

  8. [16]

    Everything2motion: Synchronizing diverse inputs via a unified framework for human motion synthesis

    Zhaoxin Fan, Longbin Ji, Pengxin Xu, Fan Shen, and Kai Chen. Everything2motion: Synchronizing diverse inputs via a unified framework for human motion synthesis. In Pro- ceedings of the AAAI Conference on Artificial Intelligence , pages 1688–1697, 2024. 2, 3

    2024

  9. [17]

    Flux that plays music

    Zhengcong Fei, Mingyuan Fan, Changqian Yu, and Jun- shi Huang. Flux that plays music. arXiv preprint arXiv:2409.00587, 2024. 3

    2024 arXiv

  10. [18]

    Chronolog- ically accurate retrieval for temporal grounding of motion- language models

    Kent Fujiwara, Mikihiro Tanaka, and Qing Yu. Chronolog- ically accurate retrieval for temporal grounding of motion- language models. In European Conference on Computer Vi- sion, pages 323–339. Springer, 2025. 1

    2025

  11. [19]

    Iterative motion editing with natural language

    Purvi Goel, Kuan-Chieh Wang, C Karen Liu, and Kayvon Fatahalian. Iterative motion editing with natural language. In ACM SIGGRAPH 2024 Conference Papers , pages 1–9,

    2024

  12. [20]

    Ac- tion2motion: Conditioned generation of 3d human motions

    Chuan Guo, Xinxin Zuo, Sen Wang, Shihao Zou, Qingyao Sun, Annan Deng, Minglun Gong, and Li Cheng. Ac- tion2motion: Conditioned generation of 3d human motions. In Proceedings of the 28th ACM International Conference on Multimedia, pages 2021–2029, 2020. 3

    2021

  13. [21]

    Generating diverse and natural 3d human motions from text

    Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5152–5161, 2022. 1, 6, 2

    2022

  14. [22]

    Tm2t: Stochastic and tokenized modeling for the reciprocal genera- tion of 3d human motions and texts

    Chuan Guo, Xinxin Zuo, Sen Wang, and Li Cheng. Tm2t: Stochastic and tokenized modeling for the reciprocal genera- tion of 3d human motions and texts. InEuropean Conference on Computer Vision, pages 580–597. Springer, 2022

    2022

  15. [23]

    Momask: Generative masked model- ing of 3d human motions

    Chuan Guo, Yuxuan Mu, Muhammad Gohar Javed, Sen Wang, and Li Cheng. Momask: Generative masked model- ing of 3d human motions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1900–1910, 2024. 3, 5

    1900

  16. [24]

    Generative human motion stylization in latent space

    Chuan Guo, Yuxuan Mu, Xinxin Zuo, Peng Dai, Youliang Yan, Juwei Lu, and Li Cheng. Generative human motion stylization in latent space. arXiv preprint arXiv:2401.13505,

    arXiv

  17. [25]

    Tstmotion: Training- free scene-aware text-to-motion generation

    Ziyan Guo, Haoxuan Qu, Hossein Rahmani, Dewen Soh, Ping Hu, Qiuhong Ke, and Jun Liu. Tstmotion: Training- free scene-aware text-to-motion generation. arXiv preprint arXiv:2505.01182, 2025. 2

    2025 arXiv

  18. [26]

    Robust motion in-betweening

    F ´elix G Harvey, Mike Yurick, Derek Nowrouzezahrai, and Christopher Pal. Robust motion in-betweening. ACM Trans- actions on Graphics (TOG), 39(4):60–1, 2020. 2

    2020

  19. [27]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 6, 1

    2022 arXiv

  20. [28]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 1

    2020

  21. [29]

    Mo- tion puzzle: Arbitrary motion style transfer by body part

    Deok-Kyeong Jang, Soomin Park, and Sung-Hee Lee. Mo- tion puzzle: Arbitrary motion style transfer by body part. ACM Transactions on Graphics (TOG) , 41(3):1–16, 2022. 1, 3

    2022

  22. [30]

    Motiongpt: Human motion as a foreign lan- guage

    Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, and Tao Chen. Motiongpt: Human motion as a foreign lan- guage. Advances in Neural Information Processing Systems, 36:20067–20079, 2023. 2, 3, 6

    2023

  23. [31]

    Motionchain: Conversational motion controllers via multimodal prompts

    Biao Jiang, Xin Chen, Chi Zhang, Fukun Yin, Zhuoyuan Li, Gang Yu, and Jiayuan Fan. Motionchain: Conversational motion controllers via multimodal prompts. In European Conference on Computer Vision , pages 54–74. Springer,

  24. [32]

    Guided motion diffusion for controllable human motion synthesis

    Korrawe Karunratanakul, Konpat Preechakul, Supasorn Suwajanakorn, and Siyu Tang. Guided motion diffusion for controllable human motion synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 2151–2162, 2023. 3, 6

    2023

  25. [33]

    Flame: Free- form language-based motion synthesis & editing

    Jihoon Kim, Jiseob Kim, and Sungjoon Choi. Flame: Free- form language-based motion synthesis & editing. In Pro- ceedings of the AAAI Conference on Artificial Intelligence , pages 8255–8263, 2023. 3

    2023

  26. [34]

    Unimotion: Unify- ing 3d human motion synthesis and understanding

    Chuqiao Li, Julian Chibane, Yannan He, Naama Pearl, An- dreas Geiger, and Gerard Pons-Moll. Unimotion: Unify- ing 3d human motion synthesis and understanding. arXiv preprint arXiv:2409.15904, 2024. 3

    2024 arXiv

  27. [35]

    Motion-x: A large-scale 3d expressive whole-body human motion dataset

    Jing Lin, Ailing Zeng, Shunlin Lu, Yuanhao Cai, Ruimao Zhang, Haoqian Wang, and Lei Zhang. Motion-x: A large-scale 3d expressive whole-body human motion dataset. Advances in Neural Information Processing Systems , 36: 25268–25280, 2023. 3

    2023

  28. [36]

    Motionllama: A unified framework for motion synthesis and comprehension

    Zeyu Ling, Bo Han, Shiyang Li, Hongdeng Shen, Jikang Cheng, and Changqing Zou. Motionllama: A unified framework for motion synthesis and comprehension. arXiv preprint arXiv:2411.17335, 2024. 3

    2024 arXiv

  29. [37]

    Flow matching for generative mod- eling

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximil- ian Nickel, and Matt Le. Flow matching for generative mod- eling. arXiv preprint arXiv:2210.02747, 2022. 2, 3

    2022 arXiv

  30. [38]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003, 2022. 2, 3

    2022 arXiv

  31. [39]

    Smpl: A skinned multi- person linear model

    Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black. Smpl: A skinned multi- person linear model. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2, pages 851–866. 2023. 1

    2023

  32. [40]

    Humantomato: Text-aligned whole-body motion generation

    Shunlin Lu, Ling-Hao Chen, Ailing Zeng, Jing Lin, Ruimao Zhang, Lei Zhang, and Heung-Yeung Shum. Humantomato: Text-aligned whole-body motion generation. arXiv preprint arXiv:2310.12978, 2023. 3

    2023 arXiv

  33. [41]

    M3gpt: An advanced multimodal, multitask framework for motion comprehension and generation

    Mingshuang Luo, Ruibing Hou, Hong Chang, Zimo Liu, Yaowei Wang, and Shiguang Shan. M3gpt: An advanced multimodal, multitask framework for motion comprehension and generation. arXiv preprint arXiv:2405.16273, 2024. 2, 3

    2024 arXiv

  34. [42]

    Sit: Explor- ing flow and diffusion-based generative models with scalable interpolant transformers

    Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijnden, and Saining Xie. Sit: Explor- ing flow and diffusion-based generative models with scalable interpolant transformers. arXiv preprint arXiv:2401.08740,

    arXiv

  35. [43]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 4195–4205,

  36. [44]

    Action- conditioned 3d human motion synthesis with transformer vae

    Mathis Petrovich, Michael J Black, and G ¨ul Varol. Action- conditioned 3d human motion synthesis with transformer vae. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 10985–10995, 2021. 3

    2021

  37. [45]

    Mmm: Generative masked motion model

    Ekkasit Pinyoanuntapong, Pu Wang, Minwoo Lee, and Chen Chen. Mmm: Generative masked motion model. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1546–1555, 2024. 3

    2024

  38. [46]

    The kit motion-language dataset

    Matthias Plappert, Christian Mandery, and Tamim Asfour. The kit motion-language dataset. Big data, 4(4):236–252,

  39. [47]

    Sampson, Shikai Li, Simone Parmeggiani, Steve Fine, Tara Fowler, Vladan Petro- vic, and Yuming Du

    Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih- Yao Ma, Ching-Yao Chuang, David Yan, Dhruv Choudhary, Dingkang Wang, Geet Sethi, Guan Pang, Haoyu Ma, Ishan Misra, Ji Hou, Jialiang Wang, Kiran Jagadeesh, Kunpeng Li, L...

    2024

  40. [48]

    Motion in-betweening via two-stage transformers

    Jia Qin, Youyi Zheng, and Kun Zhou. Motion in-betweening via two-stage transformers. ACM Trans. Graph., 41(6):184– 1, 2022. 3

    2022

  41. [49]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  42. [50]

    Length-aware motion synthesis via latent diffusion

    Alessio Sampieri, Alessio Palma, Indro Spinelli, and Fabio Galasso. Length-aware motion synthesis via latent diffusion. arXiv preprint arXiv:2407.11532, 2024. 1

    2024 arXiv

  43. [51]

    Human motion diffusion as a generative prior

    Yonatan Shafir, Guy Tevet, Roy Kapon, and Amit H Bermano. Human motion diffusion as a generative prior. arXiv preprint arXiv:2303.01418, 2023. 3, 6

    2023 arXiv

  44. [52]

    Generating physically realistic and directable human motions from multi-modal inputs

    Aayam Shrestha, Pan Liu, German Ros, Kai Yuan, and Alan Fern. Generating physically realistic and directable human motions from multi-modal inputs. In European Conference on Computer Vision, pages 1–17. Springer, 2025. 2, 3

    2025

  45. [53]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 1

    2010 arXiv

  46. [54]

    Loss-guided diffusion models for plug-and-play controllable generation

    Jiaming Song, Qinsheng Zhang, Hongxu Yin, Morteza Mar- dani, Ming-Yu Liu, Jan Kautz, Yongxin Chen, and Arash Vahdat. Loss-guided diffusion models for plug-and-play controllable generation. In International Conference on Ma- chine Learning, pages 32483–32498. PMLR, 2023. 1

    2023

  47. [55]

    Arbitrary motion style transfer with multi-condition motion latent dif- fusion model

    Wenfeng Song, Xingliang Jin, Shuai Li, Chenglizhao Chen, Aimin Hao, Xia Hou, Ning Li, and Hong Qin. Arbitrary motion style transfer with multi-condition motion latent dif- fusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)...

    2024

  48. [56]

    Roformer: Enhanced transformer with rotary position embedding

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063,

  49. [57]

    Lgtm: Local-to-global text- driven human motion diffusion model

    Haowen Sun, Ruikun Zheng, Haibin Huang, Chongyang Ma, Hui Huang, and Ruizhen Hu. Lgtm: Local-to-global text- driven human motion diffusion model. In ACM SIGGRAPH 2024 Conference Papers, pages 1–9, 2024. 1

    2024

  50. [58]

    Motionclip: Exposing human motion generation to clip space

    Guy Tevet, Brian Gordon, Amir Hertz, Amit H Bermano, and Daniel Cohen-Or. Motionclip: Exposing human motion generation to clip space. In European Conference on Com- puter Vision, pages 358–374. Springer, 2022. 1, 2, 3

    2022

  51. [59]

    Human motion diffu- sion model

    Guy Tevet, Sigal Raab, Brian Gordon, Yoni Shafir, Daniel Cohen-or, and Amit Haim Bermano. Human motion diffu- sion model. In The Eleventh International Conference on Learning Representations, 2023. 1, 2, 3, 6

    2023

  52. [60]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017. 2

    2017

  53. [61]

    Frieren: Efficient video-to-audio generation with rectified flow matching

    Yongqi Wang, Wenxiang Guo, Rongjie Huang, Jiawei Huang, Zehan Wang, Fuming You, Ruiqi Li, and Zhou Zhao. Frieren: Efficient video-to-audio generation with rectified flow matching. arXiv preprint arXiv:2406.00320, 2024. 3

    2024 arXiv

  54. [62]

    Motiongpt-2: A general-purpose motion- language model for motion generation and understanding

    Yuan Wang, Di Huang, Yaqi Zhang, Wanli Ouyang, Jile Jiao, Xuetao Feng, Yan Zhou, Pengfei Wan, Shixiang Tang, and Dan Xu. Motiongpt-2: A general-purpose motion- language model for motion generation and understanding. arXiv preprint arXiv:2410.21747, 2024. 3

    arXiv

  55. [63]

    Motionllm: Multimodal motion-language learning with large language models

    Qi Wu, Yubo Zhao, Yifan Wang, Yu-Wing Tai, and Chi- Keung Tang. Motionllm: Multimodal motion-language learning with large language models. arXiv preprint arXiv:2405.17013, 2024. 3

    2024 arXiv

  56. [64]

    Omnicontrol: Control any joint at any time for human motion generation

    Yiming Xie, Varun Jampani, Lei Zhong, Deqing Sun, and Huaizu Jiang. Omnicontrol: Control any joint at any time for human motion generation. arXiv preprint arXiv:2310.08580,

    arXiv

  57. [65]

    Unimumo: Uni- fied text, music and motion generation

    Han Yang, Kun Su, Yutong Zhang, Jiaben Chen, Kaizhi Qian, Gaowen Liu, and Chuang Gan. Unimumo: Uni- fied text, music and motion generation. arXiv preprint arXiv:2410.04534, 2024. 2, 3

    2024 arXiv

  58. [66]

    Generating human motion from textual descrip- tions with discrete representations

    Jianrong Zhang, Yangsong Zhang, Xiaodong Cun, Yong Zhang, Hongwei Zhao, Hongtao Lu, Xi Shen, and Ying Shan. Generating human motion from textual descrip- tions with discrete representations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, ...

    2023

  59. [67]

    Motiondif- fuse: Text-driven human motion generation with diffusion model

    Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondif- fuse: Text-driven human motion generation with diffusion model. arXiv preprint arXiv:2208.15001, 2022. 1, 2, 3, 6

    2022 arXiv

  60. [68]

    Finemogen: Fine-grained spatio- temporal motion generation and editing

    Mingyuan Zhang, Huirong Li, Zhongang Cai, Jiawei Ren, Lei Yang, and Ziwei Liu. Finemogen: Fine-grained spatio- temporal motion generation and editing. Advances in Neural Information Processing Systems, 36:13981–13992, 2023. 1, 3

    2023

  61. [69]

    Large motion model for unified multi-modal motion generation

    Mingyuan Zhang, Daisheng Jin, Chenyang Gu, Fangzhou Hong, Zhongang Cai, Jingfang Huang, Chongzhi Zhang, Xinying Guo, Lei Yang, Ying He, et al. Large motion model for unified multi-modal motion generation. In European Conference on Computer Vision , pages 397–421. Springer,

  62. [70]

    Flowturbo: Towards real-time flow-based image genera- tion with velocity refiner

    Wenliang Zhao, Minglei Shi, Xumin Yu, Jie Zhou, and Jiwen Lu. Flowturbo: Towards real-time flow-based image genera- tion with velocity refiner. arXiv preprint arXiv:2409.18128,

    arXiv

  63. [71]

    Smoodi: Stylized motion diffusion model

    Lei Zhong, Yiming Xie, Varun Jampani, Deqing Sun, and Huaizu Jiang. Smoodi: Stylized motion diffusion model. In European Conference on Computer Vision, pages 405–421. Springer, 2025. 1, 2, 3

    2025

  64. [72]

    Ude: A unified driv- ing engine for human motion generation

    Zixiang Zhou and Baoyuan Wang. Ude: A unified driv- ing engine for human motion generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5632–5641, 2023. 2, 3

    2023

  65. [73]

    A unified frame- work for multimodal, multi-part human motion synthesis

    Zixiang Zhou, Yu Wan, and Baoyuan Wang. A unified frame- work for multimodal, multi-part human motion synthesis. arXiv preprint arXiv:2311.16471, 2023. 2, 3

    2023 arXiv

  66. [74]

    Avatargpt: All- in-one framework for motion understanding planning gener- ation and beyond

    Zixiang Zhou, Yu Wan, and Baoyuan Wang. Avatargpt: All- in-one framework for motion understanding planning gener- ation and beyond. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 1357–1366, 2024. 3 MotionLab: Unified Human Motion ...

    2024

  67. [75]

    Demonstration of the difference between diffusion mod- els and rectified flows

    Details of Rectified Flows source 𝑥0 target 𝑥1 Diffusion Models 𝑥𝑡 𝛼𝑡𝑥1 (1 − 𝛼𝑡)𝑥0 trajectory Rectified Flows 𝑥𝑡 trajectory 𝑡𝑥1 (1 − 𝑡)𝑥0 velocity 𝑣𝑡 source 𝑥0 target 𝑥1 Figure 7. Demonstration of the difference between diffusion mod- els and rectified flows. This difference l...

  68. [76]

    MotionLab Inference During inference, Classifier-Free Guidance (CFG) [27] is incorporated for both motion generation and motion editing to boost sampling quality and align conditions and target motion. For all motion generation tasks, we generate target mo- tion MT with the gu...

  69. [77]

    The memory usage and the time spent during inference are summarized in the following Table 7

    Memory Usage and Time Cost The maximum memory usage during training is 23 GB for each GPU. The memory usage and the time spent during inference are summarized in the following Table 7. Metric text gen traj. gen text edit traj. edit in-between style transfermemory usage (GB) 4....

  70. [78]

    Additional Quantitative Results As shown in Table 8, our framework outperforms CondMDI on all settings, illustrating the effectiveness of our frame- work in motion in-between. Method Frames FID↓ R-precisionTop-3↑ Diversity→ Foot skatingratio↓ Keyframeerror↓ CondMDI [11]1 0.155...

  71. [79]

    w/o task instruction modulation

    Additional Ablation Studies To further validate the designs in our framework, we per- form traditional ablation studies in this section. To further validate the Aligned ROPE, we also intro- duce the variant of 3D-Learnable and 3D-ROPE to distin- guish the source motion, target...

  72. [80]

    Specifically, source motion and target motion are represented as MS ∈ RN ×D and MT ∈ RN ×D, and we first ignore timestep t here

    Representation for Each Modality We represent the features of all modalities as tokens for the attention mechanism [60]. Specifically, source motion and target motion are represented as MS ∈ RN ×D and MT ∈ RN ×D, and we first ignore timestep t here. For the instruction, it is ...

  73. [81]

    Instructions for Each Task As shown in the Table 13, the instructions in the Task In- struction Modulations for each task are presented, which benefits our framework to distinguish different tasks

  74. [82]

    We conduct ablation experiments based on the hyperparameters provided by the baseline and finally obtain the above hyperparameters

    Classifier Free Guidance for Each Task As shown in Table 14, strengths of classifier-free guidance for each task are presented, which contribute to the results’ quality during sampling. We conduct ablation experiments based on the hyperparameters provided by the baseline and f...

  75. [83]

    reconstruct given masked source motion

    3D Assets We have borrowed some 3D assets for our video and figure from the Internet, including Dojo Matrix Drunken Wrestlers, Basketball Court, Grandma‘s Place, DAE Dio- rama retake – Small farm, DAE Diorama retake – Small farm, Japanese Small Shrine Temple 0002. Ablation Stu...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.