Pith. sign in

REVIEW 4 major objections 3 minor 4 cited by

MoLingo claims that a masked auto-regressive rectified-flow text-to-motion model achieves state-of-the-art fidelity and text alignment by semantically aligning the motion latent space and conditioning on multiple text tokens via cross-atten

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 16:18 UTC pith:FMRCLMQ2

load-bearing objection Solid engineering with a useful SAE idea, but the SOTA claim is not supported by the numbers as reported. the 4 major comments →

arxiv 2512.13840 v3 pith:FMRCLMQ2 submitted 2025-12-15 cs.CV

MoLingo: Motion-Language Alignment for Text-to-Human Motion Generation

Yannan He , Garvita Tiwari , Xiaohan Zhang , Pankaj Bora , Tolga Birdal , Jan Eric Lenssen , Gerard Pons-Moll This is my paper
classification cs.CV
keywords text-to-motion generationlatent space diffusionsemantic alignmentcross-attention conditioningauto-regressive generationrectified flowHumanML3DBABEL
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that two design choices determine how well text-to-motion diffusion works in a continuous latent space: the latent space must be organized so that motions sharing text meaning sit close together, and the text condition should be injected through all of its tokens via cross-attention rather than as a single summary token. The authors introduce a semantically aligned autoencoder (SAE) that pulls motion latents toward class tokens derived from frame-level action labels, and a masked auto-regressive rectified-flow generator with multi-token cross-attention. If the claims hold, MoLingo outperforms previous text-to-motion models on HumanML3D, with state-of-the-art FID, R-Precision, CLIP-Score, and a user study favoring it in 77–85 percent of pairwise comparisons. The result matters because faithfully following a text prompt is the main bottleneck in animation, AR/VR avatars, and instruction-following agents.

Core claim

On its own terms, the paper's central claim is that continuous-latent text-to-motion generation is most effective when the latent space is semantically aligned and the text conditioning is multi-token cross-attention. The semantic-aligned autoencoder encodes motion frames into latents while a cosine-similarity loss pulls each latent toward a class token obtained from frame-level BABEL labels, so latents with similar meanings cluster. The generator is a masked auto-regressive transformer with rectified-flow heads that denoise latents conditioned on a T5 text encoding through cross-attention. The SAE variant reports R-Precision Top-1 of 0.542 on the MARDM-67 protocol (compared with 0.522 for t

What carries the argument

The central mechanism is the semantically aligned autoencoder (SAE) plus a masked auto-regressive rectified-flow generator. The SAE is a causal temporal autoencoder whose latent sequence is encouraged, through a soft cosine-similarity loss L_sem, to align with class tokens computed from frame-level text labels; a filtering step drops consecutive latents whose class tokens are near-duplicates (cosine similarity > 0.995) to prevent over-collapse. The generator is a transformer decoder that cross-attends to all text tokens and feeds conditioning vectors to an MLP that performs rectified-flow denoising of each latent. Together these components make the latent space 'diffusion-friendly' and the t

Load-bearing premise

The paper assumes that a soft cosine-similarity loss, trained only on the overlap of BABEL and HumanML3D and filtered by a near-duplicate threshold, produces a latent space that improves text-following for the entire HumanML3D prompt distribution, including compound prompts that string several actions together.

What would settle it

Train the identical generative pipeline with the semantic-alignment loss removed and with the loss applied to randomly permuted class tokens. If R-Precision and CLIP-Score do not drop substantially, the semantic-alignment claim is not the cause of the reported gains. A sharper test: compute R-Precision separately for single-action and multi-action test prompts; if the SAE's advantage appears only on single-action prompts, the claim that alignment improves compositional text-following fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Semantic alignment of a continuous latent space is a transferable recipe: any latent diffusion model with access to per-token or per-frame labels could apply the same soft alignment loss.
  • Multi-token cross-attention conditioning beats single-token conditioning for text-to-motion, suggesting that compressing text into one embedding before conditioning sacrifices faithfulness.
  • Continuous auto-regressive flow with masking is a viable alternative to vector-quantized token prediction, avoiding quantization artifacts while preserving temporal detail.
  • The reported gains persist across three evaluation protocols (MARDM-67, MS-272, TMR-263), so the improvement is not tied to one metric or representation.
  • The method feeds a pre-trained physics-tracking controller, showing that the generated motions are stable enough for simulated robotics transfer.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • I would test whether the semantic-alignment loss is doing what the paper claims: evaluate the SAE and VAE variants separately on compound, multi-action prompts with an action-order accuracy metric; if the SAE advantage vanishes there, the alignment mainly helps single-action descriptions.
  • The same semantic-aligned latent space could serve as a reusable representation for motion retrieval or captioning, not just generation, since it is explicitly organized by text meaning.
  • The finding about multi-token conditioning likely transfers to other fine-grained conditional generation tasks, such as text-to-video or text-to-audio, where pooled tokens are still common.
  • Because the SAE variant trades a slightly worse FID for better R-Precision, a promising extension is adaptive weighting of the semantic loss during training to get both.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper proposes MoLingo, a text-to-motion generation model based on masked auto-regressive rectified-flow diffusion in a continuous latent space. The main contributions are (i) a semantically aligned autoencoder (SAE) trained with frame-level BABEL labels using a cosine-similarity loss to structure the latent space, and (ii) a multi-token cross-attention text conditioning mechanism. Experiments on HumanML3D report quantitative results under multiple evaluators (MARDM-67, TMR-263, MS-272) and a user study, claiming state-of-the-art FID, R-Precision, and CLIP-Score. The paper also includes ablations on conditioning mechanisms, latent dimension, semantic regularization, and repetitive class-token filtering, and reports 95% confidence intervals over 20 runs.

Significance. If substantiated, the paper would make a useful empirical contribution to text-to-motion generation: it systematically compares text-conditioning mechanisms, studies latent-space design choices, and commits to releasing code and models. Reporting confidence intervals over 20 runs is good practice. The cross-attention finding is well supported by Table 2. However, the central SOTA claim is not supported by the reported numbers, as discussed below; the two variants excel on different metrics and no single configuration dominates prior methods.

major comments (4)
  1. [Abstract, §4.3, Table 1] The claim that MoLingo sets a new state of the art on standard metrics is not supported by a single configuration. In Table 1, MoLingo-SAE has FID 0.064±0.002, which is significantly worse than DisCoRD's 0.053±0.004 (95% CIs [0.062,0.066] vs [0.049,0.057] do not overlap). MoLingo-VAE has FID 0.049±0.003, which is not a significant improvement over DisCoRD (the CIs overlap), and this variant does not use the semantic alignment the abstract credits for the result. The R-Precision gain (0.542) is only from the SAE. Thus no single MoLingo model dominates prior methods on both realism and text alignment. Please either provide a single configuration that is SOTA on both metric groups or revise the central claim.
  2. [§4.2 'Effect of SAE', Table 1] The text states that the SAE 'maintains a comparable FID to SOTA models.' This is contradicted by Table 1: SAE FID 0.064±0.002 is 0.011 worse than DisCoRD's 0.053±0.004, with non-overlapping confidence intervals. Semantic alignment comes at a significant FID cost in the current configuration, so the claim that alignment preserves motion quality is not supported.
  3. [§4.3 'Quantitative comparison', Table 1] The reported FID improvement from 0.053 (DisCoRD) to 0.049 (MoLingo-VAE) is not statistically significant given the reported 95% CIs: the intervals [0.049,0.057] and [0.046,0.052] overlap. A paired or bootstrap hypothesis test on the 20 runs should be reported before claiming a reduction. Comparing the SAE FID with DisCoRD, a formal significance test would also strengthen the discussion.
  4. [§3.1 Eq. (1), Table 1] The semantic alignment loss is a cosine-similarity objective between motion latents and class tokens derived from text embeddings. The R-Precision metric measures retrieval of text from motion using cross-modal embeddings, also based on cosine similarity in a learned space. Thus the SAE's large R-Precision gain is partly a direct consequence of training toward the same objective family used by the evaluator. To establish that semantic alignment improves compositional text understanding (e.g., 'run, turn around, cartwheel'), please evaluate on compound prompts not covered by the BABEL label set, or use a retrieval evaluator whose text encoder/embedding space is not aligned by the training loss. Without such evidence, the claim that SAE improves text-following is not isolated from the metric being optimized.
minor comments (3)
  1. [Figure 5 caption] The caption is internally contradictory: it says 'the 4× setting gives comparable or better performance than 2×, showing that is beneficial that a single latent encodes 4 frame, even 2× preserves more fine-grained temporal information.' Please clarify the intended conclusion.
  2. [§4.1 Evaluation protocol] 'piorneering' should be 'pioneering'.
  3. [Table 7] The 'Matching Score' column is labeled with a down arrow, but the Real row has a high value and MoLingo a slightly lower value; if lower is better, this is unusual for a matching score and should be clarified.

Circularity Check

0 steps flagged

No significant circularity; semantic-alignment loss is related to but not identical to the retrieval metric.

full rationale

MoLingo's pipeline is a standard empirical construction: a causal motion autoencoder, a semantic cosine loss (Eq. 1) against T5-derived class tokens from BABEL frame labels, and an auto-regressive rectified-flow transformer with T5 cross-attention. The claimed outputs are FID, R-Precision, CLIP-Score, and a user study on the held-out HumanML3D test set, compared against external baselines under fixed evaluators. The possible circularity is the semantic loss: it optimizes cosine similarity between SAE latents and text-derived class tokens, while R-Precision also measures cosine-similarity retrieval. However, the two are not the same objective by construction. Eq. 1 operates in the SAE latent space, with frame-level T5 label embeddings averaged over temporal windows, whereas R-Precision is computed by an external evaluator's pre-trained motion/text feature extractor on full test prompts and decoded motions. No fitted parameter is renamed as a prediction, and no equation reduces to its own input. The only self-citation (UniMotion [29]) appears in related work for limitations of the 263D representation and is not load-bearing. The abstract's SOTA wording is imprecise because no single configuration simultaneously achieves the best FID and best R-Precision — the SAE variant wins text alignment but has FID 0.064, losing to DisCoRD's 0.053, while the VAE variant wins FID without semantic alignment. That is a correctness/consistency concern, not circularity. The derivation chain itself is self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The central claim rests on two public datasets (HumanML3D, BABEL) and a set of evaluators. The method introduces no new physical or conceptual entities; it learns a semantically aligned latent representation. However, several hyperparameters (λ_sem, τ, adapter depth, latent dimensions, CFG) are fitted using test-set metrics, and the evaluation is assumed to be unbiased despite potential autoencoder leakage and metric sensitivity issues.

free parameters (6)
  • λ_sem (semantic loss weight) = 0.001
    Weight of the cosine-similarity semantic alignment loss; selected via ablation in Table 3, evaluating on test-set metrics.
  • τ (repetitive class-token filtering threshold) = 0.995
    Cosine-similarity threshold above which adjacent class tokens are discarded; chosen without a sensitivity analysis.
  • adapter depth = 6 layers
    Depth of the text adapter transformer; selected via ablation in Table 5.
  • latent dimension and temporal downsampling = d=16, 4× downsampling
    Chosen from the latent-size ablation in Figure 5, though the main text and figure caption contradict each other on whether 2× or 4× is better.
  • CFG scale = 6.0 (7.0 for 272D)
    Classifier-free guidance scale at inference; selected for best performance.
  • denoising steps / inference steps = 32 / 16
    Fixed values used in all experiments; no ablation is reported.
axioms (5)
  • domain assumption HumanML3D provides reliable paired text-motion data for training and evaluation.
    The entire method trains and evaluates on HumanML3D; if its annotations or splits are flawed, the reported results are not meaningful.
  • domain assumption BABEL frame-level labels are semantically accurate and temporally aligned with motion.
    The SAE relies on BABEL labels as supervision; if these labels are noisy or misaligned, the learned latent space may be incorrectly structured.
  • domain assumption FID, R-Precision, and CLIP-Score are valid proxies for motion realism and text alignment.
    The paper uses these metrics to claim SOTA; if the metrics are not sensitive or reliable, the central claim is unsupported. The paper itself notes limitations of some evaluators.
  • domain assumption Training the autoencoder on the full HumanML3D dataset, including test-set motions, does not introduce material leakage.
    The SAE is trained with reconstruction losses on all data; if test motions are seen during autoencoder training, FID and reconstruction metrics could be artificially inflated. This is not discussed in the paper.
  • domain assumption T5-Large and the evaluator feature extractors provide useful embeddings for semantic alignment.
    The SAE uses T5 text embeddings as alignment targets; the evaluators use different pre-trained encoders. The transferability of the alignment is assumed but not proven.

pith-pipeline@v1.3.0-alltime-deepseek · 20244 in / 13857 out tokens · 113800 ms · 2026-08-03T16:18:54.438222+00:00 · methodology

0 comments
read the original abstract

We introduce MoLingo, a text-to-motion (T2M) model that generates realistic, lifelike human motion by denoising in a continuous latent space. Recent works perform latent space diffusion, either on the whole latent at once or auto-regressively over multiple latents. In this paper, we study how to make diffusion on continuous motion latents work best. We focus on two questions: (1) how to build a semantically aligned latent space so diffusion becomes more effective, and (2) how to best inject text conditioning so the motion follows the description closely. We propose a semantic-aligned motion encoder trained with frame-level text labels so that latents with similar text meaning stay close, which makes the latent space more diffusion-friendly. We also compare single-token conditioning with a multi-token cross-attention scheme and find that cross-attention gives better motion realism and text-motion alignment. With semantically aligned latents, auto-regressive generation, and cross-attention text conditioning, our model sets a new state of the art in human motion generation on standard metrics and in a user study. We will release our code and models for further research and downstream usage.

Figures

Figures reproduced from arXiv: 2512.13840 by Garvita Tiwari, Gerard Pons-Moll, Jan Eric Lenssen, Pankaj Bora, Tolga Birdal, Xiaohan Zhang, Yannan He.

Figure 1
Figure 1. Figure 1: Left: Given text prompts, MoLingo generates realistic and text-aligned motions, ranging from daily movements like sweeping to more challenging movements like dancing. Right: MoLingo significantly outperforms previous works in both FID and R-Precision scores. The difference can best be seen in motion, hence we urge the reader to view the supplementary video. Abstract We introduce MoLingo, a text-to-motion (… view at source ↗
Figure 3
Figure 3. Figure 3: Auto-regressive flow-based latent denoising. Our generation model uses a standard transformer decoder to obtain conditioning vectors z, which guides an MLP in iteratively refining latents. During training the motion latents are randomly masked and replaced with learnable tokens. During inference, we initialize with fully masked latents, iteratively denoise them, and decode the final latents to obtain the g… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparisons with MARDM [39], ACMDM [40], and MotionStreamer [62]. The color transitions from light to dark to indicate temporal flow. Our method generates more natural and text-aligned motions, ranging from everyday movements to challenging scenarios like cartwheel. In contrast, other methods either fail to follow fine-grained text instructions (e.g., run first, then turn around, then do a cart… view at source ↗
Figure 5
Figure 5. Figure 5: Effect of latent dimension and temporal downsam￾pling. We vary the latent dimension (16–128) under two temporal compression settings: (4×) and (2×). Overall, the 4× setting gives comparable or better performance than 2×, showing that is benefi￾cial that a single latent encodes 4 frame, even 2× preserves more fine-grained temporal information. ferences, we recommend viewing the supplementary videos for a be… view at source ↗
Figure 6
Figure 6. Figure 6: Incorporating MoLingo with RL Tracking Con￾troller Our method produces motion with consistently realistic foot–ground interaction across the entire sequence. In contrast, Mo￾tionStreamer frequently exhibits balance artifacts. By leveraging MoLingo within the RL tracking controller, our results maintain stable, physically plausible floor contact, leading to more balanced, grounded, and visually convincing c… view at source ↗
Figure 7
Figure 7. Figure 7: User study interface. text–motion consistency by checking whether generated mo￾tion embeddings retrieve their paired text prompts among the top candidates. (3) MultiModality (MModality) reflects variation under identical prompts, measuring how diverse the produced motion embeddings are when generating multiple motions for the same text. (4) CLIP-Score measures textual faithfulness via CLIP [49], computed a… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. CoMoVi: Co-Generation of 3D Human Motions and Realistic Videos

    cs.CV 2026-01 unverdicted novelty 7.0

    CoMoVi co-generates 3D human motions and 2D videos synchronously in a single diffusion denoising loop using 3D-to-2D projection and dual-branch diffusion with 3D-2D cross attentions.

  2. SCRIPT: Scalable Diffusion Policy with Multi-stage Training for Language-driven Physics-based Humanoid Control

    cs.GR 2026-05 unverdicted novelty 6.0

    A new diffusion transformer policy with joint attention over actions, states, and text plus RL post-training outperforms prior methods on language alignment and motion quality for humanoid control.

  3. SCRIPT: Scalable Diffusion Policy with Multi-stage Training for Language-driven Physics-based Humanoid Control

    cs.GR 2026-05 unverdicted novelty 6.0

    SCRIPT presents a scalable diffusion policy with JAST-DiT architecture, nonlinear history conditioning, and RLHR post-training that claims to outperform prior methods on text alignment, motion quality, and physical re...

  4. Exploring Motion-Language Alignment for Text-driven Motion Generation

    cs.CV 2026-04 unverdicted novelty 5.0

    MLA-Gen advances text-driven motion synthesis by aligning global motion patterns with fine-grained text semantics and mitigating attention sink effects via new masking techniques.

Reference graph

Works this paper leans on

78 extracted references · 19 linked inside Pith · cited by 3 Pith papers

  1. [1]

    TMR++: A cross-dataset study for text-based 3d human motion re- trieval

    Léore Bensabath, Mathis Petrovich, and Gül Varol. TMR++: A cross-dataset study for text-based 3d human motion re- trieval. InCVPR Workshop on Human Motion Generation,

  2. [2]

    Keep it smpl: Automatic estimation of 3d human pose and shape from a single image

    Federica Bogo, Angjoo Kanazawa, Christoph Lassner, Peter Gehler, Javier Romero, and Michael J Black. Keep it smpl: Automatic estimation of 3d human pose and shape from a single image. InEuropean conference on computer vision, pages 561–578. Springer, 2016. 6

    2016

  3. [3]

    The language of motion: Unifying verbal and non- verbal language of 3d human motion

    Changan Chen, Juze Zhang, Shrinidhi K Lakshmikanth, Yusu Fang, Ruizhi Shao, Gordon Wetzstein, Li Fei-Fei, and Ehsan Adeli. The language of motion: Unifying verbal and non- verbal language of 3d human motion. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 6200–6211, 2025. 2

    2025

  4. [4]

    Taming diffusion probabilistic models for character control

    Rui Chen, Mingyi Shi, Shaoli Huang, Ping Tan, Taku Komura, and Xuelin Chen. Taming diffusion probabilistic models for character control. InACM SIGGRAPH 2024 Conference Pa- pers, New York, NY , USA, 2024. Association for Computing Machinery. 1, 2

    2024

  5. [5]

    Free-t2m: Frequency enhanced text-to-motion diffusion model with consistency loss.arXiv preprint arXiv:2501.18232, 2025

    Wenshuo Chen, Haozhe Jia, Songning Lai, Keming Wu, Hon- gru Xiao, Lijie Hu, and Yutao Yue. Free-t2m: Frequency enhanced text-to-motion diffusion model with consistency loss.arXiv preprint arXiv:2501.18232, 2025. 2

    arXiv 2025

  6. [6]

    Executing your commands via motion diffusion in latent space

    Xin Chen, Biao Jiang, Wen Liu, Zilong Huang, Bin Fu, Tao Chen, and Gang Yu. Executing your commands via motion diffusion in latent space. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18000–18010, 2023. 2, 4, 3

    2023

  7. [7]

    Dis- cord: Discrete tokens to continuous motion via rectified flow decoding.arXiv preprint arXiv:2411.19527, 2024

    Jungbin Cho, Junwan Kim, Jisoo Kim, Minseo Kim, Mingu Kang, Sungeun Hong, Tae-Hyun Oh, and Youngjae Yu. Dis- cord: Discrete tokens to continuous motion via rectified flow decoding.arXiv preprint arXiv:2411.19527, 2024. 4, 8, 2, 3

    Pith/arXiv arXiv 2024

  8. [8]

    Mofusion: A framework for denoising-diffusion-based motion synthesis

    Rishabh Dabral, Muhammad Hamza Mughal, Vladislav Golyanik, and Christian Theobalt. Mofusion: A framework for denoising-diffusion-based motion synthesis. InComputer Vision and Pattern Recognition (CVPR), 2023. 1, 2

    2023

  9. [9]

    Motionlcm-v2: Improved compression rate for multi-latent-token diffusion,

    Wenxun Dai, Ling-Hao Chen, Yufei Huo, Jingbo Wang, Jinpeng Liu, Bo Dai, and Yansong Tang. Motionlcm-v2: Improved compression rate for multi-latent-token diffusion,

  10. [10]

    Motionlcm: Real-time control- lable motion generation via latent consistency model.arXiv preprint arXiv:2404.19759, 2024

    Wenxun Dai, Ling-Hao Chen, Jingbo Wang, Jinpeng Liu, Bo Dai, and Yansong Tang. Motionlcm: Real-time control- lable motion generation via latent consistency model.arXiv preprint arXiv:2404.19759, 2024. 2

    Pith/arXiv arXiv 2024

  11. [11]

    Sigmoid- weighted linear units for neural network function approx- imation in reinforcement learning.Neural networks, 107: 3–11, 2018

    Stefan Elfwing, Eiji Uchibe, and Kenji Doya. Sigmoid- weighted linear units for neural network function approx- imation in reinforcement learning.Neural networks, 107: 3–11, 2018. 1

    2018

  12. [12]

    Fluid: Scaling autoregressive text-to-image generative models with continuous tokens.arXiv preprint arXiv:2410.13863, 2024

    Lijie Fan, Tianhong Li, Siyang Qin, Yuanzhen Li, Chen Sun, Michael Rubinstein, Deqing Sun, Kaiming He, and Yonglong Tian. Fluid: Scaling autoregressive text-to-image generative models with continuous tokens.arXiv preprint arXiv:2410.13863, 2024. 5

    Pith/arXiv arXiv 2024

  13. [13]

    Remos: 3d motion- conditioned reaction synthesis for two-person interactions

    Anindita Ghosh, Rishabh Dabral, Vladislav Golyanik, Chris- tian Theobalt, and Philipp Slusallek. Remos: 3d motion- conditioned reaction synthesis for two-person interactions. In European Conference on Computer Vision (ECCV), 2024. 1

    2024

  14. [14]

    Duetgen: Music driven two-person dance generation via hierarchical masked modeling

    Anindita Ghosh, Bing Zhou, Rishabh Dabral, Jian Wang, Vladislav Golyanik, Christian Theobalt, Philipp Slusallek, and Chuan Guo. Duetgen: Music driven two-person dance generation via hierarchical masked modeling. InProceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers, pages 1–11, 2025. 2

    2025

  15. [15]

    Ac- tion2motion: Conditioned generation of 3d human motions

    Chuan Guo, Xinxin Zuo, Sen Wang, Shihao Zou, Qingyao Sun, Annan Deng, Minglun Gong, and Li Cheng. Ac- tion2motion: Conditioned generation of 3d human motions. InProceedings of the 28th ACM International Conference on Multimedia, pages 2021–2029, 2020. 5

    2021

  16. [16]

    Generating diverse and natural 3d human motions from text

    Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 1, 2, 5, 3

    2022

  17. [17]

    Tm2t: Stochastic and tokenized modeling for the reciprocal genera- tion of 3d human motions and texts

    Chuan Guo, Xinxin Zuo, Sen Wang, and Li Cheng. Tm2t: Stochastic and tokenized modeling for the reciprocal genera- tion of 3d human motions and texts. InECCV, 2022. 2

    2022

  18. [18]

    Momask: Generative masked modeling of 3d human motions

    Chuan Guo, Yuxuan Mu, Muhammad Gohar Javed, Sen Wang, and Li Cheng. Momask: Generative masked modeling of 3d human motions. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1900– 1910, 2024. 2, 3, 4, 8

    1900

  19. [19]

    Snap- mogen: Human motion generation from expressive texts

    Chuan Guo, Inwoo Hwang, Jian Wang, and Bing Zhou. Snap- mogen: Human motion generation from expressive texts. arXiv preprint arXiv:2507.09122, 2025. 2, 3, 4

    arXiv 2025

  20. [20]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bern- hard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017. 2

    2017

  21. [21]

    Egolm: Multi-modal language model of egocentric motions.arXiv preprint arXiv:2409.18127, 2024

    Fangzhou Hong, Vladimir Guzov, Hyo Jin Kim, Yuting Ye, Richard Newcombe, Ziwei Liu, and Lingni Ma. Egolm: Multi-modal language model of egocentric motions.arXiv preprint arXiv:2409.18127, 2024. 2

    Pith/arXiv arXiv 2024

  22. [22]

    Stablemofusion: Towards robust and efficient diffusion-based motion generation framework

    Yiheng Huang, Hui Yang, Chuanchen Luo, Yuxi Wang, Shib- iao Xu, Zhaoxiang Zhang, Man Zhang, and Junran Peng. Stablemofusion: Towards robust and efficient diffusion-based motion generation framework. InProceedings of the 32nd ACM International Conference on Multimedia, pages 224– 232, 2024. 1

    2024

  23. [23]

    Intermask: 3d human interaction generation via collabo- rative masked modelling.arXiv preprint arXiv:2410.10010,

    Muhammad Gohar Javed, Chuan Guo, Li Cheng, and Xingyu Li. Intermask: 3d human interaction generation via collabo- rative masked modelling.arXiv preprint arXiv:2410.10010,

    Pith/arXiv arXiv

  24. [24]

    Motiongpt: Human motion as a foreign language

    Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, and Tao Chen. Motiongpt: Human motion as a foreign language. Advances in Neural Information Processing Systems, 36, 2024. 2, 3

    2024

  25. [25]

    Motionpcm: Real-time motion synthesis with phased consistency model.arXiv preprint arXiv:2501.19083, 2025

    Lei Jiang, Ye Wei, and Hao Ni. Motionpcm: Real-time motion synthesis with phased consistency model.arXiv preprint arXiv:2501.19083, 2025. 2

    Pith/arXiv arXiv 2025

  26. [26]

    Guided motion diffusion for con- trollable human motion synthesis

    Korrawe Karunratanakul, Konpat Preechakul, Supasorn Suwa- janakorn, and Siyu Tang. Guided motion diffusion for con- trollable human motion synthesis. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 2151–2162, 2023. 1, 2

    2023

  27. [27]

    Latent diffusion models with masked autoencoders

    Junho Lee, Jeongwoo Shin, Hyungwook Choi, and Joonseok Lee. Latent diffusion models with masked autoencoders. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17422–17431, 2025. 3

    2025

  28. [28]

    Repa-e: Unlocking vae for end-to-end tuning with latent diffusion transformers

    Xingjian Leng, Jaskirat Singh, Yunzhong Hou, Zhenchang Xing, Saining Xie, and Liang Zheng. Repa-e: Unlocking vae for end-to-end tuning with latent diffusion transformers. arXiv preprint arXiv:2504.10483, 2025. 2, 3

    arXiv 2025

  29. [29]

    Unimotion: Unifying 3d human motion synthesis and understanding.arXiv preprint arXiv:2409.15904, 2024

    Chuqiao Li, Julian Chibane, Yannan He, Naama Pearl, An- dreas Geiger, and Gerard Pons-Moll. Unimotion: Unifying 3d human motion synthesis and understanding.arXiv preprint arXiv:2409.15904, 2024. 1, 2, 3

    Pith/arXiv arXiv 2024

  30. [30]

    Autoregressive image generation without vector quantization

    Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization. InAdvances in Neural Information Processing Systems, 2024. 2, 5

    2024

  31. [31]

    Intergen: Diffusion-based multi-human motion gener- ation under complex interactions.International Journal of Computer Vision, pages 1–21, 2024

    Han Liang, Wenqian Zhang, Wenxuan Li, Jingyi Yu, and Lan Xu. Intergen: Diffusion-based multi-human motion gener- ation under complex interactions.International Journal of Computer Vision, pages 1–21, 2024. 1, 2

    2024

  32. [32]

    Character controllers using motion vaes.ACM Trans

    Hung Yu Ling, Fabio Zinno, George Cheng, and Michiel Van De Panne. Character controllers using motion vaes.ACM Trans. Graph., 2020. 2

    2020

  33. [33]

    Gesturelsm: Latent shortcut based co-speech gesture generation with spatial-temporal modeling.arXiv preprint arXiv:2501.18898, 2025

    Pinxin Liu, Luchuan Song, Junhua Huang, Haiyang Liu, and Chenliang Xu. Gesturelsm: Latent shortcut based co-speech gesture generation with spatial-temporal modeling.arXiv preprint arXiv:2501.18898, 2025. 2

    Pith/arXiv arXiv 2025

  34. [34]

    Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003, 2022

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003, 2022. 3, 5

    Pith/arXiv arXiv 2022

  35. [35]

    Matthew Loper, Naureen Mahmood, Javier Romero, Ger- ard Pons-Moll, and Michael J. Black. SMPL: A skinned multi-person linear model.ACM Trans. Graphics (Proc. SIG- GRAPH Asia), 34, 2015. 6

    2015

  36. [36]

    Perpetual humanoid control for real-time simulated avatars

    Zhengyi Luo, Jinkun Cao, Kris Kitani, Weipeng Xu, et al. Perpetual humanoid control for real-time simulated avatars. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 10895–10904, 2023. 2

    2023

  37. [37]

    Troje, Ger- ard Pons-Moll, and Michael J

    Naureen Mahmood, Nima Ghorbani, Nikolaus F. Troje, Ger- ard Pons-Moll, and Michael J. Black. AMASS: archive of motion capture as surface shapes. InIEEE/CVF International Conference on Computer Vision, ICCV, 2019. 1, 2, 3, 5

    2019

  38. [38]

    Isaac gym: High performance gpu-based physics simulation for robot learning.arXiv preprint arXiv:2108.10470, 2021

    Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, et al. Isaac gym: High performance gpu-based physics simulation for robot learning.arXiv preprint arXiv:2108.10470, 2021. 2

    Pith/arXiv arXiv 2021

  39. [39]

    Rethinking diffusion for text-driven human motion generation.arXiv preprint arXiv:2411.16575, 2024

    Zichong Meng, Yiming Xie, Xiaogang Peng, Zeyu Han, and Huaizu Jiang. Rethinking diffusion for text-driven human motion generation.arXiv preprint arXiv:2411.16575, 2024. 2, 3, 4, 5, 6, 7, 1

    Pith/arXiv arXiv 2024

  40. [40]

    Absolute coordinates make motion generation easy.arXiv preprint arXiv:2505.19377, 2025

    Zichong Meng, Zeyu Han, Xiaogang Peng, Yiming Xie, and Huaizu Jiang. Absolute coordinates make motion generation easy.arXiv preprint arXiv:2505.19377, 2025. 2, 4, 6, 7, 3

    Pith/arXiv arXiv 2025

  41. [41]

    Semantic-vae: Semantic- alignment latent representation for better speech synthesis

    Zhikang Niu, Shujie Hu, Jeongsoo Choi, Yushen Chen, Pein- ing Chen, Pengcheng Zhu, Yunting Yang, Bowen Zhang, Jian Zhao, Chunhui Wang, et al. Semantic-vae: Semantic- alignment latent representation for better speech synthesis. arXiv preprint arXiv:2509.22167, 2025. 3

    arXiv 2025

  42. [42]

    Black, and Gül Varol

    Mathis Petrovich, Michael J. Black, and Gül Varol. TMR: Text-to-motion retrieval using contrastive 3D human motion synthesis. InInternational Conference on Computer Vision (ICCV), 2023. 2, 3, 7, 1

    2023

  43. [43]

    Stmc: Multi- track timeline control for text-driven 3d human motion gener- ation.arXiv preprint arXiv:2401.08559, 2024

    Mathis Petrovich, Or Litany, Umar Iqbal, Michael J Black, Gül Varol, Xue Bin Peng, and Davis Rempe. Stmc: Multi- track timeline control for text-driven 3d human motion gener- ation.arXiv preprint arXiv:2401.08559, 2024. 2, 3

    Pith/arXiv arXiv 2024

  44. [44]

    Bamm: bidirectional autoregressive motion model

    Ekkasit Pinyoanuntapong, Muhammad Usama Saleem, Pu Wang, Minwoo Lee, Srijan Das, and Chen Chen. Bamm: bidirectional autoregressive motion model. InEuropean Con- ference on Computer Vision, pages 172–190. Springer, 2024. 2, 3, 4

    2024

  45. [45]

    Mmm: Generative masked motion model

    Ekkasit Pinyoanuntapong, Pu Wang, Minwoo Lee, and Chen Chen. Mmm: Generative masked motion model. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 2, 3, 4

    2024

  46. [46]

    Maskcon- trol: Spatio-temporal control for masked motion synthesis

    Ekkasit Pinyoanuntapong, Muhammad Saleem, Korrawe Karunratanakul, Pu Wang, Hongfei Xue, Chen Chen, Chuan Guo, Junli Cao, Jian Ren, and Sergey Tulyakov. Maskcon- trol: Spatio-temporal control for masked motion synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9955–9965, 2025. 2

    2025

  47. [47]

    The kit motion-language dataset.Big data, 4(4):236–252,

    Matthias Plappert, Christian Mandery, and Tamim Asfour. The kit motion-language dataset.Big data, 4(4):236–252,

  48. [48]

    Punnakkal, Arjun Chandrasekaran, Nikos Athanasiou, Alejandra Quiros-Ramirez, and Michael J

    Abhinanda R. Punnakkal, Arjun Chandrasekaran, Nikos Athanasiou, Alejandra Quiros-Ramirez, and Michael J. Black. BABEL: Bodies, action and behavior with english labels. InProceedings IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2021. 4, 5

    2021

  49. [49]

    Learning transferable visual models from natural language supervision, 2021

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. 2

    2021

  50. [50]

    Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of machine learning research, 21(140):1–67, 2020

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of machine learning research, 21(140):1–67, 2020. 5

    2020

  51. [51]

    Humor: 3d human motion model for robust pose estimation

    Davis Rempe, Tolga Birdal, Aaron Hertzmann, Jimei Yang, Srinath Sridhar, and Leonidas J Guibas. Humor: 3d human motion model for robust pose estimation. InProceedings of the IEEE/CVF international conference on computer vision, pages 11488–11499, 2021. 2

    2021

  52. [52]

    Priormdm: Human motion diffusion as a generative prior

    Yoni Shafir, Guy Tevet, Roy Kapon, and Amit Haim Bermano. Priormdm: Human motion diffusion as a generative prior. In ICLR, 2023. 2

    2023

  53. [53]

    Human motion diffusion model

    Guy Tevet, Sigal Raab, Brian Gordon, Yoni Shafir, Daniel Cohen-or, and Amit Haim Bermano. Human motion diffusion model. InThe Eleventh International Conference on Learning Representations, 2023. 1, 2, 4, 3

    2023

  54. [54]

    Mujoco: A physics engine for model-based control

    Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In2012 IEEE/RSJ international conference on intelligent robots and systems, pages 5026–5033. IEEE, 2012. 2

    2012

  55. [55]

    Autoregressive motion generation with gaussian mixture-guided latent sampling

    Linnan Tu, Lingwei Meng, Zongyi Li, Hefei Ling, and Shi- juan Huang. Autoregressive motion generation with gaussian mixture-guided latent sampling. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems. 2

  56. [56]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017. 1

    2017

  57. [57]

    What is the best automated metric for text to motion generation? InSIGGRAPH Asia 2023 Conference Papers, pages 1–11, 2023

    Jordan V oas, Yili Wang, Qixing Huang, and Raymond Mooney. What is the best automated metric for text to motion generation? InSIGGRAPH Asia 2023 Conference Papers, pages 1–11, 2023. 2

    2023

  58. [58]

    Tlcontrol: Trajectory and language control for human motion synthesis

    Weilin Wan, Zhiyang Dou, Taku Komura, Wenping Wang, Dinesh Jayaraman, and Lingjie Liu. Tlcontrol: Trajectory and language control for human motion synthesis. InEuropean Conference on Computer Vision, pages 37–54. Springer, 2024. 2

    2024

  59. [59]

    Aligning motion generation with human perceptions

    Haoru Wang, Wentao Zhu, Luyi Miao, Yishu Xu, Feng Gao, Qi Tian, and Yizhou Wang. Aligning motion generation with human perceptions. InInternational Conference on Learning Representations (ICLR), 2025. 2

    2025

  60. [60]

    Mo- tiondreamer: One-to-many motion synthesis with localized generative masked transformer

    Yilin Wang, chuan guo, Yuxuan Mu, Muhammad Gohar Javed, Xinxin Zuo, Juwei Lu, Hai Jiang, and Li cheng. Mo- tiondreamer: One-to-many motion synthesis with localized generative masked transformer. InThe Thirteenth Interna- tional Conference on Learning Representations, 2025. 2

    2025

  61. [61]

    Representation entanglement for generation: Training diffusion transformers is much easier than you think

    Ge Wu, Shen Zhang, Ruijing Shi, Shanghua Gao, Zhenyuan Chen, Lei Wang, Zhaowei Chen, Hongcheng Gao, Yao Tang, Jian Yang, et al. Representation entanglement for generation: Training diffusion transformers is much easier than you think. arXiv preprint arXiv:2507.01467, 2025. 2, 3

    arXiv 2025

  62. [62]

    Motionstreamer: Streaming motion genera- tion via diffusion-based autoregressive model in causal latent space.arXiv preprint arXiv:2503.15451, 2025

    Lixing Xiao, Shunlin Lu, Huaijin Pi, Ke Fan, Liang Pan, Yueer Zhou, Ziyong Feng, Xiaowei Zhou, Sida Peng, and Jingbo Wang. Motionstreamer: Streaming motion genera- tion via diffusion-based autoregressive model in causal latent space.arXiv preprint arXiv:2503.15451, 2025. 2, 3, 4, 5, 6, 7, 8, 1

    Pith/arXiv arXiv 2025

  63. [63]

    Representa- tion alignment for generation: Training diffusion transform- ers is easier than you think

    Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representa- tion alignment for generation: Training diffusion transform- ers is easier than you think. InInternational Conference on Learning Representations, 2025. 2, 3

    2025

  64. [64]

    Geometric neural distance fields for learning human motion priors.arXiv preprint arXiv:2509.09667, 2025

    Zhengdi Yu, Simone Foti, Linguang Zhang, Amy Zhao, Cem Keskin, Stefanos Zafeiriou, and Tolga Birdal. Geometric neural distance fields for learning human motion priors.arXiv preprint arXiv:2509.09667, 2025. 2

    Pith/arXiv arXiv 2025

  65. [65]

    Mogents: Motion generation based on spatial-temporal joint modeling.Advances in Neural Information Processing Sys- tems, 37:130739–130763, 2024

    Weihao Yuan, Yisheng He, Weichao Shen, Yuan Dong, Xi- aodong Gu, Zilong Dong, Liefeng Bo, and Qixing Huang. Mogents: Motion generation based on spatial-temporal joint modeling.Advances in Neural Information Processing Sys- tems, 37:130739–130763, 2024. 4, 3

    2024

  66. [66]

    T2m-gpt: Generating human motion from textual descriptions with discrete representations

    Jianrong Zhang, Yangsong Zhang, Xiaodong Cun, Shaoli Huang, Yong Zhang, Hongwei Zhao, Hongtao Lu, and Xi Shen. T2m-gpt: Generating human motion from textual descriptions with discrete representations. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 2, 3

    2023

  67. [67]

    Motiondiffuse: Text- driven human motion generation with diffusion model.arXiv preprint arXiv:2208.15001, 2022

    Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondiffuse: Text- driven human motion generation with diffusion model.arXiv preprint arXiv:2208.15001, 2022. 1, 2

    Pith/arXiv arXiv 2022

  68. [68]

    Re- modiffuse: Retrieval-augmented motion diffusion model

    Mingyuan Zhang, Xinying Guo, Liang Pan, Zhongang Cai, Fangzhou Hong, Huirong Li, Lei Yang, and Ziwei Liu. Re- modiffuse: Retrieval-augmented motion diffusion model. arXiv preprint arXiv:2304.01116, 2023. 1

    Pith/arXiv arXiv 2023

  69. [69]

    Finemogen: Fine-grained spatio- temporal motion generation and editing.NeurIPS, 36, 2024

    Mingyuan Zhang, Huirong Li, Zhongang Cai, Jiawei Ren, Lei Yang, and Ziwei Liu. Finemogen: Fine-grained spatio- temporal motion generation and editing.NeurIPS, 36, 2024

    2024

  70. [70]

    Large motion model for unified multi-modal motion generation

    Mingyuan Zhang, Daisheng Jin, Chenyang Gu, Fangzhou Hong, Zhongang Cai, Jingfang Huang, Chongzhi Zhang, Xinying Guo, Lei Yang, Ying He, et al. Large motion model for unified multi-modal motion generation. InEuropean Con- ference on Computer Vision, pages 397–421. Springer, 2025. 2

    2025

  71. [71]

    Kinmo: Kinematic-aware human motion understanding and generation

    Pengfei Zhang, Pinxin Liu, Pablo Garrido, Hyeongwoo Kim, and Bindita Chaudhuri. Kinmo: Kinematic-aware human motion understanding and generation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 11187–11197, 2025. 2

    2025

  72. [72]

    Flashmo: Geometric interpolants and frequency-aware sparsity for scalable efficient motion gen- eration

    Zeyu Zhang, Yiran Wang, Danning Li, Dong Gong, Ian Reid, and Richard Hartley. Flashmo: Geometric interpolants and frequency-aware sparsity for scalable efficient motion gen- eration. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems. 2

  73. [73]

    Motion mamba: Efficient and long se- quence motion generation with hierarchical and bidirectional selective ssm.arXiv preprint arXiv:2403.07487, 2024

    Zeyu Zhang, Akide Liu, Ian Reid, Richard Hartley, Bohan Zhuang, and Hao Tang. Motion mamba: Efficient and long se- quence motion generation with hierarchical and bidirectional selective ssm.arXiv preprint arXiv:2403.07487, 2024. 2

    Pith/arXiv arXiv 2024

  74. [74]

    A diffusion-based autoregressive motion model for real-time text-driven mo- tion control

    Kaifeng Zhao, Gen Li, and Siyu Tang. A diffusion-based autoregressive motion model for real-time text-driven mo- tion control. InThe Thirteenth International Conference on Learning Representations (ICLR 2025), 2025. 2

    2025

  75. [75]

    Emdm: Efficient motion dif- fusion model for fast, high-quality motion generation.arXiv preprint arXiv:2312.02256, 2023

    Wenyang Zhou, Zhiyang Dou, Zeyu Cao, Zhouyingcheng Liao, Jingbo Wang, Wenjia Wang, Yuan Liu, Taku Komura, Wenping Wang, and Lingjie Liu. Emdm: Efficient motion dif- fusion model for fast, high-quality motion generation.arXiv preprint arXiv:2312.02256, 2023. 1, 2

    Pith/arXiv arXiv 2023

  76. [76]

    Mo- tiongpt3: Human motion as a second modality.arXiv preprint arXiv:2506.24086, 2025

    Bingfan Zhu, Biao Jiang, Sunyi Wang, Shixiang Tang, Tao Chen, Linjie Luo, Youyi Zheng, and Xin Chen. Mo- tiongpt3: Human motion as a second modality.arXiv preprint arXiv:2506.24086, 2025. 2, 3 MoLingo: Motion–Language Alignment for Text-to-Human Motion Generation Supplementary Material Table 4.Notation Table. The main notation used in our paper. Symbol De...

    arXiv 2025

  77. [77]

    More Implementation Details Autoencoder training.We adopt the causal autoencoder ar- chitecture from [62] with a hidden size of 1024, and provide a detailed specification in Tab. 9. Unlike [ 62], we address an issue happening when the 1D convolutional kernel slides from the beginning to the end of the motion sequence: ker- nels at the start of the sequenc...

    2048

  78. [78]

    5 reports an ablation over dif- ferent numbers of text adapter layers

    More Quantitative Results Effect of the text adapter.Tab. 5 reports an ablation over dif- ferent numbers of text adapter layers. Using the adapter im- Table 7.Quantitative comparison with MotionStreamer.MotionStreamer proposed a TMR-style [ 42] feature extractor used as an evaluator in their own 272D representation. To ensure a fair comparison, we train o...