Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

One unified autoregressive diffusion transformer can generate whole-body motion from text, music, speech, and spatial-temporal control signals, using reference motion as a conditioning signal, and is trained on a 286-hour unified motion-cap

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 08:33 UTC pith:RUVCLNH4

load-bearing objection Big curated dataset and a plausible unified model, but the SOTA claim rests on a closed in-house evaluation with unmatched baselines — send to review but demand external benchmarks. the 5 major comments →

arxiv 2510.19789 v2 pith:RUVCLNH4 submitted 2025-10-22 cs.CV

OmniMotion-X: Versatile Multimodal Whole-Body Motion Generation

Guowei Xu , Yuxuan Bian , Ailing Zeng , Zhuo Chen , Mingyi Shi , Shaoli Huang , Wen Li , Lixin Duan
show 1 more author
Qiang Xu
This is my paper
classification cs.CV
keywords whole-body motion generationmultimodal generationautoregressive diffusion transformerreference motion conditioningmotion capture datasettext-to-motionmusic-to-dancespeech-to-gesture
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that a single model can generate whole-body human motion from text, music, speech, and spatial-temporal controls, and can combine these conditions in one interactive session. The model is an autoregressive diffusion transformer in which every condition is encoded into tokens and prepended to the noisy motion sequence; motion from an earlier clip can be fed back as a reference-motion condition to stabilize style, content, and timing across long generations. To train it, the authors assembled what they describe as the largest unified motion-capture dataset (28 public sources, 286.2 hours, standardized to one whole-body format at 30 fps) and produced hierarchical captions by rendering motion clips to video and asking a vision-language model to describe them. On their own 280-clip test set, they report text-motion matching, FID, and diversity numbers that they interpret as state of the art, with the reference-motion variant strongest. The broader claim is that reference-motion conditioning plus a progressive weak-to-strong training schedule resolves the conflict between semantic conditions and hard spatial-temporal constraints.

Core claim

The central proposal is that reference motion—user-supplied or from previously generated clips—acts as a conditioning signal that lets a single autoregressive diffusion transformer maintain consistency of content, style, and temporal dynamics while accepting text, speech, music, and global spatial-temporal control tokens as prefix context. Combined with a progressive training schedule that first aligns motion to text and then adds finer-grained conditions (reference motion, global motion, audio), the model is claimed to avoid the modality conflicts that earlier unified models suffer. The authors also introduce a dataset unification pipeline: 28 motion-capture datasets are retargeted to one w

What carries the argument

The load-bearing mechanism is an autoregressive diffusion transformer (DiT) in which all conditions—text, global motion, speech, music, and reference motion—are encoded by modality-specific encoders, linearly projected, and concatenated as prefix tokens with the noisy motion tokens. A set of spatial-temporal masks converts the same backbone into motion prediction, in-betweening, completion, and joint/trajectory-guided synthesis. The reference-motion condition, together with a progressive weak-to-strong training curriculum, is what the paper credits for consistent, controllable generation.

Load-bearing premise

The central superiority claim rests on the assumption that the self-constructed evaluation is a fair test: a 280-clip benchmark sampled from the authors' own dataset and scored with feature extractors retrained on that same dataset, so if that protocol favors the trained model, the claimed advantage over prior methods collapses.

What would settle it

Run the released OmniMotion-X model on standard public text-to-motion and music-to-dance benchmarks, scoring with the original published feature extractors rather than extractors retrained on OmniMoCap-X, and compare FID and R-precision to published baselines. If the gap reverses or vanishes, the central state-of-the-art claim is refuted.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If correct, a single model can replace task-specific pipelines for text-to-motion, music-to-dance, speech-to-gesture, and trajectory/joint control.
  • Interactive generation becomes possible: a user can feed back a generated clip as reference motion to extend or edit a scene while keeping style and timing consistent.
  • The unified 28-source, 286-hour dataset in one whole-body format could serve as a shared training and test bed for future multimodal motion models.
  • The weak-to-strong conditioning schedule offers a recipe for combining semantic and hard constraints in other multimodal generative models.
  • The render-and-caption pipeline could scale hierarchical text annotation to additional motion datasets beyond the ones collected here.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The reference-motion mechanism points toward a more general method for long-horizon and iteratively editable motion generation, applicable beyond animation to robotics and embodied control.
  • The reported superiority should be read as provisional: the evaluation uses a test set sampled from the authors' own training distribution and metric extractors retrained on that same distribution, so cross-method comparability with established benchmarks is not established in this paper.
  • The weak-to-strong training strategy may transfer to other multimodal generation domains where low-level signals (e.g., camera trajectories in text-to-video) tend to overwhelm high-level semantics.
  • A natural testable extension is to run the released model on standard public text-to-motion and dance benchmarks with the original published metric extractors; that would show whether the gains generalize or are specific to the paper's evaluation protocol.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces OmniMotion-X, a unified autoregressive diffusion transformer for whole-body human motion generation from multiple modalities (text, music, speech, global spatial-temporal controls, and reference motion), together with a progressive weak-to-strong mixed-condition training strategy. It also contributes OmniMoCap-X, a dataset of 28 public sources standardized to SMPL-X at 30 fps, totaling 286.2 hours, with GPT-4o-generated hierarchical captions. The paper claims state-of-the-art performance on text-to-motion, music-to-dance, speech-to-gesture, and global spatial-temporal controllable generation. The T2M and GSTC evaluations are on a 280-clip test set sampled from the authors' own OmniMoCap-X, while M2D/S2G are evaluated on BEAT2 and AIST++/FineDance/PhantomDance. Results are mixed: OmniMotion-X is better on several T2M/GSTC metrics and on diversity, but worse than MotionCraft on whole-body FID for M2D and on hand FID for S2G.

Significance. If the claimed results are supported, this would be a substantial step toward a single model that handles multiple whole-body motion generation tasks, and the OmniMoCap-X dataset could be a useful community resource. The idea of using reference motion as a conditioning signal and the progressive training schedule are plausible and interesting. The paper also offers code and dataset links, which would aid reproducibility. However, the current evaluation does not convincingly establish the central 'significantly surpasses existing methods' claim, because the comparisons are confounded by architectural differences, text-encoder differences, and an in-house evaluation setup. The paper's contribution is promising, but the evidence needs to be strengthened before the SOTA claim can be accepted.

major comments (5)
  1. [Experiments, Table 4] The T2M comparison to MDM, MLD, and MoMask is not well-controlled. These methods are body-only and do not natively predict SMPL-X hand/face parameters, yet they are evaluated on a whole-body SMPL-X test set. The paper never describes how they are adapted to produce whole-body output (e.g., zero hand/face values, retargeting, or additional regressors). If the baselines output zeros or fixed hands/face, the re-trained whole-body feature extractor will penalize them for reasons orthogonal to generation quality, making the large FID gap (e.g., 72.928 vs. 3.199) uninterpretable. Please specify the adaptation and, if possible, include whole-body baselines trained on the same representation.
  2. [Experiments / Supplementary 'More Implementation Details'] The main evaluation is a closed loop. The model is trained on OmniMoCap-X, the test set is uniformly sampled from OmniMoCap-X, and the FID/R-precision feature extractors are re-trained on OmniMoCap-X. This makes the scores incommensurable with published numbers and systematically favors the model whose training distribution matches the evaluation distribution. No standard external benchmarks (e.g., HumanML3D for T2M, AIST++ protocol for M2D, BEAT2 protocol for S2G) are reported. Consequently, the abstract's 'state-of-the-art' and 'significantly surpasses' claims rest entirely on an in-house benchmark. Please report at least one standard external benchmark, or provide baselines retrained on the same data with identical feature extractors and text encoders.
  3. [Experiments, Table 6] Table 6 directly contradicts the broad 'significantly surpasses existing methods' claim. OmniMotion-X is worse than MotionCraft on whole-body FID for M2D (16.209 vs. 9.875) and worse on hand FID for S2G (9.095 vs. 5.370). The sentence 'The lower FID is mainly due to the relatively small test sets...' is not a quantitative explanation and does not address why the deficit appears on these specific metrics while the model is better on others. The paper should either acknowledge the trade-off and provide statistical significance or restrict the SOTA claim to the metrics/tasks where the improvement is consistent.
  4. [T2M experiments, Table 4] The paper acknowledges that OmniMotion-X uses T5-XXL while the MoMask* and MotionCraft* baselines use CLIP. Since the text encoder is a major component of text-conditioned generation, this difference alone could explain a substantial part of the R-precision and FID improvements. To isolate the contribution of the proposed architecture and training strategy, an ablation with a shared text encoder (e.g., CLIP for all methods, or T5-XXL for the retrained baselines) is needed. Without this, the superiority of OmniMotion-X over the '*' baselines is confounded.
  5. [OmniMoCap-X Dataset, Tables 2 and 3] The abstract and text describe OmniMoCap-X as integrating '28 publicly available MoCap sources,' but Table 2 indicates only 21/28 sources are mocap, and Table 3 lists five Single-View RGB sources (IDEA400, AIOZ, etc.). These are not marker-based motion capture and likely involve pose-estimation noise. The 'largest unified multimodal MoCap dataset' claim should be qualified accordingly, and the paper should report the quality-control procedure for the non-marker sources, since the data-quality argument is central to the dataset's contribution.
minor comments (5)
  1. [Unified Motion Representation] The pose tuple p_i is defined as (˙r_a, ˙r_x, ˙r_z, r_y, j_p, j_v, c_f, f), but the following sentence discusses j_r ∈ R^{6N'} (joint 6D rotations), which is not included in the tuple. Either add j_r to the tuple or remove the discussion.
  2. [Table 6 caption] The caption contains a typo: 'F IDW holeBody' should be 'FID (Whole-Body)'.
  3. [Supplementary 'Dataset Text Quality'] The average sentence length of 276.78 words across the collection seems implausibly large for a motion description and may be a typo (e.g., characters instead of words). Please verify and clarify.
  4. [Implementation Details] The paper states 'the default length of motion reference and prediction is 150' but does not explain how variable-length sequences are handled at inference or whether the autoregressive scheme supports arbitrary lengths. Clarify.
  5. [Conclusion and Discussion] The limitation states the method 'lacks scene, object, and human interaction constraints,' yet the dataset includes HOI, HSI, and HHI data. Clarify that these data are included for future tasks but are not currently modeled, to avoid an apparent contradiction.

Circularity Check

0 steps flagged

No circular derivation found; the SOTA claim is supported by experiments rather than by a self-referential reduction, though the evaluation is substantially in-house.

full rationale

The paper's central claim is empirical: OmniMotion-X is trained on the self-constructed OmniMoCap-X and evaluated on a test set sampled from that same dataset, with text/motion feature extractors re-trained on OmniMoCap-X (Supplementary, 'More Implementation Details'). This is an evaluation-validity concern — the benchmark is in-house and the metrics are in-distribution for the proposed model — but it is not a circularity in the derivation-chain sense: no equation equates a predicted quantity to a fitted input, and the baseline comparisons (e.g., MoMask* and MotionCraft* retrained on the same OmniMoCap-X) are not forced to produce the reported outcome by construction. The paper also reports external comparisons on AIST++/FineDance/PhantomDance and BEAT2 in Table 6, where results are mixed and the text itself concedes 'some distribution differences' and that 'superior diversity, which may also impact the FID' — a caveat rather than a guaranteed-win construction. Several self-citations exist (MotionCraft, Motion-X, HumanTomato), but they are used for architecture components and the evaluation protocol, not as load-bearing uniqueness theorems or as definitions of the target result. Overall, no specific circular step can be exhibited; the score reflects only the minor presence of self-cited evaluation infrastructure that makes the headline SOTA claim harder to independently verify.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

No novel physical or theoretical entities are introduced. The free parameters are the hand-chosen training schedule, the contrastive margin, and the clip length; the axioms are the key domain assumptions about representation fidelity, caption quality, metric validity, and benchmark fairness. These are the unexamined premises on which the SOTA claim rests.

free parameters (3)
  • Training stage durations = 460K / 460K / 230K / 920K steps
    Hand-chosen schedule in the weak-to-strong progressive training; no sensitivity analysis or ablation for each duration is provided.
  • Contrastive margin d = 10
    Margin in the re-trained text-motion feature extractor loss (Eq. 4 in supplementary); chosen without reported justification.
  • Reference/prediction clip length = 150 frames
    Default motion length used throughout training and evaluation, corresponding to 5-second clips; chosen for the dataset segmentation.
axioms (4)
  • domain assumption SMPL-X is a sufficient representation of whole-body motion across all tasks.
    All source formats (BVH, FBX, SMPL, keypoints) are converted to SMPL-X; any information lost in conversion is not accounted for in the claims.
  • domain assumption GPT-4o-generated hierarchical captions are accurate enough for text-motion alignment.
    Captions are used as ground truth for text-to-motion training and evaluation; no human validation or quality control is reported beyond sample statistics in the appendix.
  • domain assumption Re-trained contrastive feature extractors yield metrics comparable across methods.
    The paper states pre-trained extractors are not directly applicable and thus trains new ones on OmniMoCap-X; the resulting FID/R-precision values are assumed to support cross-method ranking.
  • ad hoc to paper The 280-sample self-built test set is a fair benchmark for SOTA comparisons.
    The test set is uniformly sampled from OmniMoCap-X, the same distribution the model is trained on and the extractors are trained on, so it is not an external validation.

pith-pipeline@v1.3.0-alltime-deepseek · 20262 in / 8639 out tokens · 77461 ms · 2026-08-04T08:33:26.210516+00:00 · methodology

0 comments
read the original abstract

This paper introduces OmniMotion-X, a versatile multimodal framework for whole-body human motion generation, leveraging an autoregressive diffusion transformer in a unified sequence-to-sequence manner. OmniMotion-X efficiently supports diverse multimodal tasks, including text-to-motion, music-to-dance, speech-to-gesture, and global spatial-temporal control scenarios (e.g., motion prediction, in-betweening, completion, and joint/trajectory-guided synthesis), as well as flexible combinations of these tasks. Specifically, we propose the use of reference motion as a novel conditioning signal, substantially enhancing the consistency of generated content, style, and temporal dynamics crucial for realistic animations. To handle multimodal conflicts, we introduce a progressive weak-to-strong mixed-condition training strategy. To enable high-quality multimodal training, we construct OmniMoCap-X, the largest unified multimodal motion dataset to date, integrating 28 publicly available MoCap sources across 10 distinct tasks, standardized to the SMPL-X format at 30 fps. To ensure detailed and consistent annotations, we render sequences into videos and use GPT-4o to automatically generate structured and hierarchical captions, capturing both low-level actions and high-level semantics. Extensive experimental evaluations confirm that OmniMotion-X significantly surpasses existing methods, demonstrating state-of-the-art performance across multiple multimodal tasks and enabling the interactive generation of realistic, coherent, and controllable long-duration motions.

Figures

Figures reproduced from arXiv: 2510.19789 by Ailing Zeng, Guowei Xu, Lixin Duan, Mingyi Shi, Qiang Xu, Shaoli Huang, Wen Li, Yuxuan Bian, Zhuo Chen.

Figure 1
Figure 1. Figure 1: We present OmniMotion-X, a unified sequence-to-sequence autoregressive motion diffusion transformer designed for flexible and interactive whole-body human motion generation. It supports a variety of tasks, including text-to-motion, music-to￾dance, speech-to-gesture, and globally spatial-temporal controllable motion generation, which encompasses motion prediction, in-betweening, completion, and joint/trajec… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of OmniMotion-X, a unified multimodal autoregressive transformer diffusion model for whole-body human motion generation. OmniMotion-X integrates text, global motion, speech, music, and reference motion as conditions through condition-specific encoders mapped into a unified space. The model fuses multimodal information to produce coherent motion, with spatial-temporal guidance ensuring consistent g… view at source ↗
Figure 3
Figure 3. Figure 3: We propose the weak-to-strong progressive train [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Diverse motion synthesis capabilities of OmniMotion-X. OmniMotion-X supports multiple tasks: (a) text-to-motion, (b) speech-to-gesture, (c) music-to-dance, (d) trajectory-guided motion, (e) motion in-betweening, and (f) motion prediction. Method Speech-to-Gesture Music-to-Dance FID (Whole-Body)↓ FID (Hands)↓ F aceMSE ↓ Diversity↑ FID (Whole Body)↓ FID (Hands)↓ Diversity↑ MotionCraft 3.422 5.370 0.182 1.003… view at source ↗
Figure 5
Figure 5. Figure 5: Example of GPT-4o Captioning Quality. mask zeroes out attention weights for missing inputs, al￾lowing the model to focus only on valid ones. (2) Global task-dependent mask, a spatiotemporal dynamic mask tai￾lored to tasks like motion prediction, interpolation, com￾pletion, and trajectory-guided generation—distinguishes be￾tween observed and unobserved motion regions based on task requirements. (3) Global m… view at source ↗
Figure 6
Figure 6. Figure 6: From left to right, HHI, HOI, and HSI of our dataset. [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. ViBES: A Conversational Agent with Behaviorally-Intelligent 3D Virtual Body

    cs.CV 2025-12 unverdicted novelty 7.0

    ViBES introduces a speech-language-behavior model using modality-specific transformer experts that jointly generates dialogue and 3D body actions, showing gains over separate co-speech and text-to-motion baselines on ...

Reference graph

Works this paper leans on

86 extracted references · 13 linked inside Pith · cited by 1 Pith paper

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al

    Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    Pith/arXiv arXiv 2023

  4. [4]

    ???? Mixamo

    Adobe . ???? Mixamo . https://www.mixamo.com

  5. [5]

    Alexanderson, S.; Nagy, R.; Beskow, J.; and Henter, G. E. 2023. Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models. ACM Trans. Graph., 42(4): 44:1--44:20

    2023

  6. [6]

    Ao, T. 2024. Body of Her: A Preliminary Study on End-to-End Humanoid Agent. arXiv preprint arXiv:2408.02879

    Pith/arXiv arXiv 2024

  7. [7]

    P.; Li, J.; Vetrivel, K.; Agarwal, R.; Wu, J.; Gopinath, D.; Clegg, A

    Ara \'u jo, J. P.; Li, J.; Vetrivel, K.; Agarwal, R.; Wu, J.; Gopinath, D.; Clegg, A. W.; and Liu, K. 2023. Circle: Capture in rich contextual environments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 21211--21221

    2023

  8. [8]

    ???? Autodesk MotionBuilder

    Autodesk Inc. ???? Autodesk MotionBuilder

  9. [9]

    L.; Xie, X.; Petrov, I

    Bhatnagar, B. L.; Xie, X.; Petrov, I. A.; Sminchisescu, C.; Theobalt, C.; and Pons-Moll, G. 2022. Behave: Dataset and method for tracking human object interactions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15935--15946

    2022

  10. [10]

    Bian, Y.; Zeng, A.; Ju, X.; Liu, X.; Zhang, Z.; Liu, W.; and Xu, Q. 2024. MotionCraft: Crafting Whole-Body Motion with Plug-and-Play Multimodal Controls. arXiv preprint arXiv:2407.21136

    Pith/arXiv arXiv 2024

  11. [11]

    ???? Blender

    Blender Foundation . ???? Blender

  12. [12]

    Chen, J.; Liu, Y.; Wang, J.; Zeng, A.; Li, Y.; and Chen, Q. 2024 a . Diffsheg: A diffusion-based approach for real-time speech-driven holistic 3d expression and gesture generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7352--7361

    2024

  13. [13]

    Chen, K.; Tan, Z.; Lei, J.; Zhang, S.-H.; Guo, Y.-C.; Zhang, W.; and Hu, S.-M. 2021. Choreomaster: choreography-oriented music-driven dance synthesis. ACM Transactions on Graphics (TOG), 40(4): 1--13

    2021

  14. [14]

    Chen, L.-H.; Zhang, J.; Li, Y.; Pang, Y.; Xia, X.; and Liu, T. 2023 a . Humanmac: Masked motion completion for human motion prediction. In ICCV, 9544--9555

    2023

  15. [15]

    Chen, R.; Shi, M.; Huang, S.; Tan, P.; Komura, T.; and Chen, X. 2024 b . Taming diffusion probabilistic models for character control. In ACM SIGGRAPH 2024 Conference Papers, 1--10

    2024

  16. [16]

    Chen, X.; Jiang, B.; Liu, W.; Huang, Z.; Fu, B.; Chen, T.; and Yu, G. 2023 b . Executing your commands via motion diffusion in latent space. In CVPR, 18000--18010

    2023

  17. [17]

    B.; and van de Panne, M

    Cohan, S.; Tevet, G.; Reda, D.; Peng, X. B.; and van de Panne, M. 2024. Flexible motion in-betweening with diffusion models. In ACM SIGGRAPH, 1--9

    2024

  18. [18]

    Dai, W.; Chen, L.-H.; Wang, J.; Liu, J.; Dai, B.; and Tang, Y. 2024. MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Model. ECCV

    2024

  19. [19]

    Delmas, G.; Weinzaepfel, P.; Lucas, T.; Moreno-Noguer, F.; and Rogez, G. 2022. Posescript: 3d human poses from natural language. In European Conference on Computer Vision, 346--362. Springer

    2022

  20. [20]

    J.; and Hilliges, O

    Fan, Z.; Taheri, O.; Tzionas, D.; Kocabas, M.; Kaufmann, M.; Black, M. J.; and Hilliges, O. 2023. ARCTIC: A dataset for dexterous bimanual hand-object manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 12943--12954

    2023

  21. [21]

    Feng, A.; Shin, S.; and Yoon, Y. 2022. A tool for extracting 3D avatar-ready gesture animations from monocular videos. In Proceedings of the 15th ACM SIGGRAPH Conference on Motion, Interaction and Games, 1--7

    2022

  22. [22]

    Fieraru, M.; Zanfir, M.; Oneata, E.; Popa, A.-I.; Olaru, V.; and Sminchisescu, C. 2021 a . Learning complex 3d human self-contact. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 1343--1351

    2021

  23. [23]

    Fieraru, M.; Zanfir, M.; Pirlea, S.-C.; Olaru, V.; and Sminchisescu, C. 2021 b . AIFit: Automatic 3D Human-Interpretable Feedback Models for Fitness Training. In The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2021

  24. [24]

    G.; Wang, S.; and Cheng, L

    Guo, C.; Mu, Y.; Javed, M. G.; Wang, S.; and Cheng, L. 2024 a . Momask: Generative masked modeling of 3d human motions. In CVPR, 1900--1910

    2024

  25. [25]

    Guo, C.; Zou, S.; Zuo, X.; Wang, S.; Ji, W.; Li, X.; and Cheng, L. 2022. Generating diverse and natural 3d human motions from text. In CVPR, 5152--5161

    2022

  26. [26]

    Guo, X.; Zhang, M.; Xie, H.; Gu, C.; and Liu, Z. 2024 b . Crowdmogen: Zero-shot text-driven collective motion generation. arXiv preprint arXiv:2407.06188

    Pith/arXiv arXiv 2024

  27. [27]

    Han, B.; Peng, H.; Dong, M.; Ren, Y.; Shen, Y.; and Xu, C. 2024. AMD: Autoregressive Motion Diffusion. In AAAI, 2022--2030

    2024

  28. [28]

    G.; Yurick, M.; Nowrouzezahrai, D.; and Pal, C

    Harvey, F. G.; Yurick, M.; Nowrouzezahrai, D.; and Pal, C. 2020. Robust Motion In-Betweening. 39(4)

    2020

  29. [29]

    P.; Yi, H.; H \"o schle, M.; Safroshkin, M.; Alexiadis, T.; Polikovsky, S.; Scharstein, D.; and Black, M

    Huang, C.-H. P.; Yi, H.; H \"o schle, M.; Safroshkin, M.; Alexiadis, T.; Polikovsky, S.; Scharstein, D.; and Black, M. J. 2022. Capturing and Inferring Dense Full-Body Human-Scene Contact. In Proceedings IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 13274--13285

    2022

  30. [30]

    Ionescu, C.; Papava, D.; Olaru, V.; and Sminchisescu, C. 2013. Human3. 6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE transactions on pattern analysis and machine intelligence, 36(7): 1325--1339

    2013

  31. [31]

    T.; and Zheng, W.-S

    Ji, Y.; Xu, F.; Yang, Y.; Shen, F.; Shen, H. T.; and Zheng, W.-S. 2018. A large-scale RGB-D database for arbitrary-view human action recognition. In ACM MM, 1510--1518

    2018

  32. [32]

    Jiang, B.; Chen, X.; Liu, W.; Yu, J.; Yu, G.; and Chen, T. 2024 a . Motiongpt: Human motion as a foreign language. NeurIPS

    2024

  33. [33]

    Jiang, N.; Liu, T.; Cao, Z.; Cui, J.; Chen, Y.; Wang, H.; Zhu, Y.; and Huang, S. 2022. Full-Body Articulated Human-Object Interaction. ICCV, 3

    2022

  34. [34]

    Jiang, N.; Zhang, Z.; Li, H.; Ma, X.; Wang, Z.; Chen, Y.; Liu, T.; Zhu, Y.; and Huang, S. 2024 b . Scaling up dynamic human-scene interaction modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1737--1747

    2024

  35. [35]

    J.; and Hilliges, O

    Kaufmann, M.; Song, J.; Guo, C.; Shen, K.; Jiang, T.; Tang, C.; Z \'a rate, J. J.; and Hilliges, O. 2023. EMDB : The E lectromagnetic D atabase of G lobal 3 D H uman P ose and S hape in the W ild. In International Conference on Computer Vision (ICCV)

    2023

  36. [36]

    Lauterbach, C.; Garland, M.; Sengupta, S.; Luebke, D.; and Manocha, D. 2009. Fast BVH construction on GPUs. In Computer Graphics Forum, volume 28, 375--384. Wiley Online Library

    2009

  37. [37]

    D.; and Nguyen, A

    Le, N.; Pham, T.; Do, T.; Tjiputra, E.; Tran, Q. D.; and Nguyen, A. 2023. Music-driven group choreography. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8673--8682

    2023

  38. [38]

    Li, B.; Zhao, Y.; Zhelun, S.; and Sheng, L. 2022. Danceformer: Music conditioned 3d dance generation with parametric motion transformer. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, 1272--1279

    2022

  39. [39]

    Li, J.; Wu, J.; and Liu, C. K. 2023. Object motion guided human motion synthesis. ACM TOG, 42(6): 1--11

    2023

  40. [40]

    A.; and Kanazawa, A

    Li, R.; Yang, S.; Ross, D. A.; and Kanazawa, A. 2021. Learn to Dance with AIST++: Music Conditioned 3D Dance Generation. arXiv:2101.08779

    Pith/arXiv arXiv 2021

  41. [41]

    Li, R.; Zhao, J.; Zhang, Y.; Su, M.; Ren, Z.; Zhang, H.; Tang, Y.; and Li, X. 2023. Finedance: A fine-grained choreography dataset for 3d full body dance generation. In ICCV, 10234--10243

    2023

  42. [42]

    J.; Li, H.; and Romero, J

    Li, T.; Bolkart, T.; Black, M. J.; Li, H.; and Romero, J. 2017. Learning a model of facial shape and expression from 4D scans. ACM Trans. Graph., 36(6): 194--1

    2017

  43. [43]

    Liang, H.; Bao, J.; Zhang, R.; Ren, S.; Xu, Y.; Yang, S.; Chen, X.; Yu, J.; and Xu, L. 2024 a . Omg: Towards open-vocabulary motion generation via mixture of controllers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 482--493

    2024

  44. [44]

    Liang, H.; Zhang, W.; Li, W.; Yu, J.; and Xu, L. 2024 b . Intergen: Diffusion-based multi-human motion generation under complex interactions. IJCV, 1--21

    2024

  45. [45]

    Liao, Y.; Fu, Y.; Cheng, Z.; and Wang, J. 2024. AnimationGPT:An AIGC tool for generating game combat motion assets. https://github.com/fyyakaxyy/AnimationGPT

    2024

  46. [46]

    Lin, G.; Jiang, J.; Yang, J.; Zheng, Z.; and Liang, C. 2025. OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Models. arXiv preprint arXiv:2502.01061

    Pith/arXiv arXiv 2025

  47. [47]

    Lin, J.; Zeng, A.; Lu, S.; Cai, Y.; Zhang, R.; Wang, H.; and Zhang, L. 2024. Motion-x: A large-scale 3d expressive whole-body human motion dataset. NeurIPS

    2024

  48. [48]

    Ling, Z.; Han, B.; Li, S.; Shen, H.; Cheng, J.; and Zou, C. 2024. MotionLLaMA: A Unified Framework for Motion Synthesis and Comprehension. arXiv preprint arXiv:2411.17335

    Pith/arXiv arXiv 2024

  49. [49]

    Ling, Z.; Han, B.; Wong, Y.; Kangkanhalli, M.; and Geng, W. 2023. Mcm: Multi-condition motion synthesis framework for multi-scenario. arXiv preprint arXiv:2309.03031

    Pith/arXiv arXiv 2023

  50. [50]

    Liu, H.; Zhu, Z.; Becherini, G.; Peng, Y.; Su, M.; Zhou, Y.; Zhe, X.; Iwamoto, N.; Zheng, B.; and Black, M. J. 2023. Emage: Towards unified holistic co-speech gesture generation via masked audio gesture modeling. arXiv e-prints, arXiv--2401

    2023

  51. [51]

    Liu, H.; Zhu, Z.; Iwamoto, N.; Peng, Y.; Li, Z.; Zhou, Y.; Bozkurt, E.; and Zheng, B. 2022. Beat: A large-scale semantic and emotional multi-modal dataset for conversational gestures synthesis. In European conference on computer vision, 612--630. Springer

    2022

  52. [52]

    Liu, Y.; Yang, H.; Si, X.; Liu, L.; Li, Z.; Zhang, Y.; Liu, Y.; and Yi, L. 2024. Taco: Benchmarking generalizable bimanual tool-action-object understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 21740--21751

    2024

  53. [53]

    Loper, M.; Mahmood, N.; Romero, J.; Pons-Moll, G.; and Black, M. J. 2023. SMPL: A skinned multi-person linear model. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2, 851--866

    2023

  54. [54]

    Lu, S.; Chen, L.-H.; Zeng, A.; Lin, J.; Zhang, R.; Zhang, L.; and Shum, H.-Y. 2024 a . Humantomato: Text-aligned whole-body motion generation. ICML

    2024

  55. [55]

    Lu, S.; Wang, J.; Lu, Z.; Chen, L.-H.; Dai, W.; Dong, J.; Dou, Z.; Dai, B.; and Zhang, R. 2024 b . ScaMo: Exploring the Scaling Law in Autoregressive Motion Generation Model. arXiv preprint arXiv:2412.14559

    Pith/arXiv arXiv 2024

  56. [56]

    Luo, M.; Hou, R.; Li, Z.; Chang, H.; Liu, Z.; Wang, Y.; and Shan, S. 2024. M^3 GPT: An Advanced Multimodal, Multitask Framework for Motion Comprehension and Generation . arXiv preprint arXiv:2405.16273

    Pith/arXiv arXiv 2024

  57. [57]

    Mao, J.; Zhao, S.; Song, S.; Shi, T.; Ye, J.; Zhang, M.; Geng, H.; Malik, J.; Guizilini, V.; and Wang, Y. 2024. Learning from Massive Human Videos for Universal Humanoid Pose Control. arXiv preprint arXiv:2412.14172

    Pith/arXiv arXiv 2024

  58. [58]

    Mason, I.; Starke, S.; and Komura, T. 2022. Real-time style modelling of human locomotion via feature-wise transformations and local motion phases. Proceedings of the ACM on Computer Graphics and Interactive Techniques, 5(1): 1--18

    2022

  59. [59]

    P.; McVicar, M.; Battenberg, E.; and Nieto, O

    McFee, B.; Raffel, C.; Liang, D.; Ellis, D. P.; McVicar, M.; Battenberg, E.; and Nieto, O. 2015. librosa: Audio and music signal analysis in python. SciPy, 2015: 18--24

    2015

  60. [60]

    A.; Tzionas, D.; and Black, M

    Pavlakos, G.; Choutas, V.; Ghorbani, N.; Bolkart, T.; Osman, A. A.; Tzionas, D.; and Black, M. J. 2019. Expressive body capture: 3d hands, face, and body from a single image. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 10975--10985

    2019

  61. [61]

    Peebles, W.; and Xie, S. 2023. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, 4195--4205

    2023

  62. [62]

    J.; and Varol, G

    Petrovich, M.; Black, M. J.; and Varol, G. 2021. Action-conditioned 3d human motion synthesis with transformer vae. In ICCV, 10985--10995

    2021

  63. [63]

    J.; and Varol, G

    Petrovich, M.; Black, M. J.; and Varol, G. 2023. TMR: Text-to-motion retrieval using contrastive 3D human motion synthesis. In ICCV, 9488--9497

    2023

  64. [64]

    Plappert, M.; Mandery, C.; and Asfour, T. 2016. The KIT motion-language dataset. Big data, 4(4): 236--252

    2016

  65. [65]

    R.; Chandrasekaran, A.; Athanasiou, N.; Quiros-Ramirez, A.; and Black, M

    Punnakkal, A. R.; Chandrasekaran, A.; Athanasiou, N.; Quiros-Ramirez, A.; and Black, M. J. 2021. BABEL: Bodies, action and behavior with english labels. In CVPR, 722--731

    2021

  66. [66]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748--8763. PmLR

    2021

  67. [67]

    Raffel, C.; Shazeer, N.; Roberts, A.; Lee, K.; Narang, S.; Matena, M.; Zhou, Y.; Li, W.; and Liu, P. J. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140): 1--67

    2020

  68. [68]

    C.; and Liu, Z

    Siyao, L.; Yu, W.; Gu, T.; Lin, C.; Wang, Q.; Qian, C.; Loy, C. C.; and Liu, Z. 2022. Bailando: 3d dance generation by actor-critic gpt with choreographic memory. In CVPR, 11050--11059

    2022

  69. [69]

    Tevet, G.; Raab, S.; Gordon, B.; Shafir, Y.; Cohen-Or, D.; and Bermano, A. H. 2022. Human motion diffusion model. In ICLR

    2022

  70. [70]

    Tseng, J.; Castellon, R.; and Liu, K. 2023. Edge: Editable dance generation from music. In CVPR, 448--458

    2023

  71. [71]

    E.; Beskow, J.; Holzapfel, A.; Oudeyer, P.-Y.; and Alexanderson, S

    Valle-P \'e rez, G.; Henter, G. E.; Beskow, J.; Holzapfel, A.; Oudeyer, P.-Y.; and Alexanderson, S. 2021. Transflower: Probabilistic Autoregressive Dance Generation with Multimodal Attention. ACM Trans. Graph., 40(6): 195:1--195:14

    2021

  72. [72]

    N.; Kaiser, .; and Polosukhin, I

    Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. NeurIPS

    2017

  73. [73]

    Wikipedia contributors . 2024. FBX. Wikipedia, The Free Encyclopedia. Last edited on 1 October 2024, at 12:38 (UTC). Accessed [insert access date]

    2024

  74. [74]

    Xie, Y.; Jampani, V.; Zhong, L.; Sun, D.; and Jiang, H. 2024. Omnicontrol: Control any joint at any time for human motion generation. In ICLR

    2024

  75. [75]

    Xu, G.; Tao, J.; Li, W.; and Duan, L. 2024 a . Learning semantic latent directions for accurate and controllable human motion prediction. In European Conference on Computer Vision, 56--73. Springer

    2024

  76. [76]

    Xu, L.; Lv, X.; Yan, Y.; Jin, X.; Wu, S.; Xu, C.; Liu, Y.; Zhou, Y.; Rao, F.; Sheng, X.; et al. 2024 b . Inter-x: Towards versatile human-human interaction analysis. In CVPR, 22260--22271

    2024

  77. [77]

    J.; Liu, E.; Lagasse, R.; Mohammadi, H.; Cheng, L.; and Lim, A

    Yazdian, P. J.; Liu, E.; Lagasse, R.; Mohammadi, H.; Cheng, L.; and Lim, A. 2023. Motionscript: Natural language descriptions for expressive 3d human motions. arXiv preprint arXiv:2312.12634

    arXiv 2023

  78. [78]

    Yi, H.; Liang, H.; Liu, Y.; Cao, Q.; Wen, Y.; Bolkart, T.; Tao, D.; and Black, M. J. 2023. Generating holistic 3d human motion from speech. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 469--480

    2023

  79. [79]

    Zhan, X.; Yang, L.; Zhao, Y.; Mao, K.; Xu, H.; Lin, Z.; Li, K.; and Lu, C. 2024. Oakink2: A dataset of bimanual hands-object manipulation in complex task completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 445--456

    2024

  80. [80]

    Zhang, J.; Luo, H.; Yang, H.; Xu, X.; Wu, Q.; Shi, Y.; Yu, J.; Xu, L.; and Wang, J. 2023 a . NeuralDome: A Neural Modeling Pipeline on Multi-View Human-Object Interactions. In CVPR

    2023

Showing first 80 references.