Pith. sign in

REVIEW 4 major objections 5 minor 5 cited by

The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motion

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that speech, text, and expressive body motion can be unified in one multimodal language model, and that pre-training on unpaired data is what makes the unification data-efficient.

desk verdict A promising unifier for speech, text, and motion, but the data-scarcity claim is at risk because pre-training may include the test speaker's motion. read the letter →

arxiv 2412.10523 v1 pith:FQS36N7D submitted 2024-12-13 cs.CV

Changan Chen , Juze Zhang , Shrinidhi K. Lakshmikanth , Yusu Fang , Ruizhi Shao , Gordon Wetzstein , Li Fei-Fei , Ehsan Adeli This is my paper · ORCID
classification cs.CV
keywords 3Dhumanmotiongenerationmultimodallanguagemodelco-speechgesturetokenizationgenerativepre-trainingemotionpredictionfromtext-to-motioneditable
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Human communication couples what we say with how we move, but most motion-generation models accept only one input type: speech, text, or motion. This paper tries to establish that all three can be handled by one multimodal language model: it tokenizes face, hands, upper body, and lower body motion separately, merges those tokens with audio and text tokens in a single vocabulary, and trains an encoder-decoder language model to translate between any of the modalities. The payoff is on co-speech gestures: paired with a generative pre-training stage that uses only unpaired audio-text data and general body motion, the model reports state-of-the-art realism, beat correlation, and diversity, and keeps most of its quality with 1/32 of the usual paired training data. If the approach holds, the same architecture also performs editable gesture generation and predicts emotion from body motion, making one model serve several tasks that previously needed separate systems.

What carries the argument

The carrying mechanism is a unified multimodal token vocabulary built from a compositional motion tokenizer. Four vector-quantized autoencoders discretize face, hands, upper body, and lower body motion into separate codebooks, so a full-body sequence becomes a sequence of part-specific token indices; audio and text are reduced to discrete tokens by standard self-supervised and subword tokenizers. All modalities share one vocabulary, so a pretrained encoder-decoder language model treats every input as text-like tokens and generates the target modality by next-token prediction. Before any paired speech-to-motion data is used, the model is pre-trained on two self-supervised objectives: compositional motion alignment, which translates between body parts and predicts masked motion frames, and audio-to-text alignment on large unpaired speech corpora. This pre-training is what lets the model learn body-part correlations, temporal dynamics, and semantic grounding before the downstream task.

What would settle it

A decisive check is to rerun the data-scarcity experiment with the test speaker's motion explicitly withheld from the pre-training set; if the 1/32-data benefit over the no-pre-training ablation disappears, the generalization-from-unpaired-motion claim is unsupported.

Watch

Extended reading notes

Core claim

The paper's central claim is that a language model can learn a shared 'language of motion' in which speech, text, and expressive body motion are interchangeable token sequences. The authors show on the BEATv2 co-speech benchmark that their pre-trained model beats specialist per-task systems on FGD, beat correlation, and diversity, and that the advantage grows as paired data shrinks; with only 1/32 of the training data the pre-trained model still outperforms a randomly initialized version and a strong baseline. They further claim that post-training on instruction prompts yields joint audio-text conditioning for editable gesture generation and a motion-to-emotion task that existing motion-language models fail. The unifying premise is that pre-training on unpaired modalities supplies the motion priors and semantic grounding that paired downstream data alone does not.

Load-bearing premise

The claim that pre-training makes a new speaker's gestures learnable from very little paired data assumes the pre-training corpus did not already contain that speaker's motion.

Editorial extensions

If this is right

  • A single model checkpoint can serve co-speech gesture generation, text-to-motion, editable body-part control, and motion-to-emotion understanding, replacing separate per-task pipelines.
  • New speakers can be added with a small fraction of the paired speech-to-motion data if the model has first been pre-trained on abundant unpaired motion and audio-text data.
  • Speech semantics are inherited from language-model pre-training, so gesture quality no longer depends on hand-crafted prosodic features or transcribed speech as auxiliary input.
  • Instruction-based post-training makes part-specific generation possible, such as generating lower-body locomotion from text while upper-body gestures follow the audio.
  • Because emotion can be read from motion as free text, motion-understanding tasks become answerable in language rather than restricted to fixed classification labels.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors leave the scaling implication implicit: if pre-training on unpaired data is what drives the data-scarcity gains, even larger pools of motion from video or wearables should further reduce the need for motion-capture paired data.
  • Compositional tokens suggest a natural extension not evaluated here: reconstructing or completing full-body motion from a single observed body part, a test the spatial pre-training task is already primed for.
  • The motion-to-emotion evaluation uses text-similarity metrics rather than classification accuracy, so part of the reported score may reflect paraphrase skill; a held-out emotion-class accuracy probe would separate language fluency from true emotion recognition.
  • The authors note that discrete tokenization sometimes produces incoherent motion, implying an immediate experimental fork: replace the vector-quantized codebooks with a residual or continuous tokenizer and measure whether FGD and motion coherence improve under the same pre-training scheme.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a multimodal language model that unifies text, speech, and compositional 3D body motion (face, hands, upper body, lower body) in a single encoder-decoder T5-based architecture with discrete tokenization. The training pipeline has two stages: a generative pre-training stage with spatial and temporal body-part alignment tasks plus audio-text alignment on unpaired data, and a post-training instruction-following stage on paired audio-to-motion, text-to-motion, and emotion-related tasks. The authors report state-of-the-art FGD/BC/Diversity on the BEATv2 co-speech gesture benchmark, ablations of each pre-training task, a data-scarcity experiment showing improved FGD at small post-training data fractions, qualitative editable gesture generation with joint audio-text prompts, and a new emotion-from-motion prediction task.

Significance. If the results hold, the paper makes a useful contribution by showing that a single language-model backbone can handle multiple motion-related input/output modalities and that self-supervised pre-training on unpaired motion can reduce paired-data requirements for co-speech gesture generation. The compositional body-part tokenization and the explicit audio-text alignment pre-training task are reasonable design choices, and the ablation study (Table 2) gives some credit to the proposed components. The newly introduced emotion-prediction task and the qualitative editable-generation examples broaden the scope beyond standard co-speech benchmarks. The paper is clearly written and the experimental plan is generally sensible. However, the central quantitative claims rest on a few load-bearing experimental details that are currently underspecified, as detailed below.

major comments (4)
  1. [§4.1, §4.2, Fig. 5] The data-scarcity and novel-speaker generalization claims depend on whether the pre-training motion data excludes the test speaker. Section 4.1 states that pre-training uses BEATv2 and Librispeech and that no audio-to-motion data is seen, but it never states that the BEATv2 motion used for the spatial and temporal pre-training tasks excludes speaker-2, the speaker used for post-training and evaluation in the EMAGE protocol. If speaker-2 motion clips appear in the pre-training corpus, then Fig. 5's comparison at 1/32 to 1/2 of post-training data is not a clean novel-speaker evaluation: the model would already have access to the target speaker's pose statistics, body-part co-occurrences, and temporal dynamics. The authors should state explicitly whether speaker-2 motion was held out from all pre-training tasks, and if it was not, the data-efficiency and 'never seeing speech-motion data' claims must be re-framed or re-run with a clean speaker holdout.
  2. [Table 1, Fig. 5] The core SOTA claim rests on a small numerical margin (FGD 5.301 vs. 5.512 for EMAGE, and only 0.107 better than the no-multimodal-pre-training ablation), and neither Table 1 nor Fig. 5 reports variance bars or significance across multiple seeds. Given the stochasticity of VQ-VAE training, autoregressive generation, and the metrics themselves, a single-run comparison of this size is not enough to establish that the improvements are not noise. The authors should provide mean and standard deviation over at least three seeds for the main comparison and for the data-scarcity curves, or otherwise justify why variance is negligible.
  3. [§4.1, Table 1] The sentence 'we adopt the same evaluation protocol as [43], i.e., training and testing on speaker-2 and using their motion tokenizer' is ambiguous about which tokenizer was used to produce Table 1. If Table 1 uses EMAGE's tokenizer, then the pre-trained model's own compositional tokenizer (Section 3.2) is not being evaluated in the main benchmark, and the connection between pre-training and downstream performance is unclear. If Table 1 uses the newly trained four-part VQ-VAEs, then the comparison with EMAGE, which was trained with its own tokenizer, may be unfair because the reconstruction quality of the tokenizers differs. Please clarify which tokenizer was used in each reported result and, if the new tokenizer is used, report tokenizer reconstruction metrics so that the comparison is interpretable.
  4. [§4.4, Table 3] The emotion-prediction experiment does not specify how the BEATv2 emotion labels were split between instruction-tuning and evaluation. Since the same emotion labels are used to construct the training instructions, evaluating on an overlapping set would make the reported BLEU/ROUGE/CIDEr/BERTScore numbers uninformative about generalization. Please report the exact split (e.g., speaker or clip-level holdout) and clarify whether the test emotion labels were seen during post-training. In addition, using text-similarity metrics against a single ground-truth label is unusual for a classification-style task; reporting accuracy or top-k accuracy would make the result more interpretable.
minor comments (5)
  1. [Throughout] There are several typos and spacing errors, e.g., 'modelt' at the start of Section 3.2, 'W eather' in the abstract, 'V erbal' in the title line, and 'co-speech' appearing as 'a co-speech' in several places. A careful proofreading pass is needed.
  2. [Fig. 5] The caption and text say 'as the amount of paired fine-tuning data increases, the performance reduces,' but FGD is a lower-is-better metric, so the text should say the FGD value decreases/improves. This wording inversion could confuse readers.
  3. [§4.3] The editable gesture generation results are purely qualitative; there is no quantitative metric or user study for the joint audio-text conditioning. The paper should at least state that this is a demonstrative capability and discuss how it could be evaluated, or add a small quantitative study, to avoid overstating the claim.
  4. [§7.5] The supplementary states that text-to-motion quantitative metrics are not used because they are entangled with the HumanML3D representation. This is a reasonable caveat, but it should be mentioned in the main paper when discussing editable generation or the model's general motion-quality claims, rather than only in the supplementary.
  5. [§3.3] The temporal pre-training task is described as 'randomly masking off certain motion frames,' but the masking ratio, mask span length, and whether masking applies per body part or to the whole motion are not specified. Adding these details would make the pre-training section reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: BEATv2 pretraining overlap is a data-hygiene caveat, not a reduction-to-input.

full rationale

This is an empirical systems paper with no equation-level derivation chain to unwind. The headline BEATv2 co-speech results (Table 1) are obtained under the EMAGE evaluation protocol (training and testing on speaker-2) with external FGD, BC, and Diversity metrics; ablations in Table 2 and Fig. 5 isolate the pretraining objectives. The core components—HuBERT, SentencePiece/T5, and the EMAGE-inspired compositional VQ-VAE—are external and independently specified. The emotion-prediction task in Sec. 4.4 is explicitly instruction-tuned on extracted BEATv2 emotion labels, so its Table 3 numbers report a trained capability, not an emergent zero-shot result; the paper does not claim otherwise. The editable-gesture demo composes separately post-trained audio-to-part and text-to-part skills. The only notable concern is data hygiene: Sec. 4.1 says pretraining draws from BEATv2 and Librispeech, while post-training follows the EMAGE protocol of training and testing on BEATv2 speaker-2, and the paper never states that speaker-2's motion is excluded from pretraining. If it is not excluded, the 'novel speaker' and data-scarcity advantages in Fig. 5 are partly explained by the model having seen the target speaker's unpaired pose statistics. That is a correctness/leakage risk, not a circular derivation: pretraining still contains no audio-motion pairs, and the downstream speech-to-motion mapping must be learned during post-training. The paper also acknowledges real limitations (discrete tokenization producing incoherent motion, Sec. 5; H3D metric incompatibility, Sec. 7.5), which supports the conclusion that the core empirical claims are not manufactured by definition. No load-bearing self-citation or imported uniqueness claim appears. Verdict: no significant circularity.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The framework is empirical, so there is no closed-form derivation with a small set of free parameters. The entries above capture the hand-chosen hyperparameters and untested assumptions that the central claims depend on, especially the unspecified motion VQ-VAE configuration and the overlap between pre-training and evaluation data.

free parameters (3)
  • VQ-VAE codebook sizes, code dimensions, and reconstruction loss weights = Not reported
    The four body-part VQ-VAEs define the motion token space used by the LM, and Eq. (2) lists seven loss terms without weights. These choices determine the ceiling on generation quality but are not given.
  • Pre-training task sampling mix = Not reported
    The spatial, temporal, and audio-text alignment tasks are combined in one stage; the relative sampling of each task is not quantified even though removing any one task changes FGD by up to 1.5 in Table 2.
  • Post-training instruction template count and fine-tuning schedule = 350 epochs for ablations
    Supp. Sec. 7.2 says each core task has 'dozens' of prompts, producing more than a thousand instructions; the exact set and the 350-epoch schedule are hand-selected design choices that affect the reported results.
assumptions (5)
  • domain assumption SMPL-X with FLAME face parameters is a sufficient expressive motion representation for co-speech gestures and emotion reading.
    Sec. 3.1 defines all motion in SMPL-X and FLAME space; any expressive content not captured by these parameters cannot be generated or understood by the model.
  • domain assumption HuBERT discrete tokens preserve the rhythm, prosody, and semantics needed for gesture generation.
    Sec. 3.2 replaces audio with HuBERT codes at 50 Hz; if quantization discards timing or speaker detail, co-speech synchronization suffers.
  • domain assumption Independent VQ-VAEs per body part plus cross-part translation tasks capture the spatial dependencies between face, hands, upper body, and lower body.
    Secs. 3.2 and 3.3 model each body part with separate codebooks; joint coordination must be learned from token co-occurrence rather than from a joint continuous representation.
  • domain assumption The unpaired pre-training corpora, BEATv2 motion and Librispeech audio-text, are sufficient to learn motion grammar and audio-text alignment that transfers to co-speech gesture generation.
    Sec. 4.1 uses about 60 hours of motion and 1,000 hours of audio-text; the transfer claim depends on these being large and varied enough.
  • domain assumption No test-speaker motion appears in the pre-training set.
    Sec. 4.2 interprets Fig. 5 as novel-speaker data efficiency, but the paper does not state that speaker-2 motion is excluded from the BEATv2 motion used for spatial and temporal pre-training.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motion." pith.science (2026). https://pith.science/paper/FQS36N7D

@misc{pith2026241210523,
  author       = {Pith},
  title        = {Pith review of: The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FQS36N7D}},
  note         = {Machine review of arXiv:2412.10523}
}
read the original abstract

Human communication is inherently multimodal, involving a combination of verbal and non-verbal cues such as speech, facial expressions, and body gestures. Modeling these behaviors is essential for understanding human interaction and for creating virtual characters that can communicate naturally in applications like games, films, and virtual reality. However, existing motion generation models are typically limited to specific input modalities -- either speech, text, or motion data -- and cannot fully leverage the diversity of available data. In this paper, we propose a novel framework that unifies verbal and non-verbal language using multimodal language models for human motion understanding and generation. This model is flexible in taking text, speech, and motion or any combination of them as input. Coupled with our novel pre-training strategy, our model not only achieves state-of-the-art performance on co-speech gesture generation but also requires much less data for training. Our model also unlocks an array of novel tasks such as editable gesture generation and emotion prediction from motion. We believe unifying the verbal and non-verbal language of human motion is essential for real-world applications, and language models offer a powerful approach to achieving this goal. Project page: languageofmotion.github.io.

Figures

Figures reproduced from arXiv: 2412.10523 by the authors.

Figure 1
Figure 1. We introduce a language-model-based motion understanding and generation framework that takes in any of the audio/motion/text [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Method overview. We employ modality-specific tokenizers to process various input modalities. Specifically, we train a composi [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Illustration of pre-training. We pre-train our language [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Qualitative example on co-speech gesture generation. Given a speech, we visualize the ground truth 3D motion accompanying [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Generation performance vs. the amount of post-training [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Editable gesture generation. We prompt the language with text and audio information and it outputs motions that are both [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Qualitative example of emotion prediction. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Qualitative examples for text-to-motion generation. Given a text caption, we compare the 3D motion generated by our method [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Additional qualitative example on co-speech gesture generation. Given an input speech, we visualize the ground truth 3D motion [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MoRAE: Flow-Friendly Self-Supervised Latents for Text-to-Motion Generation

    cs.CV 2026-07 conditional novelty 7.0 of 10

    Distilling frozen Motion-JEPA features into a compact 32-D latent whose geometry is coupled to the decoder lets a standard non-autoregressive flow-matching DiT reach state-of-the-art text-to-motion quality on HumanML3...

  2. Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregression

    cs.CV 2024-11 conditional novelty 7.0 of 10

    A masked-autoregressive diffusion model trained on a compact essential-feature latent space claims state-of-the-art text-to-motion generation under a new essential-dimension evaluation protocol.

  3. Being-M0.5: A Real-Time Controllable Vision-Language-Motion Model

    cs.CV 2025-08 conditional novelty 6.0 of 10

    Being-M0.5 combines part-aware residual quantization with a 5M-sequence web-video dataset to reach real-time, part-controllable 3D motion generation, though its state-of-the-art claim does not hold on every standard b...

  4. VersatileMotion: A Unified Framework for Motion Synthesis and Comprehension

    cs.CV 2024-11 conditional novelty 6.0 of 10

    A unified multimodal motion LLM that handles nine generation and comprehension tasks across text, audio, and single/multi-agent motion, backed by a new dataset and tokenizer.

  5. Human-Centric Foundation Models: Perception, Generation and Agentic Modeling

    cs.CV 2025-02 conditional novelty 4.0 of 10

    A survey proposing a four-part taxonomy for human-centric foundation models and reviewing representative methods in each.

Reference graph

Works this paper leans on

96 extracted references · 55 canonical work pages · cited by 5 Pith papers

  1. [43]

    Haiyang Liu, Zihao Zhu, Giorgio Becherini, Yichen Peng, Mingyang Su, You Zhou, Xuefei Zhe, Naoya Iwamoto, Bo Zheng, and Michael J. Black. Emage: Towards unified holis- tic co-speech gesture generation via expressive masked audio gesture modeling. In CVPR, 2024. 2, 3, 5, 6, 7, 13, 14, 17

    work page 2024

  2. [1]

    Style transfer for co-speech gesture animation: A multi-speaker conditional-mixture approach

    Chaitanya Ahuja, Dong Won Lee, Yukiko I Nakano, and Louis-Philippe Morency. Style transfer for co-speech gesture animation: A multi-speaker conditional-mixture approach. In ECCV, pages 248–265. Springer, 2020. 2

    2020

  3. [2]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Men- sch, Katherine Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob Menick, Se- bastian Borgeaud, Andrew Brock, Aida Nematzadeh, Sa- hand Sharifzadeh, Mikolaj ...

    2022

  4. [3]

    Gesturediffuclip: Gesture diffusion model with clip latents.ACM Transactions on Graphics (TOG), 42(4):1–18, 2023

    Tenglong Ao, Zeyi Zhang, and Libin Liu. Gesturediffuclip: Gesture diffusion model with clip latents.ACM Transactions on Graphics (TOG), 42(4):1–18, 2023. 2

    2023

  5. [4]

    Motionfix: Text-driven 3d human motion editing

    Nikos Athanasiou, Alp ´ar Ceske, Markos Diomataris, Michael J Black, and G ¨ul Varol. Motionfix: Text-driven 3d human motion editing. arXiv preprint arXiv:2408.00712,

    arXiv

  6. [5]

    Behave: Dataset and method for tracking human object in- teractions

    Bharat Lal Bhatnagar, Xianghui Xie, Ilya A Petrov, Cristian Sminchisescu, Christian Theobalt, and Gerard Pons-Moll. Behave: Dataset and method for tracking human object in- teractions. In CVPR, pages 15935–15946, 2022. 2

    2022

  7. [6]

    Audiolm: a language modeling approach to audio generation, 2023

    Zal ´an Borsos, Rapha ¨el Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matt Sharifi, Dominik Roblek, Olivier Teboul, David Grangier, Marco Tagliasacchi, and Neil Zeghidour. Audiolm: a language modeling approach to audio generation, 2023. 2

    2023

  8. [7]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, Sand- hini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz...

    2020

Show all 96 references
  1. [8]

    Enabling synergistic full-body control in prompt-based co-speech motion generation

    Bohong Chen, Yumeng Li, Yao-Xiang Ding, Tianjia Shao, and Kun Zhou. Enabling synergistic full-body control in prompt-based co-speech motion generation. In Proceedings of the 32nd ACM International Conference on Multimedia , pages 6774–6783, 2024. 2, 6, 7, 17

    2024

  2. [9]

    Motionllm: Understanding human behaviors from human motions and videos, 2024

    Ling-Hao Chen, Shunlin Lu, Ailing Zeng, Hao Zhang, Benyou Wang, Ruimao Zhang, and Lei Zhang. Motionllm: Understanding human behaviors from human motions and videos, 2024. 2

    2024

  3. [10]

    Executing your commands via mo- tion diffusion in latent space

    Xin Chen, Biao Jiang, Wen Liu, Zilong Huang, Bin Fu, Tao Chen, and Gang Yu. Executing your commands via mo- tion diffusion in latent space. In CVPR, pages 18000–18010,

  4. [11]

    Executing your commands via motion diffusion in latent space

    Xin Chen, Biao Jiang, Wen Liu, Zilong Huang, Bin Fu, Tao Chen, Jingyi Yu, and Gang Yu. Executing your commands via motion diffusion in latent space. In CVPR, 2023. 2

    2023

  5. [12]

    Black, and Timo Bolkart

    Kiran Chhatre, Radek Dan ˇeˇcek, Nikos Athanasiou, Giorgio Becherini, Christopher Peters, Michael J. Black, and Timo Bolkart. AMUSE: Emotional speech-driven 3D body ani- mation via disentangled latent diffusion. In CVPR, pages 1942–1953, 2024. 2

    1942

  6. [13]

    M2d2m: Multi-motion generation from text with discrete diffusion models

    Seunggeun Chi, Hyung-gun Chi, Hengbo Ma, Nakul Agar- wal, Faizan Siddiqui, Karthik Ramani, and Kwonjoon Lee. M2d2m: Multi-motion generation from text with discrete diffusion models. arXiv preprint arXiv:2407.14502 , 2024. 2

    2024 arXiv

  7. [14]

    Bert: Pre-training of deep bidirectional trans- formers for language understanding, 2019

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional trans- formers for language understanding, 2019. 3

    2019

  8. [15]

    Listen, think, and understand

    Yuan Gong, Hongyin Luo, Alexander H Liu, Leonid Karlin- sky, and James Glass. Listen, think, and understand. arXiv preprint arXiv:2305.10790, 2023. 2

    2023 arXiv

  9. [16]

    Generating diverse and natural 3d human motions from text

    Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. In CVPR, pages 5152–5161,

  10. [17]

    Tm2t: Stochastic and tokenized modeling for the reciprocal gener- ation of 3d human motions and texts

    Chuan Guo, Xinxin Zuo, Sen Wang, and Li Cheng. Tm2t: Stochastic and tokenized modeling for the reciprocal gener- ation of 3d human motions and texts. In ECCV, pages 580–

  11. [18]

    Momask: Generative masked mod- eling of 3d human motions

    Chuan Guo, Yuxuan Mu, Muhammad Gohar Javed, Sen Wang, and Li Cheng. Momask: Generative masked mod- eling of 3d human motions. In CVPR, pages 1900–1910,

    1900

  12. [19]

    Learning speech-driven 3d conversational gestures from video, 2021

    Ikhsanul Habibie, Weipeng Xu, Dushyant Mehta, Lingjie Liu, Hans-Peter Seidel, Gerard Pons-Moll, Mohamed El- gharib, and Christian Theobalt. Learning speech-driven 3d conversational gestures from video, 2021. 2, 6

    2021

  13. [20]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units

    Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM transactions on audio, speech, and language processing, 29:...

    2021

  14. [21]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 5

    2021 arXiv

  15. [22]

    Audiogpt: Understanding and 9 generating speech, music, sound, and talking head

    Rongjie Huang, Mingze Li, Dongchao Yang, Jiatong Shi, Xuankai Chang, Zhenhui Ye, Yuning Wu, Zhiqing Hong, Ji- awei Huang, Jinglin Liu, et al. Audiogpt: Understanding and 9 generating speech, music, sound, and talking head. In AAAI, pages 23802–23804, 2024. 3

    2024

  16. [23]

    Motiongpt: Human motion as a foreign language

    Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, and Tao Chen. Motiongpt: Human motion as a foreign language. In NeurIPS, 2023. 2, 4, 13, 16

    2023

  17. [24]

    Motionchain: Conversational motion controllers via multimodal prompts

    Biao Jiang, Xin Chen, Chi Zhang, Fukun Yin, Zhuoyuan Li, Gang Yu, and Jiayuan Fan. Motionchain: Conversational motion controllers via multimodal prompts. In ECCV, pages 54–74. Springer, 2025. 2

    2025

  18. [25]

    Scaling up dynamic human-scene interaction mod- eling

    Nan Jiang, Zhiyuan Zhang, Hongjie Li, Xiaoxuan Ma, Zan Wang, Yixin Chen, Tengyu Liu, Yixin Zhu, and Siyuan Huang. Scaling up dynamic human-scene interaction mod- eling. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 1737–1747,

  19. [26]

    Guided motion diffusion for controllable human motion synthesis

    Korrawe Karunratanakul, Konpat Preechakul, Supasorn Suwajanakorn, and Siyu Tang. Guided motion diffusion for controllable human motion synthesis. In ICCV, pages 2151– 2162, 2023. 2

    2023

  20. [27]

    Gen- erating images with multimodal language models

    Jing Yu Koh, Daniel Fried, and Russ R Salakhutdinov. Gen- erating images with multimodal language models. Advances in Neural Information Processing Systems, 36, 2024. 3

    2024

  21. [28]

    Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text process- ing

    T Kudo. Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text process- ing. arXiv preprint arXiv:1808.06226, 2018. 2, 4

    2018 arXiv

  22. [29]

    Subword regularization: Improving neural net- work translation models with multiple subword candidates

    Taku Kudo. Subword regularization: Improving neural net- work translation models with multiple subword candidates. arXiv preprint arXiv:1804.10959, 2018. 4

    2018 arXiv

  23. [30]

    EgoGen: An Egocentric Synthetic Data Generator

    Gen Li, Kaifeng Zhao, Siwei Zhang, Xiaozhong Lyu, Mi- hai Dusmanu, Yan Zhang, Marc Pollefeys, and Siyu Tang. EgoGen: An Egocentric Synthetic Data Generator. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 2

    2024

  24. [31]

    Audio2gestures: Generating diverse gestures from speech audio with conditional varia- tional autoencoders

    Jing Li, Di Kang, Wenjie Pei, Xuefei Zhe, Ying Zhang, Zhenyu He, and Linchao Bao. Audio2gestures: Generating diverse gestures from speech audio with conditional varia- tional autoencoders. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pages 112...

    2021

  25. [32]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In In- ternational conference on machine learning , pages 19730– 19742. PMLR, 2023. 3

    2023

  26. [33]

    BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML,

  27. [34]

    Ego-body pose es- timation via ego-head pose estimation

    Jiaman Li, Karen Liu, and Jiajun Wu. Ego-body pose es- timation via ego-head pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17142–17151, 2023. 2

    2023

  28. [35]

    Videochat: Chat-centric video understanding

    KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. Videochat: Chat-centric video understanding. arXiv preprint arXiv:2305.06355, 2023. 3

    2023 arXiv

  29. [36]

    Ross, and Angjoo Kanazawa

    Ruilong Li, Shan Yang, David A. Ross, and Angjoo Kanazawa. Ai choreographer: Music conditioned 3d dance generation with aist++, 2021. 6

    2021

  30. [37]

    Learning a model of facial shape and expression from 4d scans

    Tianye Li, Timo Bolkart, Michael J Black, Hao Li, and Javier Romero. Learning a model of facial shape and expression from 4d scans. ACM Trans. Graph., 36(6):194–1, 2017. 3

    2017

  31. [38]

    Omg: Towards open-vocabulary motion generation via mixture of controllers

    Han Liang, Jiacheng Bao, Ruichi Zhang, Sihan Ren, Yuecheng Xu, Sibei Yang, Xin Chen, Jingyi Yu, and Lan Xu. Omg: Towards open-vocabulary motion generation via mixture of controllers. In CVPR, pages 482–493, 2024. 2

    2024

  32. [39]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out , pages 74–81, 2004. 8, 13

    2004

  33. [40]

    Disco: Disentangled implicit content and rhythm learning for diverse co-speech gestures synthesis

    Haiyang Liu, Naoya Iwamoto, Zihao Zhu, Zhengqing Li, You Zhou, Elif Bozkurt, and Bo Zheng. Disco: Disentangled implicit content and rhythm learning for diverse co-speech gestures synthesis. In Proceedings of the 30th ACM Interna- tional Conference on Multimedia , pages 3764–37...

    2022

  34. [41]

    Beat: A large-scale semantic and emotional multi-modal dataset for conversational gestures synthesis

    Haiyang Liu, Zihao Zhu, Naoya Iwamoto, Yichen Peng, Zhengqing Li, You Zhou, Elif Bozkurt, and Bo Zheng. Beat: A large-scale semantic and emotional multi-modal dataset for conversational gestures synthesis. In ECCV, 2022. 6, 13

    2022

  35. [42]

    Visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. 2, 3

    2023

  36. [44]

    Plan, posture and go: To- wards open-world text-to-motion generation

    Jinpeng Liu, Wenxun Dai, Chunyu Wang, Yiji Cheng, Yan- song Tang, and Xin Tong. Plan, posture and go: To- wards open-world text-to-motion generation. arXiv preprint arXiv:2312.14828, 2023. 2

    2023 arXiv

  37. [45]

    Troje, Ger- ard Pons-Moll, and Michael J

    Naureen Mahmood, Nima Ghorbani, Nikolaus F. Troje, Ger- ard Pons-Moll, and Michael J. Black. Amass: Archive of motion capture as surface shapes. In ICCV, 2019. 13

    2019

  38. [46]

    Amass: Archive of mo- tion capture as surface shapes

    Naureen Mahmood, Nima Ghorbani, Nikolaus F Troje, Ger- ard Pons-Moll, and Michael J Black. Amass: Archive of mo- tion capture as surface shapes. In ICCV, pages 5442–5451,

  39. [47]

    Learning to listen: Modeling non-deterministic dyadic facial motion

    Evonne Ng, Hanbyul Joo, Liwen Hu, Hao Li, Trevor Darrell, Angjoo Kanazawa, and Shiry Ginosar. Learning to listen: Modeling non-deterministic dyadic facial motion. In CVPR,

  40. [48]

    Can language models learn to listen? In ICCV, 2023

    Evonne Ng, Sanjay Subramanian, Dan Klein, Angjoo Kanazawa, Trevor Darrell, and Shiry Ginosar. Can language models learn to listen? In ICCV, 2023. 2, 5

    2023

  41. [49]

    From audio to photoreal embodiment: Synthesizing humans in conversations, 2024

    Evonne Ng, Javier Romero, Timur Bagautdinov, Shaojie Bai, Trevor Darrell, Angjoo Kanazawa, and Alexander Richard. From audio to photoreal embodiment: Synthesizing humans in conversations, 2024. 2

    2024

  42. [50]

    Librispeech: an asr corpus based on public do- main audio books

    Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. Librispeech: an asr corpus based on public do- main audio books. In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 5206–5210. IEEE, 2015. 6, 13 10

    2015

  43. [51]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages 311–318,

  44. [52]

    Expressive body capture: 3d hands, face, and body from a single image

    Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed AA Osman, Dimitrios Tzionas, and Michael J Black. Expressive body capture: 3d hands, face, and body from a single image. In CVPR, pages 10975– 10985, 2019. 3

    2019

  45. [53]

    Temos: Generating diverse human motions from textual descriptions

    Mathis Petrovich, Michael J Black, and G ¨ul Varol. Temos: Generating diverse human motions from textual descriptions. In European Conference on Computer Vision , pages 480–

  46. [54]

    State of the art on diffusion models for visual computing

    Ryan Po, Wang Yifan, Vladislav Golyanik, Kfir Aberman, Jonathan T Barron, Amit Bermano, Eric Chan, Tali Dekel, Aleksander Holynski, Angjoo Kanazawa, et al. State of the art on diffusion models for visual computing. In Computer Graphics Forum, page e15063. Wiley Online Library, 2024. 3

    2024

  47. [55]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer, 2023. 3, 4, 5

    2023

  48. [56]

    Neural machine translation of rare words with subword units

    Rico Sennrich. Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909, 2015. 4

    2015 arXiv

  49. [57]

    Human motion diffusion as a generative prior

    Yonatan Shafir, Guy Tevet, Roy Kapon, and Amit H Bermano. Human motion diffusion as a generative prior. arXiv preprint arXiv:2303.01418, 2023. 2

    2023 arXiv

  50. [58]

    Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Daniel Cohen-Or, and Amit H. Bermano. Human motion diffusion model. In ICLR, 2022. 2

    2022

  51. [59]

    Human motion diffu- sion model

    Guy Tevet, Sigal Raab, Brian Gordon, Yoni Shafir, Daniel Cohen-or, and Amit Haim Bermano. Human motion diffu- sion model. In The Eleventh International Conference on Learning Representations, 2023. 2, 13, 16

    2023

  52. [60]

    Dyadic interaction modeling for social behavior generation

    Minh Tran, Di Chang, Maksim Siniukov, and Mohammad Soleymani. Dyadic interaction modeling for social behavior generation. In ECCV, 2024. 2

    2024

  53. [61]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, 2017. 3

    2017

  54. [62]

    Motiongpt-2: A general-purpose motion-language model for motion generation and understanding

    Yuan Wang, Di Huang, Yaqi Zhang, Wanli Ouyang, Jile Jiao, Xuetao Feng, Yan Zhou, Pengfei Wan, Shixiang Tang, and Dan Xu. Motiongpt-2: A general-purpose motion-language model for motion generation and understanding. arXiv,

  55. [63]

    Inter- control: Generate human motion interactions by controlling every joint

    Zhenzhi Wang, Jingbo Wang, Dahua Lin, and Bo Dai. Inter- control: Generate human motion interactions by controlling every joint. arXiv preprint arXiv:2311.15864, 2023. 2

    2023 arXiv

  56. [64]

    Move as you say, interact as you can: Language-guided human motion generation with scene af- fordance

    Zan Wang, Yixin Chen, Baoxiong Jia, Puhao Li, Jinlu Zhang, Jingze Zhang, Tengyu Liu, Yixin Zhu, Wei Liang, and Siyuan Huang. Move as you say, interact as you can: Language-guided human motion generation with scene af- fordance. In Proceedings of the IEEE/CVF Conference on Comp...

    2024

  57. [65]

    Motionllm: Multimodal motion-language learning with large language models, 2024

    Qi Wu, Yubo Zhao, Yifan Wang, Yu-Wing Tai, and Chi- Keung Tang. Motionllm: Multimodal motion-language learning with large language models, 2024. 2

    2024

  58. [66]

    Vila-u: a unified foundation model inte- grating visual understanding and generation

    Yecheng Wu, Zhuoyang Zhang, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu Yin, Li Yi, et al. Vila-u: a unified foundation model inte- grating visual understanding and generation. arXiv preprint arXiv:2409.04429, 2024. 3

    2024 arXiv

  59. [67]

    Omnicontrol: Control any joint at any time for human motion generation

    Yiming Xie, Varun Jampani, Lei Zhong, Deqing Sun, and Huaizu Jiang. Omnicontrol: Control any joint at any time for human motion generation. arXiv preprint arXiv:2310.08580,

    arXiv

  60. [68]

    Codetalker: Speech-driven 3d facial animation with discrete motion prior

    Jinbo Xing, Menghan Xia, Yuechen Zhang, Xiaodong Cun, Jue Wang, and Tien-Tsin Wong. Codetalker: Speech-driven 3d facial animation with discrete motion prior. In CVPR, pages 12780–12790, 2023. 2

    2023

  61. [69]

    Diffus- estylegesture: Stylized audio-driven co-speech gesture gen- eration with diffusion models

    Sicheng Yang, Zhiyong Wu, Minglei Li, Zhensong Zhang, Lei Hao, Weihong Bao, Ming Cheng, and Long Xiao. Diffus- estylegesture: Stylized audio-driven co-speech gesture gen- eration with diffusion models. In Proceedings of the Thirty- Second International Joint Conference on Arti...

    2023

  62. [70]

    Generating holistic 3d human motion from speech

    Hongwei Yi, Hualin Liang, Yifei Liu, Qiong Cao, Yandong Wen, Timo Bolkart, Dacheng Tao, and Michael J Black. Generating holistic 3d human motion from speech. InCVPR, pages 469–480, 2023. 2

    2023

  63. [71]

    Hongwei Yi, Hualin Liang, Yifei Liu, Qiong Cao, Yandong Wen, Timo Bolkart, Dacheng Tao, and Michael J. Black. Generating holistic 3d human motion from speech. InCVPR,

  64. [72]

    Speech ges- ture generation from the trimodal context of text, audio, and speaker identity

    Youngwoo Yoon, Bok Cha, Joo-Haeng Lee, Minsu Jang, Jaeyeon Lee, Jaehong Kim, and Geehyuk Lee. Speech ges- ture generation from the trimodal context of text, audio, and speaker identity. ACM Transactions on Graphics , 39(6),

  65. [73]

    Physdiff: Physics-guided human motion diffusion model

    Ye Yuan, Jiaming Song, Umar Iqbal, Arash Vahdat, and Jan Kautz. Physdiff: Physics-guided human motion diffusion model. In ICCV, pages 16010–16021, 2023. 2

    2023

  66. [74]

    Speechgpt: Empow- ering large language models with intrinsic cross-modal con- versational abilities, 2023

    Dong Zhang, Shimin Li, Xin Zhang, Jun Zhan, Pengyu Wang, Yaqian Zhou, and Xipeng Qiu. Speechgpt: Empow- ering large language models with intrinsic cross-modal con- versational abilities, 2023. 2, 3

    2023

  67. [75]

    Neural- dome: A neural modeling pipeline on multi-view human- object interactions

    Juze Zhang, Haimin Luo, Hongdi Yang, Xinru Xu, Qianyang Wu, Ye Shi, Jingyi Yu, Lan Xu, and Jingya Wang. Neural- dome: A neural modeling pipeline on multi-view human- object interactions. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pa...

    2023

  68. [76]

    T2m-gpt: Generating human motion from textual de- scriptions with discrete representations

    Jianrong Zhang, Yangsong Zhang, Xiaodong Cun, Shaoli Huang, Yong Zhang, Hongwei Zhao, Hongtao Lu, and Xi Shen. T2m-gpt: Generating human motion from textual de- scriptions with discrete representations. In CVPR, 2023. 2

    2023

  69. [77]

    Generating human motion from textual descriptions with discrete representations

    Jianrong Zhang, Yangsong Zhang, Xiaodong Cun, Yong Zhang, Hongwei Zhao, Hongtao Lu, Xi Shen, and Ying Shan. Generating human motion from textual descriptions with discrete representations. In CVPR, pages 14730–14740,

  70. [78]

    Hoi-mˆ 3: Capture multiple humans and objects in- teraction within contextual environment

    Juze Zhang, Jingyan Zhang, Zining Song, Zhanhe Shi, Chengfeng Zhao, Ye Shi, Jingyi Yu, Lan Xu, and Jingya Wang. Hoi-mˆ 3: Capture multiple humans and objects in- teraction within contextual environment. In CVPR, pages 516–526, 2024. 2

    2024

  71. [79]

    Motiondif- fuse: Text-driven human motion generation with diffusion model

    Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondif- fuse: Text-driven human motion generation with diffusion model. arXiv preprint arXiv:2208.15001, 2022. 2

    2022 arXiv

  72. [80]

    Re- modiffuse: Retrieval-augmented motion diffusion model

    Mingyuan Zhang, Xinying Guo, Liang Pan, Zhongang Cai, Fangzhou Hong, Huirong Li, Lei Yang, and Ziwei Liu. Re- modiffuse: Retrieval-augmented motion diffusion model. In ICCV, pages 364–373, 2023. 2

    2023

  73. [81]

    Large motion model for unified multi-modal motion generation

    Mingyuan Zhang, Daisheng Jin, Chenyang Gu, Fangzhou Hong, Zhongang Cai, Jingfang Huang, Chongzhi Zhang, Xinying Guo, Lei Yang, Ying He, et al. Large motion model for unified multi-modal motion generation. arXiv, 2024. 2

    2024

  74. [82]

    Bertscore: Evaluating text genera- tion with bert

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Wein- berger, and Yoav Artzi. Bertscore: Evaluating text genera- tion with bert. arXiv preprint arXiv:1904.09675 , 2019. 8, 13

    1904 arXiv

  75. [83]

    Mo- tiongpt: Finetuned llms are general-purpose motion genera- tors

    Yaqi Zhang, Di Huang, Bin Liu, Shixiang Tang, Yan Lu, Lu Chen, Lei Bai, Qi Chu, Nenghai Yu, and Wanli Ouyang. Mo- tiongpt: Finetuned llms are general-purpose motion genera- tors. In AAAI, 2024. 2

    2024

  76. [84]

    Freemotion: Mocap-free human motion synthesis with multimodal large language models

    Zhikai Zhang, Yitang Li, Haofeng Huang, Mingxian Lin, and Li Yi. Freemotion: Mocap-free human motion synthesis with multimodal large language models. In ECCV, pages 403–

  77. [85]

    A survey of large language models, 2024

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong W...

    2024

  78. [86]

    Responsive listening head generation: A benchmark dataset and baseline

    Mohan Zhou, Yalong Bai, Wei Zhang, Ting Yao, Tiejun Zhao, and Tao Mei. Responsive listening head generation: A benchmark dataset and baseline. In ECCV, 2022. 2

    2022

  79. [87]

    Emdm: Efficient motion diffusion model for fast and high-quality motion generation

    Wenyang Zhou, Zhiyang Dou, Zeyu Cao, Zhouyingcheng Liao, Jingbo Wang, Wenjia Wang, Yuan Liu, Taku Komura, Wenping Wang, and Lingjie Liu. Emdm: Efficient motion diffusion model for fast and high-quality motion generation. In ECCV, pages 18–38. Springer, 2025. 2 12

    2025

  80. [88]

    Supplementary In this supplementary material, we provide additional de- tails about:

  81. [89]

    Supplementary video for qualitative examples (refer- enced in Sec. 1)

  82. [90]

    Additional details on post-training (referenced in Sec. 3.4)

  83. [91]

    Additional details on editable generation (referenced in Sec. 4.3)

  84. [92]

    Additional details on emotion prediction (referenced in Sec. 4.4)

  85. [93]

    Results for text-to-motion

  86. [94]

    Additional implementation details

  87. [95]

    Additional qualitative example of co-speech gesture generation. 7.1. Supplementary Video We provide a supplemental video to illustrate our results. In the video, we present: 1) an overview of our overall frame- work, 2) detailed qualitative comparisons across four tasks: co-sp...

  88. [96]

    angered” and “upset

    examples of failure cases to inspire further research. We recommend watching this video with your headphone, as video results provide a more comprehensive understanding of our approach. 7.2. Additional Details on Post-training Existing datasets primarily provide pair-wise moti...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.