Pith. sign in

REVIEW 4 major objections 5 minor 57 references

Embracing Aleatoric Uncertainty: Generating Diverse 3D Human Motion

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Injecting noise nearly triples the diversity of text-driven 3D motion.

desk verdict The MultiModality gains are likely inflated by an unablated temporal-stretch augmentation, so the headline diversity numbers should be treated with caution until the confound is resolved. read the letter →

arxiv 2508.20604 v1 pith:4MSCDFAT submitted 2025-08-28 cs.CV

Zheng Qin , Yabing Wang , Minghui Yang , Sanping Zhou , Ming Yang , Le Wang This is my paper · ORCID
classification cs.CV
keywords text-to-motiongeneration3DhumanmotiondiversityaleatoricuncertaintynoiseguidancevariationaltransformerresidualVQ-VAEmaskedgenerative
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Text-to-motion models that discretize motion into code sequences produce precise motions but tend to generate nearly identical outputs for the same prompt, because one text maps to one code sequence. Diverse-T2M claims that injecting a fresh Gaussian noise signal as a second conditioning input, together with sampling from a learned latent distribution, turns each inference into a draw from a range of text-consistent motions. On HumanML3D the method reports a MultiModality of 3.976 against 2.799 for the next-best method, while keeping Top-1 retrieval precision at 52.5% and FID at 0.057; similar gains appear on KIT-ML. The intended consequence is that animators get many usable, semantically faithful variants from a single sentence.

What carries the argument

The load-bearing mechanism is the stochastic fusion of two code distributions. The same variational predictor is asked to map a CLIP text feature and a Gaussian noise vector separately into distributions over motion codes; the output distribution is $I=(1+w)I_{\text{text}} - w I_{\text{noise}}$, borrowing the classifier-free guidance combination. A latent space sampler inside the predictor, parameterized as a diagonal Gaussian via mean and standard-deviation transformer layers, injects reparameterized randomness $z=\mu+\sigma\odot\epsilon$ before the code-probability MLP. The fusion weight $w$ is the single dial that trades text control against diversity: the paper sweeps it on the test set and chooses $w=3$.

What would settle it

Fix a text prompt and a fixed latent-space draw, then generate many motions using only different noise signals $g_\epsilon$. If the spread of these motions, measured by pairwise distance or MultiModality, is no larger than when the noise signal is held fixed and the latent draw varies, then the noise carrier is not the diversity source the paper claims; checking whether those motions all satisfy the text through R-Precision would reveal whether the extra variety is semantically controlled.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the diversity bottleneck in two-stage text-to-motion generation is not the discrete codebook but the deterministic mapping from text to codes. By training a noise-conditional model alongside the text-conditional model and fusing their code distributions as $I=(1+w)I_{\text{text}}-w I_{\text{noise}}$, with $w=3$ at inference, the model obtains markedly higher MultiModality while preserving retrieval accuracy and low FID. The second mechanism, a variational text-to-codes predictor, replaces the one-to-one mapping with a distribution $q_\phi(z|e)=\mathcal{N}(z;\mu,\mathrm{diag}(\sigma^2))$ and samples $z=\mu+\sigma\odot\epsilon$ with $\epsilon\sim\mathcal{N}(0,I)$, adding stochasticity inside the transformer. On HumanML3D the ablation shows each component contributes, with the noise signal adding the larger part of the diversity gain over the RVQ-VAE baseline.

Load-bearing premise

The argument stands or falls on the premise that a Gaussian noise vector, combined with the text signal through $I=(1+w)I_{\text{text}}-w I_{\text{noise}}$, adds useful variation rather than random semantic corruption; if the noise term mostly bypasses the text condition, the reported diversity gain would not be text-consistent variation.

Editorial extensions

If this is right

  • A single text prompt can yield a batch of distinct, semantically valid motions, which is what character animation pipelines need for downstream selection and editing.
  • Two-stage token-based generators can reach diversity comparable to or better than diffusion models without iterative denoising at inference.
  • Diversity and text consistency are not in inherent tension: on HumanML3D the method reports R-Precision rising from 0.518 to 0.525 while MultiModality rises from 1.221 to 3.976 over its RVQ baseline.
  • The fusion weight $w$ gives practitioners a tunable trade-off; raising $w$ increases variety at the cost of retrieval accuracy and multimodal distance, as shown in the paper's sweep.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if the noise signal is truly the diversity carrier, then conditioning on structured noise, such as a latent style code, rather than isotropic Gaussian could turn the method into a controllable diversity knob that lets users request variation along a chosen axis.
  • Editorial inference: the same fusion rule could be tested in other conditional token generators, such as speech or video codec transformers, where map-like architectures also suffer from low sample diversity.
  • Editorial inference: because $w=3$ is selected on the test set, a deployment-ready version would need validation-based selection or a per-prompt adaptive weight; otherwise the reported trade-off curve may rely on test-set information.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes Diverse-T2M, a two-stage text-to-motion generation method built on residual VQ-VAE and a masked transformer. To increase diversity, the authors add a noise-conditioned generation branch alongside the text-conditioned branch and fuse the two code distributions through I = (1+w)I_text - w I_noise (Eq. 4). They also replace the deterministic text-to-code mapping with a variational predictor that samples a latent embedding from a Gaussian and maps it to code probabilities. On HumanML3D and KIT-ML the method reports MultiModality 3.976 and 3.462, substantially above prior methods, while maintaining R-Precision and FID at roughly the level of MoMask. Ablations in Table III separate the contributions of the variational predictor and the noise signal, and confidence intervals are reported for all benchmark numbers.

Significance. If the reported diversity gains are real and attributable to the proposed mechanism, the paper makes a useful practical contribution: a simple modification to a standard two-stage architecture that substantially raises the MultiModality metric without sacrificing text consistency, with qualitative examples and an application story. The paper is also commendable for reporting 95% confidence intervals and for performing component ablations on two baselines. However, the central claim is not yet established because the diversity gain may be inflated by a training-only temporal-stretch augmentation on the noise branch and by test-set selection of the fusion weight. These issues are fixable within the manuscript's scope, so the contribution is promising but requires substantive revision.

major comments (4)
  1. [III-B2, Table III] In Section III-B2, the 'Length Stochastic Enhancement' randomly stretches and compresses the motion sequences only when training the noise-conditioned branch. Because MultiModality is computed as the average pairwise feature distance among 30 generated motions for the same text, random duration or speed variation alone can inflate it without adding semantically meaningful pose-level alternatives. The +NS rows in Table III therefore conflate the proposed noise signal with this augmentation. Please ablate the length enhancement (for example, train the noise-conditioned branch without temporal stretch, or apply the same augmentation to the text branch), report the stretch/compress range and sampling probability, and evaluate text consistency on prompts containing speed or duration terms such as 'fast' or 'slow'.
  2. [IV-A, Figure 5] Figure 5 sweeps w over a range and the paper then fixes w=3 for the reported results; if this sweep is performed on the test set, the headline MultiModality value is the result of optimizing the diversity metric on the evaluation split. The figure shows MultiModality increasing with w while R-Precision and MM-Dist deteriorate, so the chosen operating point directly trades away semantic control to maximize the reported diversity. Please clarify which split was used for the sweep, use a validation split for selecting w, and report the sensitivity of the main conclusions to this choice.
  3. [III-B2, III-D2, IV-A] Two training hyperparameters central to the method are not reported: the noise-mixing probability p_noise introduced in Section III-B2 and the text/noise mixing ratio used in training in Section III-D2. In addition, the KL weight is set to 1e-5, which is effectively no regularization of the latent distribution toward N(0,1); this weakens the claim that the predictor is variational and leaves open whether the latent sampler contributes anything beyond an unconstrained stochastic mapping. Please report p_noise and the training mixture ratio, and ablate the KL weight over a range of values.
  4. [Eq. (3)-(4)] Equation (3) defines p_t^k as the probability of code k at time t, and I_text and I_noise are described as probability distributions. Equation (4) then computes I = (1+w)I_text - w I_noise, which with w>0 can produce negative entries and does not in general sum to one, so it is not a valid probability distribution. If I_text and I_noise are actually logits, Eq. (3) and the surrounding text should say so; if they are probabilities, the fusion step needs to be defined in logit space or normalized explicitly. Please resolve this inconsistency, as it determines how the method is implemented and whether the 'probability distribution' terminology is accurate.
minor comments (5)
  1. [Throughout] There are numerous typos and inconsistent naming: 'Diverser-T2M' in Section I, 'tockens' in Section III-C, 'infernce' in Section III-C, 'Digram' in Section IV-C, and 'three two benchmark datasets' in Section IV-A should be corrected.
  2. [Introduction, Figure 1] The caption of Figure 1 contains 'Derails' and 'Fid'; the Introduction states the MultiModality gain is '30% higher' than the second place, while Table I shows 3.976 vs 2.799, which is about 42% higher; please reconcile the numbers.
  3. [Section II, References] In the related work, 'MDM [27]' is cited with the same reference number as MotionCLIP [27], while the actual MDM paper appears to be reference [51]; please correct this citation error and verify that all references are uniquely assigned.
  4. [Section IV-A] The experimental settings report the codebook size as 512x512 with 512-dimensional vectors, but Eq. (2) denotes V residual codebooks; please clarify the relationship between the residual layers (V=6) and the codebook structure, and state the dimension d_c used in the experiments.
  5. [Eq. (13)] In Eq. (13), the notation [mask]_t is used both as an indicator variable and as a token placeholder; please define the notation explicitly to avoid ambiguity about which positions are masked.

Circularity Check

1 steps flagged · score 6.0 of 10

Headline diversity is reported at a noise-fusion weight w chosen by a sweep over the same diversity metric, making the quantitative MultiModality gain a tuned operating point rather than an independent prediction.

  1. fitted input called prediction [Section IV-C (Ablation Study, Figure 5) and Section IV-A (Experimental Settings; 'During inference, the mixing ratio w ... is set to 3')]
    "During inference, the mixing ratio w for the generation of the noise signal and text signal results is set to 3. ... In Figure 5, The larger w is, the more the noise signal affects the final generated motion. Digram (b) indicates that the generation diversity is continuously getting larger as w increases."

    The headline MultiModality value (3.976 vs. 2.799 for the next method) is measured at w=3 after a sweep that shows MultiModality grows monotonically with w. The paper reports no held-out selection criterion, so the quantitative diversity advantage is an in-sample operating point chosen on the same benchmark, not a prediction independent of the tuning procedure. This makes the magnitude of the diversity gain partly constructed by the choice of w; the qualitative claim that the noise branch increases diversity is still supported by the +NS ablation, so the circularity is partial.

full rationale

No equation-level derivation in the paper reduces to its inputs: the RVQ-VAE codebook, masked-transformer code prediction, latent-space KL term, and the fusion rule I=(1+w)I_text - w I_noise are ordinary generative-model components, and the headline comparisons use external HumanML3D and KIT-ML features, so the main pipeline is not circular. The one substantive circular element is the choice of w: Figure 5 shows MultiModality increasing monotonically with w, and the paper then reports its best results at w=3 without a separate validation split. The reported diversity gain is therefore a selected operating point rather than an independent, parameter-free prediction. The unablated length-stochastic augmentation applied only to the noise branch is a potential confound that could also inflate MultiModality, but it is an experimental-design issue rather than a definitional reduction, so it is not counted as a circular step. Self-citations in the paper are not load-bearing.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central claim rests on a small number of modeling assumptions about noise and latent-space stochasticity, plus several hyperparameters that are either tuned on test metrics or left unspecified. No new physical or mathematical entities are introduced.

free parameters (4)
  • Fusion weight w = 3
    Selected by the sweep in Figure 5 to balance MultiModality and R Precision; the sweep is evaluated on test-set metrics, so this is a form of test-set tuning.
  • KL divergence weight = 1e-5
    Set in Section IV-A; this near-zero weight makes the latent prior regularization negligible, which affects how much the variational predictor regularizes diversity.
  • Noise mixing probability p_noise
    Introduced in Section III-B2 as a training hyperparameter that controls how often the noise signal replaces the text signal; its value is never reported, blocking exact reproduction.
  • Latent space dimension and transformer depth
    The variational predictor's latent dimension D and number of transformer layers N are not stated; they determine the capacity of the diversity mechanism.
assumptions (6)
  • domain assumption A text description corresponds to a distribution of valid motions, and aleatoric uncertainty can be injected as Gaussian noise in the conditioning signal.
    Stated in Section I and III-A; this frames the whole diversity mechanism.
  • domain assumption CLIP text features are a sufficient condition signal, and a linear combination of text-conditioned and noise-conditioned code distributions preserves semantic alignment at w=3.
    Used in Eq. (4); if false, the fusion would corrupt text control.
  • domain assumption The mask transformer trained with mixed text/noise conditions generalizes to inference-time fusion of both conditions.
    Training in Section III-D uses one condition at a time; inference uses both via Eq. (4), an extrapolation the paper does not justify.
  • domain assumption The MultiModality metric is a valid measure of useful diversity rather than noise sensitivity.
    Section IV-A; the headline result is defined by this metric, which can be inflated by any stochastic perturbation.
  • standard math The reparameterization trick yields differentiable sampling from q_phi(z|e).
    Eq. (9); standard VAE result, not in question.
  • domain assumption RVQ-VAE discrete representation from MoMask is an adequate substrate for diverse motion generation.
    Section III-B1; if the codebook cannot represent the variation space, diversity is limited by stage 1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Embracing Aleatoric Uncertainty: Generating Diverse 3D Human Motion." pith.science (2026). https://pith.science/paper/4MSCDFAT

@misc{pith2026250820604,
  author       = {Pith},
  title        = {Pith review of: Embracing Aleatoric Uncertainty: Generating Diverse 3D Human Motion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4MSCDFAT}},
  note         = {Machine review of arXiv:2508.20604}
}
read the original abstract

Generating 3D human motions from text is a challenging yet valuable task. The key aspects of this task are ensuring text-motion consistency and achieving generation diversity. Although recent advancements have enabled the generation of precise and high-quality human motions from text, achieving diversity in the generated motions remains a significant challenge. In this paper, we aim to overcome the above challenge by designing a simple yet effective text-to-motion generation method, \textit{i.e.}, Diverse-T2M. Our method introduces uncertainty into the generation process, enabling the generation of highly diverse motions while preserving the semantic consistency of the text. Specifically, we propose a novel perspective that utilizes noise signals as carriers of diversity information in transformer-based methods, facilitating a explicit modeling of uncertainty. Moreover, we construct a latent space where text is projected into a continuous representation, instead of a rigid one-to-one mapping, and integrate a latent space sampler to introduce stochastic sampling into the generation process, thereby enhancing the diversity and uncertainty of the outputs. Our results on text-to-motion generation benchmark datasets~(HumanML3D and KIT-ML) demonstrate that our method significantly enhances diversity while maintaining state-of-the-art performance in text consistency.

Figures

Figures reproduced from arXiv: 2508.20604 by the authors.

Figure 1
Figure 1. MultiModality-R Precision-Fid comparisons of different text-to￾motion generation methods on HumanML3D dataset. The horizontal axis is MultiModality (diversity), the vertical axis is R Precision Top-1 (semantic consistency), and the radius of circle reflects Fid (fidelity). Our Diverse￾T2M achieves 3.976 MultiModality, 52.5% R Precision Top-1, 0.057 FID, outperforming all previous methods. Derails are given in Table … view at source ↗
Figure 2
Figure 2. (a) Visualization of codebook. We visualize each code feature in the codebook using the decoder. Each code comprises a spatial displacement and a [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Our Diverse-T2M follows the two-stage paradigm, employing RVQ-VAE for motion discrete representation (above) and learning text-to-code sequence [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Illustration of the Variational Text-to-Code Predictor. This model takes masked motion tokens and signal feature as input, producing a code distribution [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Evaluation sweep over the mixing ratio w for the results of the noise signal and text signal during inference, assessing metrics such as R Precision, MultiModality, FID, and MultiModal Dist. The larger the value of w, the greater the effect of the noise signal on the g…
Figure 6
Figure 6. Figure 6: Qualitative comparison. We compare our method qualitatively with T2M-GPT [15] and MoMask [14], both follow the two-stage paradigm. The other [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Visual results of our Diverse-T2M can generate precise and high-fidelity human motion consistent with the given text descriptions. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Illustration of Diverse-T2M’s application in a business scenario. With our Diverse-T2M, animators only need to input the required text (bottom) [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 26 canonical work pages

  1. [1]

    Dancing to music,

    H.-Y . Lee, X. Yang, M.-Y . Liu, T.-C. Wang, Y .-D. Lu, M.-H. Yang, and J. Kautz, “Dancing to music,”Advances in neural information processing systems, vol. 32, 2019

    2019

  2. [2]

    Danceformer: Music con- ditioned 3d dance generation with parametric motion transformer,

    B. Li, Y . Zhao, S. Zhelun, and L. Sheng, “Danceformer: Music con- ditioned 3d dance generation with parametric motion transformer,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 36, no. 2, 2022, pp. 1272–1279

    2022

  3. [3]

    Ai choreographer: Music conditioned 3d dance generation with aist++,

    R. Li, S. Yang, D. A. Ross, and A. Kanazawa, “Ai choreographer: Music conditioned 3d dance generation with aist++,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 13 401–13 412

    2021

  4. [4]

    Amp: Adversarial motion priors for stylized physics-based character control,

    X. B. Peng, Z. Ma, P. Abbeel, S. Levine, and A. Kanazawa, “Amp: Adversarial motion priors for stylized physics-based character control,” ACM Transactions on Graphics (ToG) , vol. 40, no. 4, pp. 1–20, 2021

    2021

  5. [5]

    Deepphase: Periodic autoencoders for learning motion phase manifolds,

    S. Starke, I. Mason, and T. Komura, “Deepphase: Periodic autoencoders for learning motion phase manifolds,” ACM Transactions on Graphics (TOG), vol. 41, no. 4, pp. 1–13, 2022

    2022

  6. [6]

    A spatio- temporal continuous network for stochastic 3d human motion predic- tion,

    H. Yu, Y . Hou, X. Gui, S. Feng, D. Zhou, and Q. Zhang, “A spatio- temporal continuous network for stochastic 3d human motion predic- tion,” IEEE Transactions on Circuits and Systems for Video Technology, 2025

    work page 2025

  7. [7]

    Action-conditioned 3d human motion synthesis with transformer vae,

    M. Petrovich, M. J. Black, and G. Varol, “Action-conditioned 3d human motion synthesis with transformer vae,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 10 985–10 995

    work page 2021

  8. [8]

    Action2motion: Conditioned generation of 3d human motions,

    C. Guo, X. Zuo, S. Wang, S. Zou, Q. Sun, A. Deng, M. Gong, and L. Cheng, “Action2motion: Conditioned generation of 3d human motions,” in Proceedings of the 28th ACM International Conference on Multimedia, 2020, pp. 2021–2029

    2020

Show all 57 references
  1. [9]

    Temos: Generating diverse human motions from textual descriptions,

    M. Petrovich, M. J. Black, and G. Varol, “Temos: Generating diverse human motions from textual descriptions,” in European Conference on Computer Vision. Springer, 2022, pp. 480–497

    2022

  2. [10]

    Progressive human motion generation based on text and few motion frames,

    L.-A. Zeng, G. Wu, A. Wu, J.-F. Hu, and W.-S. Zheng, “Progressive human motion generation based on text and few motion frames,” IEEE Transactions on Circuits and Systems for Video Technology , 2025

    2025

  3. [11]

    Language2pose: Natural language grounded pose forecasting,

    C. Ahuja and L.-P. Morency, “Language2pose: Natural language grounded pose forecasting,” in 2019 International Conference on 3D Vision (3DV). IEEE, 2019, pp. 719–728

    2019

  4. [12]

    Tm2t: Stochastic and tokenized modeling for the reciprocal generation of 3d human motions and texts,

    C. Guo, X. Zuo, S. Wang, and L. Cheng, “Tm2t: Stochastic and tokenized modeling for the reciprocal generation of 3d human motions and texts,” in European Conference on Computer Vision . Springer, 2022, pp. 580–597

    2022

  5. [13]

    Flame: Free-form language-based motion synthesis & editing,

    J. Kim, J. Kim, and S. Choi, “Flame: Free-form language-based motion synthesis & editing,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 7, 2023, pp. 8255–8263

    2023

  6. [14]

    Momask: Generative masked modeling of 3d human motions,

    C. Guo, Y . Mu, M. G. Javed, S. Wang, and L. Cheng, “Momask: Generative masked modeling of 3d human motions,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 1900–1910

    2024

  7. [15]

    Generating human motion from textual descriptions with discrete representations,

    J. Zhang, Y . Zhang, X. Cun, Y . Zhang, H. Zhao, H. Lu, X. Shen, and Y . Shan, “Generating human motion from textual descriptions with discrete representations,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2023, pp. 14 730–14 740

    2023

  8. [16]

    Motiondiffuse: Text-driven human motion generation with diffusion model,

    M. Zhang, Z. Cai, L. Pan, F. Hong, X. Guo, L. Yang, and Z. Liu, “Motiondiffuse: Text-driven human motion generation with diffusion model,” arXiv preprint arXiv:2208.15001 , 2022

    2022 arXiv

  9. [17]

    Remodiffuse: Retrieval-augmented motion diffusion model,

    M. Zhang, X. Guo, L. Pan, Z. Cai, F. Hong, H. Li, L. Yang, and Z. Liu, “Remodiffuse: Retrieval-augmented motion diffusion model,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 364–373

    2023

  10. [18]

    Attt2m: Text-driven hu- man motion generation with multi-perspective attention mechanism,

    C. Zhong, L. Hu, Z. Zhang, and S. Xia, “Attt2m: Text-driven hu- man motion generation with multi-perspective attention mechanism,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 509–519

    2023

  11. [19]

    Neural discrete representation learning,

    A. Van Den Oord, O. Vinyals et al. , “Neural discrete representation learning,” Advances in neural information processing systems , vol. 30, 2017

    2017

  12. [20]

    T2m-hifigpt: Generating high quality human motion from tex- tual descriptions with residual discrete representations,

    C. Wang, “T2m-hifigpt: Generating high quality human motion from tex- tual descriptions with residual discrete representations,” ar‘Xiv preprint arXiv:2312.10628, 2023

    2023 arXiv

  13. [21]

    Soundstream: An end-to-end neural audio codec,

    N. Zeghidour, A. Luebs, A. Omran, J. Skoglund, and M. Tagliasacchi, “Soundstream: An end-to-end neural audio codec,” IEEE/ACM Transac- tions on Audio, Speech, and Language Processing, vol. 30, pp. 495–507, 2021

    2021

  14. [22]

    High fidelity neural audio compression,

    A. D ´efossez, J. Copet, G. Synnaeve, and Y . Adi, “High fidelity neural audio compression,” arXiv preprint arXiv:2210.13438 , 2022

    2022 arXiv

  15. [23]

    Improving language understanding by generative pre-training,

    A. Radford, K. Narasimhan, T. Salimans, I. Sutskever et al., “Improving language understanding by generative pre-training,” 2018

    2018

  16. [24]

    Muse: Text-to- image generation via masked generative transformers,

    H. Chang, H. Zhang, J. Barber, A. Maschinot, J. Lezama, L. Jiang, M.-H. Yang, K. Murphy, W. T. Freeman, M. Rubinstein et al., “Muse: Text-to- image generation via masked generative transformers,” arXiv preprint arXiv:2301.00704, 2023

    2023 arXiv

  17. [25]

    Geodiff: A geometric diffusion model for molecular conformation generation,

    M. Xu, L. Yu, Y . Song, C. Shi, S. Ermon, and J. Tang, “Geodiff: A geometric diffusion model for molecular conformation generation,” arXiv preprint arXiv:2203.02923 , 2022

    2022 arXiv

  18. [26]

    High- resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10 684–10 695

    2022

  19. [27]

    Motionclip: Exposing human motion generation to clip space,

    G. Tevet, B. Gordon, A. Hertz, A. H. Bermano, and D. Cohen-Or, “Motionclip: Exposing human motion generation to clip space,” in European Conference on Computer Vision . Springer, 2022, pp. 358– 374

    2022

  20. [28]

    Finemogen: Fine-grained spatio-temporal motion generation and editing,

    M. Zhang, H. Li, Z. Cai, J. Ren, L. Yang, and Z. Liu, “Finemogen: Fine-grained spatio-temporal motion generation and editing,” Advances in Neural Information Processing Systems , vol. 36, 2024

    2024

  21. [29]

    Executing your commands via motion diffusion in latent space,

    X. Chen, B. Jiang, W. Liu, Z. Huang, B. Fu, T. Chen, and G. Yu, “Executing your commands via motion diffusion in latent space,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 18 000–18 010

    2023

  22. [30]

    Text2action: Generative ad- versarial synthesis from language to action,

    H. Ahn, T. Ha, Y . Choi, H. Yoo, and S. Oh, “Text2action: Generative ad- versarial synthesis from language to action,” in 2018 IEEE International Conference on Robotics and Automation (ICRA) , May 2018

    2018

  23. [31]

    Synthesis of compositional animations from textual descriptions,

    A. Ghosh, N. Cheema, C. Oguz, C. Theobalt, and P. Slusallek, “Synthesis of compositional animations from textual descriptions,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 1396–1406

    2021

  24. [32]

    Being comes from not-being: Open-vocabulary text-to-motion generation with wordless training,

    J. Lin, J. Chang, L. Liu, G. Li, L. Lin, Q. Tian, and C.-w. Chen, “Being comes from not-being: Open-vocabulary text-to-motion generation with wordless training,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2023, pp. 23 222–23 231

    2023

  25. [33]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision,” in International conference on machine learning . PMLR, 2021, pp. 8748–8763

    2021

  26. [34]

    Dual-view curricular optimal transport for cross-lingual cross-modal retrieval,

    Y . Wang, S. Wang, H. Luo, J. Dong, F. Wang, M. Han, X. Wang, and M. Wang, “Dual-view curricular optimal transport for cross-lingual cross-modal retrieval,” IEEE Transactions on Image Processing, vol. 33, pp. 1522–1533, 2024

    2024

  27. [35]

    Reading-strategy inspired visual representation learning for text-to- IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY , VOL. X, NO. X 13 video retrieval,

    J. Dong, Y . Wang, X. Chen, X. Qu, X. Li, Y . He, and X. Wang, “Reading-strategy inspired visual representation learning for text-to- IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY , VOL. X, NO. X 13 video retrieval,” IEEE Transactions on Circuits and Systems f...

    2022

  28. [36]

    Denoising reuse: Exploiting inter-frame motion consistency for efficient video generation,

    C. Wang, S. Yan, Y . Chen, X. Wang, Y . Wang, M. Dong, X. Yang, D. Li, R. Zhu, D. A. Clifton et al. , “Denoising reuse: Exploiting inter-frame motion consistency for efficient video generation,” IEEE Transactions on Circuits and Systems for Video Technology , 2025

    2025

  29. [37]

    Up-person: Unified parameter-efficient transfer learning for text-based person retrieval,

    Y . Liu, Y . Li, X. Lan, W. Yang, Z. Liu, and Q. Liao, “Up-person: Unified parameter-efficient transfer learning for text-based person retrieval,” IEEE Transactions on Circuits and Systems for Video Technology, 2025

    2025

  30. [38]

    Versatile multimodal controls for expressive talking human animation,

    Z. Qin, R. Zheng, Y . Wang, T. Li, Z. Zhu, S. Zhou, M. Yang, and L. Wang, “Versatile multimodal controls for expressive talking human animation,” arXiv preprint arXiv:2503.08714 , 2025

    2025 arXiv

  31. [39]

    Generating diverse and natural 3d human motions from text

    C. Guo, S. Zou, X. Zuo, S. Wang, W. Ji, X. Li, and L. Cheng, “Generating diverse and natural 3d human motions from text.”

  32. [40]

    Temos: Generating diverse human motions from textual descriptions,

    M. Petrovich, M. Black, and G. Varol, “Temos: Generating diverse human motions from textual descriptions,” Apr 2022

    2022

  33. [41]

    Teach: Temporal action composition for 3d humans,

    N. Athanasiou, M. Petrovich, M. Black, and G. Varol, “Teach: Temporal action composition for 3d humans,” Sep 2022

    2022

  34. [42]

    Motiongpt: Human motion as a foreign language,

    B. Jiang, X. Chen, W. Liu, J. Yu, G. Yu, and T. Chen, “Motiongpt: Human motion as a foreign language,” Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  35. [43]

    Sportscap: Monocular 3d human motion capture and fine-grained understanding in challenging sports videos,

    X. Chen, A. Pang, W. Yang, Y . Ma, L. Xu, and J. Yu, “Sportscap: Monocular 3d human motion capture and fine-grained understanding in challenging sports videos,” International Journal of Computer Vision, p. 2846–2864, Oct 2021

    2021

  36. [44]

    Vibe: Video inference for human body pose and shape estimation,

    M. Kocabas, N. Athanasiou, and M. J. Black, “Vibe: Video inference for human body pose and shape estimation,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , Jun 2020

    2020

  37. [45]

    Loper, N

    M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black, “Smpl,” ACM Transactions on Graphics , p. 1–16, Nov 2015

    2015

  38. [46]

    Neural state machine for character-scene interactions,

    S. Starke, H. Zhang, T. Komura, and J. Saito, “Neural state machine for character-scene interactions,” ACM Transactions on Graphics , p. 1–14, Dec 2019

    2019

  39. [47]

    Generating diverse and natural 3d human motions from text,

    C. Guo, S. Zou, X. Zuo, S. Wang, W. Ji, X. Li, and L. Cheng, “Generating diverse and natural 3d human motions from text,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 5152–5161

    2022

  40. [48]

    Generating animated videos of human activities from natural language descriptions,

    A. S. Lin, L. Wu, R. Corona, K. Tai, Q. Huang, and R. J. Mooney, “Generating animated videos of human activities from natural language descriptions,” Learning, vol. 1, no. 2018, p. 1, 2018

    2018

  41. [49]

    Text2gestures: A transformer-based network for generating emotive body gestures for virtual agents,

    U. Bhattacharya, N. Rewkowski, A. Banerjee, P. Guhan, A. Bera, and D. Manocha, “Text2gestures: A transformer-based network for generating emotive body gestures for virtual agents,” in 2021 IEEE virtual reality and 3D user interfaces (VR) . IEEE, 2021, pp. 1–10

    2021

  42. [50]

    Mocogan: Decomposing motion and content for video generation,

    S. Tulyakov, M.-Y . Liu, X. Yang, and J. Kautz, “Mocogan: Decomposing motion and content for video generation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 1526– 1535

    2018

  43. [51]

    Human motion diffusion as a generative prior,

    Y . Shafir, G. Tevet, R. Kapon, and A. H. Bermano, “Human motion diffusion as a generative prior,” arXiv preprint arXiv:2303.01418, 2023

    2023 arXiv

  44. [52]

    Motionlcm: Real-time controllable motion generation via latent consistency model,

    W. Dai, L.-H. Chen, J. Wang, J. Liu, B. Dai, and Y . Tang, “Motionlcm: Real-time controllable motion generation via latent consistency model,” arXiv preprint arXiv:2404.19759 , 2024

    2024 arXiv

  45. [53]

    Motion mamba: Efficient and long sequence motion generation,

    Z. Zhang, A. Liu, I. Reid, R. Hartley, B. Zhuang, and H. Tang, “Motion mamba: Efficient and long sequence motion generation,” in European Conference on Computer Vision . Springer, 2025, pp. 265–282

    2025

  46. [54]

    The kit motion-language dataset,

    M. Plappert, C. Mandery, and T. Asfour, “The kit motion-language dataset,” Big data, vol. 4, no. 4, pp. 236–252, 2016

    2016

  47. [55]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium,

    M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” Advances in neural information processing systems , vol. 30, 2017

    2017

  48. [56]

    Pytorch: An imperative style, high-performance deep learning library,

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al. , “Pytorch: An imperative style, high-performance deep learning library,” in Proc. Adv. Neural Inf. Process. Syst. , 2019. Zheng Qin received the B.S. degree in...

    2019

  49. [2015]

    His research interests include 3D/2D digital humans, dialogue, and multimodal interaction

    He is currently an Algorithm Engineer at Ant Group, responsible for digital human algorithms. His research interests include 3D/2D digital humans, dialogue, and multimodal interaction. Sanping Zhou (Member, IEEE) received Ph.D. de- gree in control science and engineering from ...

    2020

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.