Pith. sign in

REVIEW 3 major objections 5 minor 38 references

DiM-Gestor: Co-Speech Gesture Generation with Adaptive Layer Normalization Mamba-2

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a Mamba-2-based diffusion model with adaptive layer normalization generates co-speech gestures of Transformer-level quality while cutting memory by about 2.4 times and inference time by a factor of 2 to 4, validated…

desk verdict A useful new Chinese gesture dataset and a plausible Mamba-2 diffusion architecture, but the headline quality and efficiency claims rest on uncontrolled comparisons. read the letter →

arxiv 2411.16729 v1 pith:4ZICDW3D submitted 2024-11-23 cs.SD cs.AIcs.GRcs.HCcs.MMeess.AS

Fan Zhang , Siyuan Zhao , Naye Ji , Zhaohan Wang , Jingmei Wu , Fuxing Gao , Zhenqing Ye , Leyao Yan
show 7 more authors
Lanxin Dai Weidong Geng Xin Lyu Bozuo Zhao Dingguo Yu Hui Du Bin Hu
This is my paper · ORCID
classification cs.SDcs.AIcs.GRcs.HCcs.MMeess.AS
keywords co-speechgesturegenerationMamba-2adaptivelayernormalizationdiffusionmodelChinesegesturesdatasetspeech-drivensynthesisvirtualhumansefficientinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the Mamba-2 state-space architecture, combined with adaptive layer normalization and a diffusion denoiser, can generate full-body co-speech gestures with quality comparable to Transformer-based systems while using roughly 2.4 times less memory and running 2 to 4 times faster at inference. To support this, the authors introduce the CCG dataset, 15.97 hours of 3D full-body skeleton motion from professional Chinese TV broadcasters across six styles and five scenarios, and evaluate their model, DiM-Gestor, on it. The best configuration reaches the lowest Fréchet Gesture Distance among all compared synthetic models and the highest user-rated style appropriateness, while matching the top Transformer baseline on human-likeness and appropriateness. The practical interest is that linear-complexity sequence models could make gesture generation feasible for real-time or long-duration virtual-human applications.

What carries the argument

The load-bearing mechanism is the Structured State Space Duality formulation of Mamba-2, where a 1-semiseparable Structured Masked Attention matrix replaces softmax attention, reducing the state size from linear to constant and the per-layer cost from quadratic to linear in sequence length. Around this core, the paper wraps adaptive layer normalization, which regresses the scale and shift applied to every token from the extracted speech features, and a denoising diffusion probabilistic model that learns to reverse a noising process on gesture sequences. Together, the fuzzy feature extractor supplies the condition, the AdaLN Mamba-2 blocks denoise, and the DDPM provides diverse outputs.

What would settle it

Retrain the Transformer baselines, such as PG-12blocks, on the CCG training split and run all models on the same GPU with identical sequence lengths; if DiM-Gestor's FGD, BeatAlign, and 2-4x inference advantages largely disappear under these controlled conditions, the central efficiency claim would fail. A simpler check is to measure peak memory usage of both models on the same hardware for the same 20-second input to verify the reported 2.4 times reduction.

Watch

Extended reading notes

Core claim

The central claim is that replacing the Transformer backbone in a diffusion-based co-speech gesture model with Mamba-2, and conditioning it through adaptive layer normalization, preserves gesture quality while changing the scaling of inference cost from quadratic to linear in sequence length. DiM-Gestor's fuzzy feature extractor uses a Chinese pre-trained speech model plus a Mamba-2 global extractor to produce a unified latent condition, and the AdaLN Mamba-2 blocks regress per-token scale and shift parameters from that condition to guide denoising. In the authors' experiments, DiM-Gestor reaches an FGD of 17.716 on the feature space, versus 100.899 for the 12-block Persona-Gestor Transformer, with 535M parameters against 1.2B, while maintaining near-Transformer-level subjective scores. The paper also releases the CCG dataset, arguing that it fills a gap for formal, structured Chinese speech that existing English-dominated datasets do not cover.

Load-bearing premise

The comparisons are fair even though the baseline models were trained on English speech data and evaluated on a new Chinese dataset, and even though the efficiency figures compare the baseline on an A100 GPU with DiM-Gestor on a 4090 GPU.

Editorial extensions

If this is right

  • Longer gesture sequences become much cheaper: at 100 seconds, DiM-Gestor infers in 23.27 seconds while the 12-block Persona-Gestor baseline takes 99.21 seconds in the paper's measurements.
  • Deployment on consumer hardware becomes easier: the model uses 535M parameters versus 1.2B for the strongest Transformer baseline, with roughly 2.4 times lower memory usage.
  • The CCG dataset gives the community a 15.97-hour, six-style, five-scenario Chinese corpus with full-body 3D skeleton and synchronized speech, enabling training and benchmarking on formal broadcast-style Chinese.
  • Style appropriateness is the configuration's strongest edge, suggesting that speaker- and style-related conditioning can come directly from raw audio without explicit style labels.
  • The linear-complexity backbone makes the approach a plausible candidate for real-time or interactive virtual-human systems that process long utterances.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the Transformer baselines were retrained on the CCG dataset and evaluated on identical hardware, the reported FGD and speed gaps could shrink; the paper's comparisons mix English-trained baselines with a Chinese-trained model and use an A100 GPU for the baseline versus a 4090 GPU for DiM-Gestor.
  • The linear-complexity advantage should grow with sequence length, so testing beyond 100 seconds would reveal whether DiM-Gestor's inference time stays sublinear relative to Transformer baselines.
  • The style-appropriateness lead may partly reflect the dataset's own style distribution; a cross-dataset transfer test would separate model capability from dataset bias.
  • Combining the model with the accelerated diffusion sampler the paper mentions could push inference into interactive real-time range, likely at some cost in output quality.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces DiM-Gestor, an end-to-end co-speech gesture generation model based on a Mamba-2 architecture augmented with Adaptive Layer Normalization (AdaLN) and a diffusion backbone. The model consists of a Mamba-2-based fuzzy feature extractor and an AdaLN Mamba-2 mapping module, and is trained and evaluated on a newly released Chinese Co-Speech Gestures (CCG) dataset comprising 15.97 hours of full-body motion from professional broadcasters. The authors report competitive subjective quality, a lower Frechet Gesture Distance than all compared baselines, and claim roughly 2.4x memory reduction and 2-4x faster inference relative to a Transformer-based Persona-Gestor model.

Significance. If the efficiency and quality claims were substantiated, DiM-Gestor would be a useful step toward linear-time, memory-efficient gesture generation for long sequences, and the CCG dataset would fill a gap for Chinese formal speech domains. The paper ships a new dataset and promises code release, and the ablation comparisons of Mamba-1/Mamba-2 and fuzzy extractor variants are systematic. However, the central comparative evidence is currently compromised by uncontrolled training-domain and hardware differences, so the significance of the architectural contribution is not yet established.

major comments (3)
  1. [Section V.D, Table III] The objective FGD comparison is invalid as evidence of architectural superiority because all baselines (DSG+, GDC, LDA, Taming, PG-6blocks, PG-12blocks) were trained on English speech datasets (Trinity, ZEGGS, BEAT), while DiM-Gestor was trained and tested on the Chinese CCG dataset, as stated in Section V.D. The much lower FGD (17.716 vs. 100.899 for PG-12blocks) is expected from the language/domain mismatch alone. To support the claim of competitive results, all baselines must be retrained on CCG using the same training protocol and evaluated on the same test split.
  2. [Section VI, Tables IV and VI.A] The memory and inference-speed claims are not controlled. The 'approximately 2.4 times' memory reduction is computed from parameter counts (1.2B vs. 535M), not from measured peak GPU memory consumption, which also depends on activations, sequence length, and batch size. The inference-time comparison is confounded by hardware: Section V.B states that DiM-Gestor was tested on an NVIDIA GeForce 4090, while PG-12blocks was benchmarked on an A100. The authors should measure peak memory and latency for both models on the same GPU, with matched sequence lengths and batch sizes.
  3. [Section V.D.1, Table II] The subjective evaluation suffers from the same domain mismatch. The large advantage in style-appropriateness (1.30 vs. 0.664 for PG-12blocks) likely reflects the English training of the baselines rather than a property of the Mamba-2 architecture. The statistical results in Figure 10 show no significant difference between DiM m2 s m2 and PG-12blocks for human-likeness and appropriateness, so at most the paper can claim parity on those metrics. Retraining the baselines on CCG is required before any perceptual superiority of DiM-Gestor can be asserted.
minor comments (5)
  1. [Section III and Section IV.B] The notation is inconsistent across the paper: Eq. (1) defines X as audio and Y^0 as gesture, but Eq. (4) uses x0 and xn for original and noised gesture data, and Eq. (5) writes X0 for the gesture sequence. Please unify the notation throughout.
  2. [Algorithm 1] In Algorithm 1, the variable X_{1:T} is used on the right-hand side before it is defined; the input line defines G_{1:T}, so the update should likely read G_{1:T} or X_{1:T}^{(0)}. Also, the second update line has unbalanced parentheses, making the formula ambiguous.
  3. [Section V.D] There is a typo in 'ANOV A' in Section V.D.1; it should be 'ANOVA'. Also, the title of Section VII spells 'DISSCUSTION' instead of 'DISCUSSION'.
  4. [Reference [27]] Reference [27] attributes a 1908 paper to 'Paul, L.', but the cited work is by Paul Langevin on Brownian motion; the reference should be corrected to the proper author and title.
  5. [Section V.D.1] The definition of style-appropriateness as 'the similarity between the generated gestures and their original human counterparts' appears to overlap with the definition of human-likeness. Please clarify the distinction or cite the original source of these evaluation criteria.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation; headline quality/efficiency claims rest on uncontrolled empirical comparisons, not on circular reasoning.

full rationale

The claimed derivation chain—Mamba-2 fuzzy extractor, AdaLN Mamba-2 blocks, and DDPM training—contains no step in which a predicted quantity is defined in terms of its own input, no fitted parameter is relabeled as a prediction, and no load-bearing uniqueness theorem is imported from the authors' prior work. The design borrows the fuzzy-extractor and AdaLN concepts from the authors' Persona-Gestor paper via self-citation, but those are architectural components, not the target result, and the core linear-complexity mechanism is external Mamba-2. The comparison against PG is empirical, and the FGD/BeatAlign metrics are standard external measures. The efficiency claim of approximately 2.4x memory reduction and 2-4x inference speedup is based on parameter counts (535M vs 1.2B) and measured inference times, not on a circular equation; its validity is weakened by testing English-trained baselines on the Chinese CCG test set and by using different GPUs (4090 vs A100), but these are uncontrolled-comparison and external-validity concerns, not circularity. No self-citation carries a necessity or uniqueness burden, and no result reduces by construction to its inputs. Therefore no significant circularity is present.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claims rest on the validity of Mamba-2's efficiency properties, the transferability of Chinese HuBERT features to gesture synthesis, and the adequacy of the user study and objective metrics. No new physical or mathematical entities are introduced. The hand-selected hyperparameters listed above affect parameter counts, inference times, and quality, so they are the main free parameters the reader should track.

free parameters (5)
  • Diffusion noise schedule = beta_1 = 1e-4 to beta_N = 8e-2, N = 1000, linear schedule
    Hand-chosen variance schedule that controls denoising dynamics; directly affects generation quality and diversity.
  • Mamba-2 block configuration = SSM state expansion 256, local convolution width 4, block expansion 2, hidden size 1280
    Hand-selected architecture sizes; these determine parameter count and the efficiency comparison with PG.
  • Number of AdaLN Mamba-2 blocks = 6 blocks (vs 12 blocks for PG-12blocks)
    Chosen to balance quality and efficiency; the headline memory and speed ratios depend on this asymmetric comparison.
  • Downsampling Conv1D kernel size = 201
    Carried over from Persona-Gestor; sets the temporal receptive field for the unified latent representation.
  • Gesture encoder and decoder kernel sizes = Encoder kernel 3, decoder kernel 1
    Chosen by the authors based on observed jitter; these design choices affect reconstruction quality.
assumptions (4)
  • standard math Mamba-2's SSD formulation with 1-semiseparable structured masked attention achieves linear complexity and matches Transformer expressivity for this task.
    Invoked in Section IV.A.2 as the theoretical basis for efficiency gains; the paper cites Dao and Gu rather than proving it.
  • domain assumption The chinese-wav2vec2-base HuBERT model captures Chinese speech features sufficient for gesture semantics and style.
    Used in the fuzzy feature extractor (Section IV.A.1); no evidence is provided that this specific pretrained model is optimal for gesture conditioning.
  • domain assumption The gesture representation from Alexanderson et al. (exponential maps, root velocities, 20 fps, 20-second clips) is sufficient for full-body co-speech gesture synthesis.
    Adopted for data processing in Section V.A.3 without task-specific validation.
  • domain assumption Subjective ratings from 30 Chinese volunteers and FGD/BeatAlign objective metrics reflect meaningful gesture quality differences.
    Stated in Section V.D; the paper relies on these measures for its main comparative conclusions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DiM-Gestor: Co-Speech Gesture Generation with Adaptive Layer Normalization Mamba-2." pith.science (2026). https://pith.science/paper/4ZICDW3D

@misc{pith2026241116729,
  author       = {Pith},
  title        = {Pith review of: DiM-Gestor: Co-Speech Gesture Generation with Adaptive Layer Normalization Mamba-2},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4ZICDW3D}},
  note         = {Machine review of arXiv:2411.16729}
}
read the original abstract

Speech-driven gesture generation using transformer-based generative models represents a rapidly advancing area within virtual human creation. However, existing models face significant challenges due to their quadratic time and space complexities, limiting scalability and efficiency. To address these limitations, we introduce DiM-Gestor, an innovative end-to-end generative model leveraging the Mamba-2 architecture. DiM-Gestor features a dual-component framework: (1) a fuzzy feature extractor and (2) a speech-to-gesture mapping module, both built on the Mamba-2. The fuzzy feature extractor, integrated with a Chinese Pre-trained Model and Mamba-2, autonomously extracts implicit, continuous speech features. These features are synthesized into a unified latent representation and then processed by the speech-to-gesture mapping module. This module employs an Adaptive Layer Normalization (AdaLN)-enhanced Mamba-2 mechanism to uniformly apply transformations across all sequence tokens. This enables precise modeling of the nuanced interplay between speech features and gesture dynamics. We utilize a diffusion model to train and infer diverse gesture outputs. Extensive subjective and objective evaluations conducted on the newly released Chinese Co-Speech Gestures dataset corroborate the efficacy of our proposed model. Compared with Transformer-based architecture, the assessments reveal that our approach delivers competitive results and significantly reduces memory usage, approximately 2.4 times, and enhances inference speeds by 2 to 4 times. Additionally, we released the CCG dataset, a Chinese Co-Speech Gestures dataset, comprising 15.97 hours (six styles across five scenarios) of 3D full-body skeleton gesture motion performed by professional Chinese TV broadcasters.

Figures

Figures reproduced from arXiv: 2411.16729 by the authors.

Figure 1
Figure 1. We propose DiM-Gestor, an end-to-end AdaLN Mamba-2 and diffusion-based architecture for co-speech gesture generation. In addition, we present the comprehensive Chinese Co-Speech Gestures (CCG) dataset, comprising 15.97 hours of full-body gesture motion performed by professional Chinese TV broadcasters. This dataset encompasses six distinct styles across five scenarios. speech data, it primarily features spontaneous … view at source ↗
Figure 2
Figure 2. The architecture of DiM-Gestor incorporates a Mamba-2 fuzzy feature extractor and an Adaptive Layer Normalization [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The Structured Masked Attention. Compared to standard (self-)attention mechanisms, the Structured State Space (SSD) model with 1-semi separable Structured Masked Attention (SMA) introduces significant optimizations. SSD eliminates the softmax normalization step, effectively reducing the requisite state size of the model from a linear to a constant scale, thereby enhancing com￾putational efficiency from quadratic to … view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: An overview of the recorded Chinese TV broadcasters [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 7
Figure 7. Figure 7: The male broadcaster energetically delivered the excit [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: A male broadcaster is hosting a happy show (Left: GT; [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 6
Figure 6. Figure 6: The female broadcaster delivering a recitation on the [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 9
Figure 9. Figure 9: The average metric rating for each approach in comparative experiments. [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: Heatmaps of the mean ratings of user studies significant differences across all methods for each metric. Asterisks [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 11
Figure 11. Figure 11: The inference time of various gesture lengths across [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 28 canonical work pages

  1. [1]

    DiffuseStyleGesture: Stylized audio-driven co-speech gesture generation with diffusion models,

    S. Yang, Z. Wu, M. Li, Z. Zhang, L. Hao, W. Bao, M. Cheng, and L. Xiao, “DiffuseStyleGesture: Stylized audio-driven co-speech gesture generation with diffusion models,” arXiv preprint arXiv:2305.04919 , 2023

    arXiv 2023

  2. [2]

    The diffusestylegesture+ entry to the genea challenge 2023,

    S. Yang, H. Xue, Z. Zhang, M. Li, Z. Wu, X. Wu, S. Xu, and Z. Dai, “The diffusestylegesture+ entry to the genea challenge 2023,” in Proceedings of the 25th International Conference on Multimodal Interaction, 2023, pp. 779–785

    work page 2023

  3. [3]

    Gesturediffuclip: Gesture diffusion model with clip latents,

    T. Ao, Z. Zhang, and L. Liu, “Gesturediffuclip: Gesture diffusion model with clip latents,” ACM Transactions on Graphics (TOG) , vol. 42, no. 4, pp. 1–18, 2023

    work page 2023

  4. [4]

    Listen, denoise, action! audio-driven motion synthesis with diffusion models,

    S. Alexanderson, R. Nagy, J. Beskow, and G. E. Henter, “Listen, denoise, action! audio-driven motion synthesis with diffusion models,” ACM Transactions on Graphics (TOG) , vol. 42, no. 4, pp. 1–20, 2023

    2023

  5. [5]

    Speech-driven personalized gesture synthetics: Harnessing automatic fuzzy feature inference,

    F. Zhang, Z. Wang, X. Lyu, S. Zhao, M. Li, W. Geng, N. Ji, H. Du, F. Gao, and H. Wu, “Speech-driven personalized gesture synthetics: Harnessing automatic fuzzy feature inference,” IEEE Transactions on Visualization and Computer Graphics , 2024, publisher: IEEE. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/10508094/

    work page arXiv 2024

  6. [6]

    Scalable diffusion models with transformers,

    W. Peebles and S. Xie, “Scalable diffusion models with transformers,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 4195–4205

    2023

  7. [7]

    Mamba: Linear-time sequence modeling with selective state spaces,

    A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752 , 2023

    arXiv 2023

  8. [8]

    Vision mamba: Efficient visual representation learning with bidirectional state space model,

    L. Zhu, B. Liao, Q. Zhang, X. Wang, W. Liu, and X. Wang, “Vision mamba: Efficient visual representation learning with bidirectional state space model,” arXiv preprint arXiv:2401.09417 , 2024

    arXiv 2024

Show all 38 references
  1. [9]

    U-mamba: Enhancing long-range dependency for biomedical image segmentation,

    J. Ma, F. Li, and B. Wang, “U-mamba: Enhancing long-range dependency for biomedical image segmentation,” arXiv preprint arXiv:2401.04722, 2024

    2024 arXiv

  2. [10]

    Videomamba: State space model for efficient video understanding,

    K. Li, X. Li, Y . Wang, Y . He, Y . Wang, L. Wang, and Y . Qiao, “Videomamba: State space model for efficient video understanding,” Springer, pp. 237–255, 2025

    2025

  3. [11]

    Segmamba: Long-range sequential modeling mamba for 3d medical image segmentation,

    Z. Xing, T. Ye, Y . Yang, G. Liu, and L. Zhu, “Segmamba: Long-range sequential modeling mamba for 3d medical image segmentation,” in International Conference on Medical Image Computing and Computer- Assisted Intervention . Springer, 2024, pp. 578–588

    2024

  4. [12]

    Graph mamba: Towards learning on graphs with state space models,

    A. Behrouz and F. Hashemi, “Graph mamba: Towards learning on graphs with state space models,” Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pp. 119–130, 2024

    2024

  5. [13]

    Graph-mamba: Towards long-range graph sequence modeling with selective state spaces,

    C. Wang, O. Tsepa, J. Ma, and B. Wang, “Graph-mamba: Towards long-range graph sequence modeling with selective state spaces,” arXiv preprint arXiv:2402.00789, 2024

    2024 arXiv

  6. [14]

    Transformers are SSMs: Generalized models and efficient algorithms through structured state space duality,

    T. Dao and A. Gu, “Transformers are SSMs: Generalized models and efficient algorithms through structured state space duality,” arXiv preprint arXiv:2405.21060 , 2024. [Online]. Available: https: //arxiv.org/abs/2405.21060

    2024 arXiv

  7. [15]

    Investigating the use of recurrent motion modelling for speech gesture generation,

    Y . Ferstl and R. McDonnell, “Investigating the use of recurrent motion modelling for speech gesture generation,” in Proc. of the 18th Interna- tional Conf. on Intelligent Virtual Agents , 2018, pp. 93–98

    2018

  8. [16]

    Zeroeggs: Zero-shot example-based gesture generation from speech,

    S. Ghorbani, Y . Ferstl, D. Holden, N. F. Troje, and M.-A. Carbonneau, “Zeroeggs: Zero-shot example-based gesture generation from speech,” in Computer Graphics F orum, vol. 42, no. 1. Wiley Online Library, 2023, pp. 206–216

    2023

  9. [17]

    Beat: A large-scale semantic and emotional multi-modal dataset for conversational gestures synthesis,

    H. Liu, Z. Zhu, N. Iwamoto, Y . Peng, Z. Li, Y . Zhou, E. Bozkurt, and B. Zheng, “Beat: A large-scale semantic and emotional multi-modal dataset for conversational gestures synthesis,” in COMPUTER VISION, ECCV 2022, PT VII , vol. 13667, 2022, pp. 612–630

    2022

  10. [18]

    Talking with hands 16.2 m: A large-scale dataset of synchronized body-finger motion and audio for conversational motion analysis and synthesis,

    Lee Gilwoo, Deng Zhiwei, Ma Shugao, Shiratori Takaaki, Srinivasa Siddhartha S., and Sheikh Yaser, “Talking with hands 16.2 m: A large-scale dataset of synchronized body-finger motion and audio for conversational motion analysis and synthesis,” in Proceedings of the IEEE/CVF In...

    2019

  11. [19]

    Diffmotion: Speech-driven gesture synthesis using denoising diffusion model,

    F. Zhang, N. Ji, F. Gao, and Y . Li, “Diffmotion: Speech-driven gesture synthesis using denoising diffusion model,” in MultiMedia Modeling: 29th International Conf., MMM 2023, Bergen, Norway, January 9–12, 2023, Proc., Part I . Springer, 2023, pp. 231–242

    2023

  12. [20]

    Cross-modal quantization for co-speech gesture generation,

    Z. Wang, W. Zhang, L. Ye, D. Zeng, and T. Mei, “Cross-modal quantization for co-speech gesture generation,” IEEE Transactions on Multimedia, 2024

    2024

  13. [21]

    Adversarial gesture generation with realistic gesture phasing,

    Y . Ferstl, M. Neff, and R. McDonnell, “Adversarial gesture generation with realistic gesture phasing,” Computers & Graphics, vol. 89, pp. 117– 130, 2020

    2020

  14. [22]

    Robots learn social skills: End-to-end learning of co- speech gesture generation for humanoid robots,

    Yoon Youngwoo, Ko Woo-Ri, Jang Minsu, Lee Jaeyeon, Kim Jaehong, and Lee Geehyuk, “Robots learn social skills: End-to-end learning of co- speech gesture generation for humanoid robots,” in 2019 International Conference on Robotics and Automation (icra) , 2019, pp. 4303–4309, WO...

    2019

  15. [23]

    chinese speech pretrain,

    P. Guo and S. Liu, “chinese speech pretrain,” 2022. [Online]. Available: https://github.com/TencentGameMate/chinese speech pretrain

    2022

  16. [24]

    Hand and mind: What gestures reveal about thought,

    D. McNeill, “Hand and mind: What gestures reveal about thought,” Advances in Visual Semiotics , p. 351, 1992

    1992

  17. [25]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in Neural Information Processing Systems , vol. 33, pp. 6840– 6851, 2020

    2020

  18. [26]

    Deep unsupervised learning using nonequilibrium thermodynamics,

    J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” in International Conf. on Machine Learning . PMLR, 2015, pp. 2256– 2265

    2015

  19. [27]

    sur la th ´eorie du mouvement brownien,

    L. Paul, “sur la th ´eorie du mouvement brownien,” C. R. Acad. Sci. , vol. 65, no. 11, pp. 146,530–533, 1908, publisher: American Association of Physics Teachers

    1908

  20. [28]

    Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting,

    K. Rasul, C. Seward, I. Schuster, and R. V ollgraf, “Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting,” in International Conf. on Machine Learning , 2021, pp. 8857–8868

    2021

  21. [29]

    Style-Controllable Speech-Driven Gesture Synthesis Using Normal- ising Flows,

    S. Alexanderson, G. E. Henter, T. Kucherenko, and J. Beskow, “Style-Controllable Speech-Driven Gesture Synthesis Using Normal- ising Flows,” in Computer Graphics F orum , vol. 39. Wiley Online Library, 2020, pp. 487–496, issue: 2

    2020

  22. [30]

    Practical parameterization of rotations using the expo- nential map,

    F. S. Grassia, “Practical parameterization of rotations using the expo- nential map,” Journal of graphics tools , vol. 3, no. 3, pp. 29–48, 1998

    1998

  23. [31]

    Taming diffusion models for audio-driven co-speech gesture generation,

    L. Zhu, X. Liu, X. Liu, R. Qian, Z. Liu, and L. Yu, “Taming diffusion models for audio-driven co-speech gesture generation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 10 544–10 553

    2023

  24. [32]

    The genea challenge 2023: A large-scale evaluation of 12 gesture generation models in monadic and dyadic settings,

    T. Kucherenko, R. Nagy, Y . Yoon, J. Woo, T. Nikolov, M. Tsakov, and G. E. Henter, “The genea challenge 2023: A large-scale evaluation of 12 gesture generation models in monadic and dyadic settings,” Proceedings of the 25th International Conference on Multimodal Interaction , ...

    2023

  25. [33]

    To rate or not to rate: Investigating evaluation methods for generated co-speech gestures,

    P. Wolfert, J. M. Girard, T. Kucherenko, and T. Belpaeme, “To rate or not to rate: Investigating evaluation methods for generated co-speech gestures,” in Proceedings of the 2021 International Conference on Multimodal Interaction . ACM, 2021-10-18, pp. 494–502

    2021

  26. [34]

    Speech gesture generation from the trimodal context of text, audio, and speaker identity,

    Y . Yoon, B. Cha, J.-H. Lee, M. Jang, J. Lee, J. Kim, and G. Lee, “Speech gesture generation from the trimodal context of text, audio, and speaker identity,” ACM Transactions on Graphics (TOG) , vol. 39, no. 6, pp. 1–16, 2020

    2020

  27. [35]

    Ai choreographer: Music conditioned 3d dance generation with aist++,

    R. Li, S. Yang, D. A. Ross, and A. Kanazawa, “Ai choreographer: Music conditioned 3d dance generation with aist++,” in Proc. of the IEEE/CVF International Conf. on Computer Vision , 2021, pp. 13 401–13 412

    2021

  28. [36]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium,

    M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” Advances in neural information processing systems , vol. 30, 2017

    2017

  29. [37]

    Evaluating gesture-generation in a large-scale open challenge: The genea challenge 2022,

    T. Kucherenko, P. Wolfert, Y . Yoon, C. Viegas, T. Nikolov, M. Tsakov, and G. E. Henter, “Evaluating gesture-generation in a large-scale open challenge: The genea challenge 2022,” arXiv preprint arXiv:2303.08737, 2023

    2022 arXiv

  30. [38]

    Simplifying, stabilizing and scaling continuous- time consistency models

    C. Lu and Y . Song, “Simplifying, stabilizing and scaling continuous- time consistency models.” [Online]. Available: http://arxiv.org/abs/2410. 11081 13

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.