Pith. sign in

REVIEW 3 major objections 5 minor 228 references

Human Motion Video Generation: A Survey

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A five-phase pipeline organizes human motion video generation, with LLMs as motion planners.

desk verdict Useful survey with a valuable taxonomy; its benchmark is the weak link and should be revised or dropped. read the letter →

arxiv 2509.03883 v1 pith:4HP6NQRJ submitted 2025-09-04 cs.CV cs.MM

Haiwei Xue , Xiangyang Luo , Zhanghao Hu , Xin Zhang , Xunzhi Xiang , Yuqin Dai , Jianzhuang Liu , Zhensong Zhang
show 7 more authors
Minglei Li Jian Yang Fei Ma Zhiyong Wu Changpeng Yang Zonghong Dai Fei Richard Yu
This is my paper
classification cs.CVcs.MM
keywords humanmotionvideogenerationsurveyfive-phasepipelinelargelanguagemodelsplanningdiffusionpose-guideddancetalkingheadsynthesis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to give the field a shared structure by decomposing human motion video generation into five phases: input, motion planning, motion video generation, refinement, and output. It organizes more than two hundred papers into vision-, text-, and audio-driven categories and claims to be the first survey to discuss large language models as motion planners. If this framing is accepted, researchers gain a unified map for comparing methods, a taxonomy of diffusion-model architectures, and a new quantitative baseline for pose-guided dance video generation. The survey also collects 64 human-centered datasets and identifies common failure modes across current methods.

What carries the argument

The central object is the five-phase generation pipeline (input, motion planning, motion video generation, refinement, output), used as the organizing scaffold for the whole survey. The paper's key analytic tools are a three-way modality classification (audio-, text-, and vision-driven) and a taxonomy of diffusion-based frameworks, including three input schemes for the main diffusion branch and five attention-fusion arrangements. For motion planning, the distinguishing mechanism is the LLM planner, which either retrieves motion descriptions from a database or projects them into a generative model's latent space.

What would settle it

Rerun the nine pose-guided methods on a larger and independently annotated dance test set using official model weights wherever they exist; if the top rankings across SSIM, PSNR, LPIPS, FID, and FID-VID change materially, the survey's comparative conclusions about MagicAnimate and UniAnimate would not generalize.

Watch

Extended reading notes

Core claim

The paper claims that existing surveys cover individual subtasks but miss the whole generative process, so it proposes a five-phase pipeline spanning every step from input to deployed output. Within this pipeline, it distinguishes two motion-planning strategies: implicit feature mapping and LLM-based planning, with the latter shown in two forms—retrieval of fine-grained motion descriptions and direct projection into a motion latent space. For the generation phase, it classifies diffusion-based pose-driven methods by what feeds the main diffusion branch (pure noise, a reference image, or guided conditions) and by attention-fusion variants. It also contributes a comparative benchmark of nine o

Load-bearing premise

The comparative benchmark assumes that an unofficial reimplementation of Animate Anyone faithfully represents the original model and that ten TikTok test videos are enough to rank the nine methods.

Editorial extensions

If this is right

  • Any new method can be located within the five-phase pipeline and compared against its neighbors, giving the field a common coordinate system.
  • LLM-based motion planning becomes a distinct research branch, with retrieval-style and generative-style planners as two identifiable architectures.
  • The nine-method benchmark gives practitioners a quantitative starting point: MagicAnimate for structural fidelity, UniAnimate for perceptual and temporal quality, and Animate Anyone for pixel-level closeness to ground truth.
  • The catalog of common failure modes—hand blur, facial distortion, identity or background inconsistency, limb dislocation, and low resolution—serves as a concrete checklist for evaluating new methods.
  • The 64-dataset collection supplies a fast route to training data, including resolution, duration, and task-support annotations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the five-phase framing takes hold, motion planning may be evaluated separately from video rendering, leading to benchmarks that score planner quality (e.g., retrieval rank, instruction adherence) independently of pixel-level metrics.
  • The LLM-planner direction suggests a future separation of semantic motion specification from appearance rendering, which would let planners and generators be upgraded independently.
  • The benchmark could naturally extend to audio-driven and text-driven methods under the same metrics, enabling cross-modal comparison that the current pose-guided-only setup does not provide.
  • The survey's deliberate exclusion of 3D skeleton motion, NeRF, and 3D Gaussian splatting leaves room for a complementary survey that bridges the 2D video and 3D representation worlds.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This manuscript surveys human motion video generation, organizing the field into five phases (input, motion planning, motion video generation, refinement, output) and three driving modalities (vision, text, audio). It reviews over 200 papers, provides a table of 64 datasets, summarizes metrics, and reports a new comparative benchmark of nine pose-guided dance-video generation methods evaluated on 10 TikTok test videos with six metrics. The paper claims to be the first survey to discuss LLM-based motion planning and to provide a complete pipeline-oriented taxonomy.

Significance. If the survey's coverage and taxonomy hold, it would be a useful reference: the five-phase decomposition gives a unified vocabulary, the dataset table is extensive, and the LLM-planning discussion addresses an emerging topic. The paper ships an open repository of reviewed models. The independent benchmark is a potentially valuable practitioner resource, but its reliability is currently limited by the fragile experimental setup; it should not be used to rank methods until variance and faithfulness of the implementations are addressed.

major comments (3)
  1. [Section VI.B, Table VII, Fig. 12] The quantitative ranking relies on an unofficial reimplementation of Animate Anyone (footnote 1) and a 10-video test set. No error bars or significance tests are provided. For example, the gap between the best SSIM (MagicAnimate, 0.7558) and the second-best (UniAnimate, 0.7436) is 0.0122, which may be within run-to-run variance on 10 videos. This makes Table VII's rankings unsupported as stated. Please report variance/confidence intervals and per-video breakdowns, verify the unofficial implementation against the original (or relabel the column as the reimplementation), and either enlarge the test set or explicitly describe the benchmark as illustrative rather than comparative.
  2. [Table VII, Fig. 12, Section II scope note] Table VII lists 'StableAnimation [208]' while reference [208] is StableAnimator, and Fig. 12's caption says 'five open-source methods' although the text lists nine. These inconsistencies indicate the experimental presentation has not been carefully checked. Moreover, StableAnimator (arXiv:2411.17697) and Animate-X (arXiv:2410.10306) have arXiv dates after the stated August 30, 2024 cutoff, contradicting the paper's note that all statistics are current as of that date. Please correct the naming and reconcile the benchmark with the stated cutoff, or revise the scope statement.
  3. [Fig. 2 caption and Tables V-VI] The paper states it does not cover video generation through NeRF or 3DGS rendering, yet Table V includes SD-NeRF [138] (a NeRF-based talking-head method) and Table VI includes TalkingGaussian [142] and GaussianTalker [143] (3D Gaussian-based). This creates an internal inconsistency in the taxonomy. Either clarify the inclusion criteria for these 3D-representation methods so they are consistent with the stated scope, or remove them from the tables.
minor comments (5)
  1. [Author affiliations] Typo: 'Artificial Intelligence Innovation and Incubation (Al’) Institute' should be 'AI Institute'.
  2. [Fig. 12 caption] The caption says 'five open-source methods' but the text and Table VII describe nine methods; please align the caption with the actual content.
  3. [Tables III and V] The real-time support column is labeled 'Estimated FPS based on NVIDIA RTX 4090 GPU' but the basis of the estimate is not stated. Please specify whether these are reported by the original papers or measured here.
  4. [Fig. 11] The legend contains garbled/overlapping text, e.g., 'text Consider (ignore) the temporal dim.(text)'. Please redraw the figure for legibility.
  5. [Reference [110]] The Mamba reference (Gu and Dao) lacks a full bibliographic venue/year entry; please complete it. Also change 'StableAnimation' to 'StableAnimator' in Table VII.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the survey taxonomy and benchmark are independent; self-citations are not load-bearing.

full rationale

This paper is a literature survey, not a derivation chain. Its five-phase pipeline (input, motion planning, motion video generation, refinement, output) is an organizing taxonomy proposed by the authors; it is not defined in terms of a quantity the paper later predicts, and the survey does not attempt to derive the taxonomy from first principles. The novelty claims (first comprehensive five-phase survey; first survey discussing LLMs for motion planning) are literature claims supported by comparison with prior surveys [10]–[14] and by the cited LLM-based motion planning works [17]–[19]; they do not reduce to the paper's own definitions. The comparative benchmark in Section VI.B is an external empirical evaluation on the TikTok test set: the reported L1, PSNR, SSIM, LPIPS, FID, and FID-VID values are measurements of nine methods, not fitted parameters renamed as predictions. Including methods co-authored by members of this survey team (e.g., DreaMoving [97], MotionFollower [99], PoseAnimate [109], StableAnimator [208]) is normal citation practice, and none of those citations is load-bearing for the survey's central organizational or novelty claims. The concerns raised about the unofficial Animate Anyone reimplementation and the small 10-video test set are validity/reliability issues for the benchmark, not circularity: the benchmark does not assume the conclusion it reports. Accordingly, no circular step is present.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The central claim of this paper is organizational, not a quantitative derivation. It has no fitted free parameters or invented entities. Its main premises are the classification rule for driving modalities and the exclusion of NeRF/3DGS rendering, both of which are explicit scope choices.

assumptions (2)
  • domain assumption Any method that includes audio input is classified as audio-driven, even if it also uses text or vision (Section I).
    This classification rule shapes the entire organization of the survey and determines which categories methods fall into. It is a reasonable editorial choice but is not derived from external constraints.
  • domain assumption The survey excludes 3D reconstruction from images and NeRF/3DGS rendering, limiting coverage to direct video generation methods (Section I).
    This scoping decision defines the survey's boundaries and affects the claim of comprehensiveness.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Human Motion Video Generation: A Survey." pith.science (2026). https://pith.science/paper/4HP6NQRJ

@misc{pith2026250903883,
  author       = {Pith},
  title        = {Pith review of: Human Motion Video Generation: A Survey},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4HP6NQRJ}},
  note         = {Machine review of arXiv:2509.03883}
}
read the original abstract

Human motion video generation has garnered significant research interest due to its broad applications, enabling innovations such as photorealistic singing heads or dynamic avatars that seamlessly dance to music. However, existing surveys in this field focus on individual methods, lacking a comprehensive overview of the entire generative process. This paper addresses this gap by providing an in-depth survey of human motion video generation, encompassing over ten sub-tasks, and detailing the five key phases of the generation process: input, motion planning, motion video generation, refinement, and output. Notably, this is the first survey that discusses the potential of large language models in enhancing human motion video generation. Our survey reviews the latest developments and technological trends in human motion video generation across three primary modalities: vision, text, and audio. By covering over two hundred papers, we offer a thorough overview of the field and highlight milestone works that have driven significant technological breakthroughs. Our goal for this survey is to unveil the prospects of human motion video generation and serve as a valuable resource for advancing the comprehensive applications of digital humans. A complete list of the models examined in this survey is available in Our Repository https://github.com/Winn1y/Awesome-Human-Motion-Video-Generation.

Figures

Figures reproduced from arXiv: 2509.03883 by the authors.

Figure 1
Figure 1. Quantity of papers in the four categories reviewed in this survey, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Pipeline of generating human motion videos, which can be divided into five key phases. Initially, diverse input sources such as vision cues, text [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Timeline of key advances in vision-, text-, and audio-driven human motion video generation methods. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Different human data representations. formation of different body parts, facilitating the decoupling of distinct features. This approach enables targeted enhancements and independent manipulation of each body part, thereby refining the representation and interaction dy…
Figure 6
Figure 6. Figure 6: Overview of InstructAvatar [19], which employs GPT-4 and diffusion [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: Comparative overview of different generative frameworks based on diffusion models, where pure noise (A), a reference image (B), and guided [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Different attention fusion methods of diffusion-based vision-driven [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Different Text2Face pipelines for first-personal scripts. [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: Paradigm summary of audio-driven human motion video generation. [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]
Figure 11
Figure 11. Figure 11: Overview of common metrics in human motion video generation. [PITH_FULL_IMAGE:figures/full_fig_p013_11.png]
Figure 12
Figure 12. Figure 12: Quantitative performances of five open-source methods across [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]
Figure 13
Figure 13. Figure 13: Main challenges in human motion video generation: subpar fidelity with examples like hand blur and facial distortion, poor consistency with identity [PITH_FULL_IMAGE:figures/full_fig_p015_13.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

228 extracted references · 33 canonical work pages

  1. [208]

    Stableanimator: High-quality identity-preserving human image animation,

    S. Tu, Z. Xing, X. Han, Z.-Q. Cheng, Q. Dai, C. Luo, and Z. Wu, “Stableanimator: High-quality identity-preserving human image animation,” arXiv preprint arXiv:2411.17697, 2024

    arXiv 2024

  2. [138]

    Sd-nerf:Towards lifelike talking head animation via spatially-adaptive dual-driven nerfs,

    S.Shen,W.Li,X.Huang,Z.Zhu,J.Zhou,andJ.Lu,“Sd-nerf:Towards lifelike talking head animation via spatially-adaptive dual-driven nerfs,” IEEE TMM, 2023

    2023

  3. [142]

    Talkinggaussian: Structure-persistent 3d talking head synthesis via gaussian splatting,

    J. Li, J. Zhang, X. Bai, J. Zheng, X. Ning, J. Zhou, and L. Gu, “Talkinggaussian: Structure-persistent 3d talking head synthesis via gaussian splatting,” inECCV, 2024

    2024

  4. [143]

    Gaussiantalker: Speaker-specific talking head synthesis via 3d gaussian splatting,

    H. Yu, Z. Qu, Q. Yu, J. Chen, Z. Jiang, Z. Chen, S. Zhang, J. Xu, F. Wu, C. Lv, et al., “Gaussiantalker: Speaker-specific talking head synthesis via 3d gaussian splatting,” inACM MM, 2024

    2024

  5. [1]

    Deep video portraits,

    H. Kim, P. Garrido, A. Tewari, W. Xu, J. Thies, M. Niessner, P. Pérez, C. Richardt, M. Zollhöfer, and C. Theobalt, “Deep video portraits,” TOG, vol. 37, no. 4, pp. 1–14, 2018

    2018

  6. [2]

    Faceformer:Speech- driven 3d facial animation with transformers,

    Y.Fan,Z.Lin,J.Saito,W.Wang,andT.Komura,“Faceformer:Speech- driven 3d facial animation with transformers,” inCVPR, 2022

    2022

  7. [3]

    Make-a-video: Text-to-video generation without text-video data,

    U. Singer, A. Polyak, T. Hayes, X. Yin, J. An, S. Zhang, Q. Hu, H. Yang, O. Ashual, O. Gafni,et al., “Make-a-video: Text-to-video generation without text-video data,”arXiv preprint arXiv:2209.14792, 2022

    arXiv 2022

  8. [4]

    Animatediff: Animate your personalized text-to- image diffusion models without specific tuning,

    Y. Guo, C. Yang, A. Rao, Z. Liang, Y. Wang, Y. Qiao, M. Agrawala, D. Lin, and B. Dai, “Animatediff: Animate your personalized text-to- image diffusion models without specific tuning,” inICLR, 2024

    2024

Show all 228 references
  1. [5]

    Stable video diffusion: Scaling latent video diffusion models to large datasets,

    A. Blattmann, T. Dockhorn, S. Kulal, D. Mendelevitch, M. Kilian, D. Lorenz, Y. Levi, Z. English, V. Voleti, A. Letts,et al., “Stable video diffusion: Scaling latent video diffusion models to large datasets,”arXiv preprint arXiv:2311.15127, 2023

    2023 arXiv

  2. [6]

    Faces that speak: Jointly synthesising talking face and speech from text,

    Y. Jang, J.-H. Kim, J. Ahn, D. Kwak, H.-S. Yang, Y.-C. Ju, I.-H. Kim, B.-Y. Kim, and J. S. Chung, “Faces that speak: Jointly synthesising talking face and speech from text,” inCVPR, 2024

    2024

  3. [7]

    Magicdance: Realistic human dance video gen- eration with motions & facial expressions transfer,

    D. Chang, Y. Shi, Q. Gao, J. Fu, H. Xu, G. Song, Q. Yan, X. Yang, and M. Soleymani, “Magicdance: Realistic human dance video gen- eration with motions & facial expressions transfer,” arXiv preprint arXiv:2311.12052, 2023

    2023 arXiv

  4. [8]

    Nerf: Representing scenes as neural radiance fields for view synthesis,

    B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoor- thi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,”Communications of the ACM, vol. 65, no. 1, pp. 99– 106, 2021

    2021

  5. [9]

    3d gaussian splatting for real-time radiance field rendering,

    B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis, “3d gaussian splatting for real-time radiance field rendering,”TOG, vol. 42, no. 4, pp. 1–14, 2023

    2023

  6. [10]

    Deep person generation: A survey from the perspective of face, pose, and cloth synthesis,

    T. Sha, W. Zhang, T. Shen, Z. Li, and T. Mei, “Deep person generation: A survey from the perspective of face, pose, and cloth synthesis,”ACM Comput. Surv., vol. 55, no. 12, pp. 1–37, 2023

    2023

  7. [11]

    A comprehensive survey on human video generation: Challenges, methods, and insights,

    W. Lei, J. Wang, F. Ma, G. Huang, and L. Liu, “A comprehensive survey on human video generation: Challenges, methods, and insights,” arXiv preprint arXiv:2407.08428, 2024

    2024 arXiv

  8. [12]

    Image-based virtual try-on: A survey,

    D. Song, X. Zhang, J. Zhou, W. Nie, R. Tong, M. Kankanhalli, and A.-A. Liu, “Image-based virtual try-on: A survey,”IJCV, 2024

    2024

  9. [13]

    A comprehensive taxonomy and analysis of talking head synthesis: Techniques for portrait generation, driving mechanisms, and editing,

    M. Meng, Y. Zhao, B. Zhang, Y. Zhu, W. Shi, M. Wen, and Z. Fan, “A comprehensive taxonomy and analysis of talking head synthesis: Techniques for portrait generation, driving mechanisms, and editing,” arXiv preprint arXiv:2406.10553, 2024

    2024 arXiv

  10. [14]

    Multilingual video dubbing—a technology review and current challenges,

    D. Bigioi and P. Corcoran, “Multilingual video dubbing—a technology review and current challenges,”Frontiers in Signal Processing, vol. 3, p. 1230755, 2023

    2023

  11. [15]

    Difftalk: Crafting diffusion models for generalized audio-driven portraits anima- tion,

    S. Shen, W. Zhao, Z. Meng, W. Li, Z. Zhu, J. Zhou, and J. Lu, “Difftalk: Crafting diffusion models for generalized audio-driven portraits anima- tion,” inCVPR, 2023

    2023

  12. [16]

    Identity-preserving talking face generation with landmark and appear- ance priors,

    W. Zhong, C. Fang, Y. Cai, P. Wei, G. Zhao, L. Lin, and G. Li, “Identity-preserving talking face generation with landmark and appear- ance priors,” inCVPR, 2023

    2023

  13. [17]

    Af- fective faces for goal-driven dyadic communication,

    S. Geng, R. Teotia, P. Tendulkar, S. Menon, and C. Vondrick, “Af- fective faces for goal-driven dyadic communication,”arXiv preprint arXiv:2301.10939, 2023

    2023 arXiv

  14. [18]

    Agentavatar: Disentangling planning, driving and rendering for photorealistic avatar agents,

    D. Wang, B. Dai, Y. Deng, and B. Wang, “Agentavatar: Disentangling planning, driving and rendering for photorealistic avatar agents,”arXiv preprint arXiv:2311.17465, 2023

    2023 arXiv

  15. [19]

    Instructavatar: Text-guided emotion and motion control for avatar generation,

    Y. Wang, J. Guo, J. Bai, R. Yu, T. He, X. Tan, X. Sun, and J. Bian, “Instructavatar: Text-guided emotion and motion control for avatar generation,” inAAAI, 2025

    2025

  16. [20]

    Human motion generation: A survey,

    W. Zhu, X. Ma, D. Ro, H. Ci, J. Zhang, J. Shi, F. Gao, Q. Tian, and Y. Wang, “Human motion generation: A survey,”IEEE TPAMI, 2023

    2023

  17. [21]

    A survey of talking-head generation technology and its applications,

    H. Li, L. Zhang, X. Zou, J. Lin, X. Li, Y. Li, X. Chang, and J. Zheng, “A survey of talking-head generation technology and its applications,” in ICEAAI, 2025

    2025

  18. [22]

    Unsupervised high-resolution portrait gaze correction and animation,

    J. Zhang, J. Chen, H. Tang, E. Sangineto, P. Wu, Y. Yan, N. Sebe, and W. Wang, “Unsupervised high-resolution portrait gaze correction and animation,” IEEE TIP, vol. 31, pp. 5272–5286, 2022

    2022

  19. [23]

    Expression domain translation network for cross-domain head reenactment,

    T. Kang, J. Oh, J. Lee, S. Park, and J. Choo, “Expression domain translation network for cross-domain head reenactment,” inICASSP, 2024

    2024

  20. [24]

    Otavatar: One-shot talking face avatar with controllable tri-plane rendering,

    Z. Ma, X. Zhu, G.-J. Qi, Z. Lei, and L. Zhang, “Otavatar: One-shot talking face avatar with controllable tri-plane rendering,” in CVPR, 2023

    2023

  21. [25]

    Follow-your-emoji: Fine-controllable and expressive freestyle portrait animation,

    Y. Ma, H. Liu, H. Wang, H. Pan, Y. He, J. Yuan, A. Zeng, C. Cai, H.- Y. Shum, W. Liu, and Q. Chen, “Follow-your-emoji: Fine-controllable and expressive freestyle portrait animation,” inSIGGRAPH Asia, 2024

    2024

  22. [26]

    Liveportrait: Efficient portrait animation with stitching and retargeting control,

    J. Guo, D. Zhang, X. Liu, Z. Zhong, Y. Zhang, P. Wan, and D. Zhang, “Liveportrait: Efficient portrait animation with stitching and retargeting control,” arXiv preprint arXiv:2407.03168, 2024

    2024 arXiv

  23. [27]

    X-portrait: Expressive portrait animation with hierarchical motion attention,

    Y. Xie, H. Xu, G. Song, C. Wang, Y. Shi, and L. Luo, “X-portrait: Expressive portrait animation with hierarchical motion attention,” in SIGGRAPH, 2024

    2024

  24. [28]

    Mobileportrait: Real-time one-shot neural head avatars on mobile devices,

    J. Jiang, G. Lin, Z. Rong, C. Liang, Y. Zhu, J. Yang, and T. Zhong, “Mobileportrait: Real-time one-shot neural head avatars on mobile devices,” arXiv preprint arXiv:2407.05712, 2024

    2024 arXiv

  25. [29]

    Everybody dance now,

    C. Chan, S. Ginosar, T. Zhou, and A. A. Efros, “Everybody dance now,” inICCV, 2019

    2019

  26. [30]

    Human motionformer: Transferring human motions with vision transformers,

    H. Liu, X. Han, C. Jin, L. Qian, H. Wei, Z. Lin, F. Wang, H. Dong, Y. Song, J. Xu, and Q. Chen, “Human motionformer: Transferring human motions with vision transformers,” inICLR, 2023

    2023

  27. [31]

    Bidirectional temporal diffusion model for temporally consistent human animation,

    T. Adiya, J. S. Yoon, J. Lee, S. Kim, and H. Lim, “Bidirectional temporal diffusion model for temporally consistent human animation,” in ICLR, 2024

    2024

  28. [32]

    Disco: Disentangled control for realistic human dance generation,

    T. Wang, L. Li, K. Lin, Y. Zhai, C.-C. Lin, Z. Yang, H. Zhang, Z. Liu, and L. Wang, “Disco: Disentangled control for realistic human dance generation,” inCVPR, 2024

    2024

  29. [33]

    Animate anyone: Consistent and controllable image-to-video synthesis for character animation,

    L. Hu, “Animate anyone: Consistent and controllable image-to-video synthesis for character animation,” inCVPR, 2024

    2024

  30. [34]

    Follow-your-pose v2: Multiple- condition guided character image animation for stable pose control,

    J. Xue, H. Wang, Q. Tian, Y. Ma, A. Wang, Z. Zhao, S. Min, W. Zhao, K. Zhang, H.-Y. Shum,et al., “Follow-your-pose v2: Multiple- condition guided character image animation for stable pose control,” arXiv preprint arXiv:2406.03035, 2024

    2024 arXiv

  31. [35]

    Human4dit: Free-view human video generation with 4d diffusion transformer,

    R. Shao, Y. Pang, Z. Zheng, J. Sun, and Y. Liu, “Human4dit: Free-view human video generation with 4d diffusion transformer,”arXiv preprint arXiv:2405.17405, 2024

    2024 arXiv

  32. [36]

    Mimicmotion: High-quality human motion video generation with confidence-aware pose guidance,

    Y. Zhang, J. Gu, L.-W. Wang, H. Wang, J. Cheng, Y. Zhu, and F. Zou, “Mimicmotion: High-quality human motion video generation with confidence-aware pose guidance,” arXiv preprint arXiv:2406.19680, 2024

    2024 arXiv

  33. [37]

    I2v-adapter: A general image-to-video adapter for diffusion models,

    X. Guo, M. Zheng, L. Hou, Y. Gao, Y. Deng, P. Wan, D. Zhang, Y. Liu, W. Hu, and Z. Zha, “I2v-adapter: A general image-to-video adapter for diffusion models,” inSIGGRAPH, 2024

    2024

  34. [38]

    Vivid: Video virtual try-on using diffusion models,

    Z. Fang, W. Zhai, A. Su, H. Song, K. Zhu, M. Wang, Y. Chen, Z. Liu, Y. Cao, and Z.-J. Zha, “Vivid: Video virtual try-on using diffusion models,” arXiv preprint arXiv:2405.11794, 2024

    2024 arXiv

  35. [39]

    Dreampose: Fashion image-to-video synthesis via stable diffusion,

    J. Karras, A. Holynski, T.-C. Wang, and I. Kemelmacher-Shlizerman, “Dreampose: Fashion image-to-video synthesis via stable diffusion,” in ICCV, 2023

    2023

  36. [40]

    Make-your-anchor: A diffusion-based 2d avatar generation frame- work,

    Z. Huang, F. Tang, Y. Zhang, X. Cun, J. Cao, J. Li, and T.-Y. Lee, “Make-your-anchor: A diffusion-based 2d avatar generation frame- work,” inCVPR, 2024

    2024

  37. [41]

    Write-a-speaker: Text-based emotional and rhythmic talking-head gen- eration,

    L. Li, S. Wang, Z. Zhang, Y. Ding, Y. Zheng, X. Yu, and C. Fan, “Write-a-speaker: Text-based emotional and rhythmic talking-head gen- eration,” inAAAI, 2021

    2021

  38. [42]

    Id-animator: Zero-shot identity-preserving human video generation,

    X. He, Q. Liu, S. Qian, X. Wang, T. Hu, K. Cao, K. Yan, M. Zhou, and J. Zhang, “Id-animator: Zero-shot identity-preserving human video generation,” arXiv preprint arXiv:2404.15275, 2024

    2024 arXiv

  39. [43]

    Edit-your-motion: Space-time diffusion decoupling learning for video motion editing,

    Y. Zuo, L. Li, L. Jiao, F. Liu, X. Liu, W. Ma, S. Yang, and Y. Guo, “Edit-your-motion: Space-time diffusion decoupling learning for video motion editing,”arXiv preprint arXiv:2405.04496, 2024. XUE et al.: HUMAN MOTION VIDEO GENERATION: A SURVEY 17

    2024 arXiv

  40. [44]

    Follow your pose: Pose-guided text-to-video generation using pose- free videos,

    Y. Ma, Y. He, X. Cun, X. Wang, S. Chen, X. Li, and Q. Chen, “Follow your pose: Pose-guided text-to-video generation using pose- free videos,” inAAAI, 2024

    2024

  41. [45]

    Text2performer: Text-driven human video generation,

    Y. Jiang, S. Yang, T. L. Koh, W. Wu, C. C. Loy, and Z. Liu, “Text2performer: Text-driven human video generation,” inICCV, 2023

    2023

  42. [46]

    Styleheat: One-shot high-resolution editable talking face generation via pre-trained stylegan,

    F. Yin, Y. Zhang, X. Cun, M. Cao, Y. Fan, X. Wang, Q. Bai, B. Wu, J. Wang, and Y. Yang, “Styleheat: One-shot high-resolution editable talking face generation via pre-trained stylegan,” inECCV, 2022

    2022

  43. [47]

    Pose- controllable talking face generation by implicitly modularized audio- visual representation,

    H. Zhou, Y. Sun, W. Wu, C. C. Loy, X. Wang, and Z. Liu, “Pose- controllable talking face generation by implicitly modularized audio- visual representation,” inCVPR, 2021

    2021

  44. [48]

    Edtalk: Efficient disentanglement for emotional talking head synthesis,

    S. Tan, B. Ji, M. Bi, and Y. Pan, “Edtalk: Efficient disentanglement for emotional talking head synthesis,” inECCV, 2024

    2024

  45. [49]

    Echomimic: Lifelike audio-drivenportraitanimationsthrougheditablelandmarkconditions,

    Z. Chen, J. Cao, Z. Chen, Y. Li, and C. Ma, “Echomimic: Lifelike audio-drivenportraitanimationsthrougheditablelandmarkconditions,” arXiv preprint arXiv:2407.08136, 2024

    2024 arXiv

  46. [50]

    Emo: Emote portrait alive generating expressive portrait videos with audio2video diffusion model under weak conditions,

    L. Tian, Q. Wang, B. Zhang, and L. Bo, “Emo: Emote portrait alive generating expressive portrait videos with audio2video diffusion model under weak conditions,” inECCV, 2024

    2024

  47. [51]

    Hallo: Hierarchical audio-driven visual synthesis for portrait image animation,

    M. Xu, H. Li, Q. Su, H. Shang, L. Zhang, C. Liu, J. Wang, L. Van Gool, Y. Yao, and S. Zhu, “Hallo: Hierarchical audio-driven visual synthesis for portrait image animation,”arXiv preprint arXiv:2406.08801, 2024

    2024 arXiv

  48. [52]

    Emotional conversation: Empowering talking faces with cohesive expression, gaze and pose generation,

    J. Liang and F. Lu, “Emotional conversation: Empowering talking faces with cohesive expression, gaze and pose generation,”arXiv preprint arXiv:2406.07895, 2024

    2024 arXiv

  49. [53]

    Makeittalk: Speaker-aware talking-head animation,

    Y. Zhou, X. Han, E. Shechtman, J. Echevarria, E. Kalogerakis, and D. Li, “Makeittalk: Speaker-aware talking-head animation,” TOG, vol. 39, no. 6, pp. 1–15, 2020

    2020

  50. [54]

    Live speech portraits: Real-time photore- alistic talking-head animation,

    Y. Lu, J. Chai, and X. Cao, “Live speech portraits: Real-time photore- alistic talking-head animation,”TOG, vol. 40, no. 6, pp. 1–17, 2021

    2021

  51. [55]

    Vlogger: Multimodal diffusion for embodied avatar synthesis,

    E. Corona, A. Zanfir, E. G. Bazavan, N. Kolotouros, T. Alldieck, and C. Sminchisescu, “Vlogger: Multimodal diffusion for embodied avatar synthesis,” arXiv preprint arXiv:2403.08764, 2024

    2024 arXiv

  52. [56]

    Dance any beat: Blend- ing beats with visuals in dance video generation,

    X. Wang, H. Wang, D. Liu, and W. Cai, “Dance any beat: Blend- ing beats with visuals in dance video generation,” arXiv preprint arXiv:2405.09266, 2024

    2024 arXiv

  53. [57]

    Auto-encoding variational bayes,

    D. P. Kingma and M. Welling, “Auto-encoding variational bayes,”arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  54. [58]

    Geneface: Generalized and high-fidelity audio-driven 3d talking face synthesis,

    Z. Ye, Z. Jiang, Y. Ren, J. Liu, J. He, and Z. Zhao, “Geneface: Generalized and high-fidelity audio-driven 3d talking face synthesis,” in ICLR, 2023

    2023

  55. [59]

    Geneface++: Generalized and stable real-time audio-driven 3d talking face generation,

    Z. Ye, J. He, Z. Jiang, R. Huang, J. Huang, J. Liu, Y. Ren, X. Yin, Z. Ma, and Z. Zhao, “Geneface++: Generalized and stable real-time audio-driven 3d talking face generation,” arXiv preprint arXiv:2305.00787, 2023

    2023 arXiv

  56. [60]

    Neural discrete representation learning,

    A. Van Den Oord, O. Vinyals,et al., “Neural discrete representation learning,” inNeurIPS, 2017

    2017

  57. [61]

    Generative adversarial nets,

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in NeurIPS, 2014

    2014

  58. [62]

    A style-based generator architecture for generative adversarial networks,

    T. Karras, S. Laine, and T. Aila, “A style-based generator architecture for generative adversarial networks,” inCVPR, 2019

    2019

  59. [63]

    Analyzing and improving the image quality of stylegan,

    T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila, “Analyzing and improving the image quality of stylegan,” inCVPR, 2020

    2020

  60. [64]

    An identity-preserved framework for human motion transfer,

    J. Ma, X. Zhang, and S. Yu, “An identity-preserved framework for human motion transfer,” IEEE Trans. Inf. Forensics Secur., vol. 19, pp. 3495–3509, 2024

    2024

  61. [65]

    Deep unsupervised learning using nonequilibrium thermodynamics,

    J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” in ICML, 2015

    2015

  62. [66]

    Improved techniques for training score-based generative models,

    Y. Song and S. Ermon, “Improved techniques for training score-based generative models,” inNeurIPS, 2020

    2020

  63. [67]

    Improved denoising diffusion proba- bilistic models,

    A. Q. Nichol and P. Dhariwal, “Improved denoising diffusion proba- bilistic models,” inICML, 2021

    2021

  64. [68]

    Denoising diffusion implicit mod- els,

    J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit mod- els,” inICLR, 2020

    2020

  65. [69]

    Diffusion models beat gans on image synthesis,

    P. Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,” inNeurIPS, 2021

    2021

  66. [70]

    A survey on generative diffusion models,

    H. Cao, C. Tan, Z. Gao, Y. Xu, G. Chen, P.-A. Heng, and S. Z. Li, “A survey on generative diffusion models,”IEEE Trans. Knowl. Data Eng., vol. 36, no. 7, p. 2814–2830, 2024

    2024

  67. [71]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” inNeurIPS, 2020

    2020

  68. [72]

    Dance your latents: Consistent dance generation through spatial-temporal subspace attention guided by motion flow,

    H. Fang, Z. Sun, Z. Huang, F. Tang, J. Cao, and S. Tang, “Dance your latents: Consistent dance generation through spatial-temporal subspace attention guided by motion flow,”arXiv preprint arXiv:2310.14780, 2023

    2023 arXiv

  69. [73]

    Human modelling and pose estimation overview,

    P. Knap, “Human modelling and pose estimation overview,” arXiv preprint arXiv:2406.19290, 2024

    2024 arXiv

  70. [74]

    Champ: Controllable and consistent human image animation with 3d parametric guidance,

    S. Zhu, J. L. Chen, Z. Dai, Z. Dong, Y. Xu, X. Cao, Y. Yao, H. Zhu, and S. Zhu, “Champ: Controllable and consistent human image animation with 3d parametric guidance,” inECCV, 2024

    2024

  71. [75]

    Openpose: Realtime multi-person 2d pose estimation using part affinity fields,

    Z. Cao, G. Hidalgo, T. Simon, S.-E. Wei, and Y. Sheikh, “Openpose: Realtime multi-person 2d pose estimation using part affinity fields,” IEEE TPAMI, vol. 43, no. 1, pp. 172–186, 2021

    2021

  72. [76]

    Effective whole-body pose estimation with two-stages distillation,

    Z. Yang, A. Zeng, C. Yuan, and Y. Li, “Effective whole-body pose estimation with two-stages distillation,” inICCVW, 2023

    2023

  73. [77]

    Vividpose: Advancing stable video diffusion for realistic human image animation,

    Q. Wang, Z. Jiang, C. Xu, J. Zhang, Y. Wang, X. Zhang, Y. Cao, W. Cao, C. Wang, and Y. Fu, “Vividpose: Advancing stable video diffusion for realistic human image animation,”arXiv preprint arXiv:2405.18156, 2024

    2024 arXiv

  74. [78]

    Magicanimate: Temporally consistent human image animation using diffusion model,

    Z. Xu, J. Zhang, J. H. Liew, H. Yan, J.-W. Liu, C. Zhang, J. Feng, and M. Z. Shou, “Magicanimate: Temporally consistent human image animation using diffusion model,” inCVPR, 2024

    2024

  75. [79]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark,et al., “Learning transferable visual models from natural language supervision,” inICML, 2021

    2021

  76. [80]

    Conformer: Convolution- augmented transformer for speech recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y. Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y. Wu, and R. Pang, “Conformer: Convolution- augmented transformer for speech recognition,” inInterspeech, 2020

    2020

  77. [81]

    Omniavatar: Geometry-guided controllable 3d head synthesis,

    H. Xu, G. Song, Z. Jiang, J. Zhang, Y. Shi, J. Liu, W. Ma, J. Feng, and L. Luo, “Omniavatar: Geometry-guided controllable 3d head synthesis,” inCVPR, 2023

    2023

  78. [82]

    Megactor: Harness the power of raw video for vivid portrait anima- tion,

    S. Yang, H. Li, J. Wu, M. Jing, L. Li, R. Ji, J. Liang, and H. Fan, “Megactor: Harness the power of raw video for vivid portrait anima- tion,” arXiv preprint arXiv:2405.20851, 2024

    2024 arXiv

  79. [83]

    Faceoff: A video-to-video face swapping system,

    A. Agarwal, B. Sen, R. Mukhopadhyay, V. P. Namboodiri, and C. Jawa- har, “Faceoff: A video-to-video face swapping system,” inWACV, 2023

    2023

  80. [84]

    Finemogen:Fine- grained spatio-temporal motion generation and editing,

    M.Zhang,H.Li,Z.Cai,J.Ren,L.Yang,andZ.Liu,“Finemogen:Fine- grained spatio-temporal motion generation and editing,” inNeurIPS, 2024

    2024

  81. [85]

    Plan, posture and go: Towards open-world text-to-motion generation,

    J. Liu, W. Dai, C. Wang, Y. Cheng, Y. Tang, and X. Tong, “Plan, posture and go: Towards open-world text-to-motion generation,”arXiv preprint arXiv:2312.14828, 2023

    2023 arXiv

  82. [86]

    Avatargpt: All-in-one framework for motion understanding planning generation and beyond,

    Z. Zhou, Y. Wan, and B. Wang, “Avatargpt: All-in-one framework for motion understanding planning generation and beyond,” inCVPR, 2024

    2024

  83. [87]

    Motiongpt:Finetunedllmsaregeneral-purpose motion generators,

    Y. Zhang, D. Huang, B. Liu, S. Tang, Y. Lu, L. Chen, L. Bai, Q. Chu, N.Yu,andW.Ouyang,“Motiongpt:Finetunedllmsaregeneral-purpose motion generators,” inAAAI, 2024

    2024

  84. [88]

    Motionscript: Natural language descriptions for expressive 3d human motions,

    P. J. Yazdian, E. Liu, L. Cheng, and A. Lim, “Motionscript: Natural language descriptions for expressive 3d human motions,”arXiv preprint arXiv:2312.12634, 2023

    2023

  85. [89]

    Can language models learn to listen?,

    E. Ng, S. Subramanian, D. Klein, A. Kanazawa, T. Darrell, and S. Ginosar, “Can language models learn to listen?,” inICCV, 2023

    2023

  86. [90]

    Intercontrol: Zero-shot human interaction generation by controlling every joint,

    Z. Wang, J. Wang, Y. Li, D. Lin, and B. Dai, “Intercontrol: Zero-shot human interaction generation by controlling every joint,” inNeurIPS, 2024

    2024

  87. [91]

    Digital life project: Autonomous 3d characters with social intelligence,

    Z. Cai, J. Jiang, Z. Qing, X. Guo, M. Zhang, Z. Lin, H. Mei, C. Wei, R. Wang, W. Yin,et al., “Digital life project: Autonomous 3d characters with social intelligence,” inCVPR, 2024

    2024

  88. [92]

    Style-preservinglipsyncvia audio-aware style reference,

    W.Zhong,J.Li,Y.Cai,L.Lin,andG.Li,“Style-preservinglipsyncvia audio-aware style reference,”arXiv preprint arXiv:2408.05412, 2024

    2024 arXiv

  89. [93]

    Dae-talker: High fidelity speech-driven talking face generation with diffusion autoencoder,

    C. Du, Q. Chen, T. He, X. Tan, X. Chen, K. Yu, S. Zhao, and J. Bian, “Dae-talker: High fidelity speech-driven talking face generation with diffusion autoencoder,” inACM MM, 2023

    2023

  90. [94]

    High-fidelity generalized emotional talking face generation with multi-modal emotion space learning,

    C. Xu, J. Zhu, J. Zhang, Y. Han, W. Chu, Y. Tai, C. Wang, Z. Xie, and Y. Liu, “High-fidelity generalized emotional talking face generation with multi-modal emotion space learning,” inCVPR, 2023

    2023

  91. [95]

    Do as i do: Pose guided human motion copy,

    S. Wu, Z. Liu, B. Zhang, R. Zimmermann, Z. Ba, X. Zhang, and K. Ren, “Do as i do: Pose guided human motion copy,”IEEE Trans. Dependable Secur. Comput., no. 01, pp. 1–16, 2024

    2024

  92. [96]

    Magicpose: Realistic human poses and facial expressions retargeting with identity-aware diffusion,

    D. Chang, Y. Shi, Q. Gao, J. Fu, H. Xu, G. Song, Q. Yan, X. Yang, and M. Soleymani, “Magicpose: Realistic human poses and facial expressions retargeting with identity-aware diffusion,” inICML, 2024

    2024

  93. [97]

    Dreamoving: A human dance video generation framework based on diffusion models,

    M. Feng, J. Liu, K. Yu, Y. Yao, Z. Hui, X. Guo, X. Lin, H. Xue, C. Shi, X. Li,et al., “Dreamoving: A human dance video generation framework based on diffusion models,”arXiv preprint arXiv:2312.05107, 2023

    2023 arXiv

  94. [98]

    Disentangling fore- ground and background motion for enhanced realism in human video generation,

    J. Liu, K. Yu, M. Feng, X. Guo, and M. Cui, “Disentangling fore- ground and background motion for enhanced realism in human video generation,” arXiv preprint arXiv:2405.16393, 2024. XUE et al.: HUMAN MOTION VIDEO GENERATION: A SURVEY 18

    2024 arXiv

  95. [99]

    Motionfollower: Editing video motion via lightweight score-guided diffusion,

    S. Tu, Q. Dai, Z. Zhang, S. Xie, Z.-Q. Cheng, C. Luo, X. Han, Z. Wu, and Y.-G. Jiang, “Motionfollower: Editing video motion via lightweight score-guided diffusion,”arXiv preprint arXiv:2405.20325, 2024

    2024 arXiv

  96. [100]

    Unianimate: Taming unified video diffusion models for consistent human image animation,

    X. Wang, S. Zhang, C. Gao, J. Wang, X. Zhou, Y. Zhang, L. Yan, and N. Sang, “Unianimate: Taming unified video diffusion models for consistent human image animation,”arXiv preprint arXiv:2406.01188, 2024

    2024 arXiv

  97. [101]

    Synthesizing moving people with 3d control,

    B. Li, J. Rajasegaran, Y. Gandelsman, A. A. Efros, and J. Ma- lik, “Synthesizing moving people with 3d control,” arXiv preprint arXiv:2401.10889, 2024

    2024 arXiv

  98. [102]

    Tcan: Animating human im- ages with temporally consistent pose guidance using diffusion models,

    J. Kim, M.-J. Kim, J. Lee, and J. Choo, “Tcan: Animating human im- ages with temporally consistent pose guidance using diffusion models,” in ECCV, 2024

    2024

  99. [103]

    Idol: Unified dual-modal latent diffusion for human-centric joint video-depth generation,

    Y. Zhai, K. Lin, L. Li, C.-C. Lin, J. Wang, Z. Yang, D. Doermann, J. Yuan, Z. Liu, and L. Wang, “Idol: Unified dual-modal latent diffusion for human-centric joint video-depth generation,” inECCV, 2024

    2024

  100. [104]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, and S. Gelly, “An image is worth 16x16 words: Transformers for image recognition at scale,” inICLR, 2020

    2020

  101. [105]

    Gromov-wasserstein averaging of kernel and distance matrices,

    G. Peyré, M. Cuturi, and J. Solomon, “Gromov-wasserstein averaging of kernel and distance matrices,” inICML, 2016

    2016

  102. [106]

    Adding conditional control to text-to-image diffusion models,

    L. Zhang, A. Rao, and M. Agrawala, “Adding conditional control to text-to-image diffusion models,” inICCV, 2023

    2023

  103. [107]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” in ICLR, 2022

    2022

  104. [108]

    U-net: Convolutional net- works for biomedical image segmentation,

    O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional net- works for biomedical image segmentation,” inMICCAI, 2015

    2015

  105. [109]

    Zero-shot high-fidelity and pose-controllable character animation,

    B. Zhu, F. Wang, T. Lu, P. Liu, J. Su, J. Liu, Y. Zhang, Z. Wu, G.- J. Qi, and Y.-G. Jiang, “Zero-shot high-fidelity and pose-controllable character animation,” inIJCAI, 2024

    2024

  106. [110]

    Mamba: Linear-time sequence modeling with selective state spaces,

    A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” 2024

    2024

  107. [111]

    Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models,

    H. Ye, J. Zhang, S. Liu, X. Han, and W. Yang, “Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models,” arXiv preprint arXiv:2308.06721, 2023

    2023 arXiv

  108. [112]

    Scalable diffusion models with transformers,

    W. Peebles and S. Xie, “Scalable diffusion models with transformers,” in ICCV, 2023

    2023

  109. [113]

    Visual autoregres- sive modeling: Scalable image generation via next-scale prediction,

    K. Tian, Y. Jiang, Z. Yuan, B. PENG, and L. Wang, “Visual autoregres- sive modeling: Scalable image generation via next-scale prediction,” in NeurIPS, 2024

    2024

  110. [114]

    Tunnel try-on: Excavating spatial-temporal tunnels for high-quality virtual try-on in videos,

    Z. Xu, M. Chen, Z. Wang, L. XING, Z. Zhai, N. Sang, J. Lan, S. Xiao, and C. Gao, “Tunnel try-on: Excavating spatial-temporal tunnels for high-quality virtual try-on in videos,” inACM MM, 2024

    2024

  111. [115]

    Wildvidfit: Video virtual try-on in the wild via image-based controlled diffusion models,

    Z. He, P. Chen, G. Wang, G. Li, P. H. Torr, and L. Lin, “Wildvidfit: Video virtual try-on in the wild via image-based controlled diffusion models,” inECCV, 2024

    2024

  112. [116]

    Diffusion models: A comprehensive survey ofmethodsandapplications,

    L. Yang, Z. Zhang, Y. Song, S. Hong, R. Xu, Y. Zhao, W. Zhang, B. Cui, and M.-H. Yang, “Diffusion models: A comprehensive survey ofmethodsandapplications,” ACMComput.Surv. ,vol.56,no.4,pp.1– 39, 2023

    2023

  113. [117]

    Towards a simultaneous and granular identity-expression control in personalized face generation,

    R. Liu, B. Ma, W. Zhang, Z. Hu, C. Fan, T. Lv, Y. Ding, and X. Cheng, “Towards a simultaneous and granular identity-expression control in personalized face generation,” inCVPR, 2024

    2024

  114. [118]

    Videocomposer: Compositional video synthesis with motion controllability,

    X. Wang, H. Yuan, S. Zhang, D. Chen, J. Wang, Y. Zhang, Y. Shen, D. Zhao, and J. Zhou, “Videocomposer: Compositional video synthesis with motion controllability,” inNeurIPS, 2024

    2024

  115. [119]

    Text-to-video: A two-stage framework for zero-shot identity-agnostic talking-head generation,

    Z. Wang, M. Dai, and K. Lundgaard, “Text-to-video: A two-stage framework for zero-shot identity-agnostic talking-head generation,” arXiv preprint arXiv:2308.06457, 2023

    2023 arXiv

  116. [120]

    Neural text to articulate talk: Deep text to audiovisual speech synthesis achieving both auditory and photo-realism,

    G. Milis, P. P. Filntisis, A. Roussos, and P. Maragos, “Neural text to articulate talk: Deep text to audiovisual speech synthesis achieving both auditory and photo-realism,”arXiv preprint arXiv:2312.06613, 2023

    2023 arXiv

  117. [121]

    Ft2tf: First-person state- ment text-to-talking face generation,

    X. Diao, M. Cheng, W. Barrios, and S. Jin, “Ft2tf: First-person state- ment text-to-talking face generation,”arXiv preprint arXiv:2312.05430, 2023

    2023 arXiv

  118. [122]

    Yourtts: Towards zero-shot multi-speaker tts and zero-shot voice conversion for everyone,

    E. Casanova, J. Weber, C. D. Shulby, A. C. Junior, E. Gölge, and M. A. Ponti, “Yourtts: Towards zero-shot multi-speaker tts and zero-shot voice conversion for everyone,” inICML, 2022

    2022

  119. [123]

    Visual speech recognition for multiple languages in the wild,

    P. Ma, S. Petridis, and M. Pantic, “Visual speech recognition for multiple languages in the wild,” Nat. Mach. Intell., vol. 4, no. 11, pp. 930–939, 2022

    2022

  120. [124]

    Magic-me: Identity-specific video customized diffusion,

    Z. Ma, D. Zhou, C.-H. Yeh, X.-S. Wang, X. Li, H. Yang, Z. Dong, K. Keutzer, and J. Feng, “Magic-me: Identity-specific video customized diffusion,” arXiv preprint arXiv:2402.09368, 2024

    2024 arXiv

  121. [125]

    Text2video-zero: Text-to-image diffusion models are zero-shot video generators,

    L. Khachatryan, A. Movsisyan, V. Tadevosyan, R. Henschel, Z. Wang, S. Navasardyan, and H. Shi, “Text2video-zero: Text-to-image diffusion models are zero-shot video generators,” inICCV, 2023

    2023

  122. [126]

    Bivdiff: A training-free framework for general-purpose video synthesis via bridging image and video diffusion models,

    F. Shi, J. Gu, H. Xu, S. Xu, W. Zhang, and L. Wang, “Bivdiff: A training-free framework for general-purpose video synthesis via bridging image and video diffusion models,” inCVPR, 2024

    2024

  123. [127]

    Magica- vatar: Multimodal avatar generation and animation,

    J. Zhang, H. Yan, Z. Xu, J. Feng, and J. H. Liew, “Magica- vatar: Multimodal avatar generation and animation,” arXiv preprint arXiv:2308.14748, 2023

    2023 arXiv

  124. [128]

    Densepose: Dense human pose estimation in the wild,

    R. A. Güler, N. Neverova, and I. Kokkinos, “Densepose: Dense human pose estimation in the wild,” inCVPR, 2018

    2018

  125. [129]

    Videocrafter1: Open diffusion models for high-quality video generation,

    H. Chen, M. Xia, Y. He, Y. Zhang, X. Cun, S. Yang, J. Xing, Y. Liu, Q. Chen, X. Wang,et al., “Videocrafter1: Open diffusion models for high-quality video generation,”arXiv preprint arXiv:2310.19512, 2023

    2023 arXiv

  126. [130]

    Videocrafter2: Overcoming data limitations for high-quality video diffusion models,

    H. Chen, Y. Zhang, X. Cun, M. Xia, X. Wang, C. Weng, and Y. Shan, “Videocrafter2: Overcoming data limitations for high-quality video diffusion models,” inCVPR, 2024

    2024

  127. [131]

    Animatezero: Video diffusion models are zero-shot image animators,

    J. Yu, X. Cun, C. Qi, Y. Zhang, X. Wang, Y. Shan, and J. Zhang, “Animatezero: Video diffusion models are zero-shot image animators,” arXiv preprint arXiv:2312.03793, 2023

    2023 arXiv

  128. [132]

    Photo- realistic audio-driven video portraits,

    X. Wen, M. Wang, C. Richardt, Z.-Y. Chen, and S.-M. Hu, “Photo- realistic audio-driven video portraits,”IEEE TVCG, vol. 26, no. 12, pp. 3457–3466, 2020

    2020

  129. [133]

    Hierarchical cross-modal talking face generation with dynamic pixel-wise loss,

    L. Chen, R. K. Maddox, Z. Duan, and C. Xu, “Hierarchical cross-modal talking face generation with dynamic pixel-wise loss,” inCVPR, 2019

    2019

  130. [134]

    Capture, learning, and synthesis of 3d speaking styles,

    D. Cudeiro, T. Bolkart, C. Laidlaw, A. Ranjan, and M. J. Black, “Capture, learning, and synthesis of 3d speaking styles,” in CVPR, 2019

    2019

  131. [135]

    Speech driven video editing via an audio- conditioned diffusion model,

    D. Bigioi, S. Basak, M. Stypułkowski, M. Zieba, H. Jordan, R. Mc- Donnell, and P. Corcoran, “Speech driven video editing via an audio- conditioned diffusion model,”Image Vis. Comput., vol. 142, 2024

    2024

  132. [136]

    A lip sync expert is all you need for speech to lip generation in the wild,

    K. R. Prajwal, R. Mukhopadhyay, V. P. Namboodiri, and C. Jawahar, “A lip sync expert is all you need for speech to lip generation in the wild,” inACM MM, 2020

    2020

  133. [137]

    Aniportrait: Audio-driven synthesis of photorealistic portrait animation,

    H. Wei, Z. Yang, and Z. Wang, “Aniportrait: Audio-driven synthesis of photorealistic portrait animation,”arXiv preprint arXiv:2403.17694, 2024

    2024 arXiv

  134. [139]

    Diffused heads: Diffusion models beat gans on talking- face generation,

    M. Stypułkowski, K. Vougioukas, S. He, M. Zięba, S. Petridis, and M. Pantic, “Diffused heads: Diffusion models beat gans on talking- face generation,” inWACV, 2024

    2024

  135. [140]

    Anitalker: animate vivid and diverse talking faces through identity- decoupled facial motion encoding,

    T. Liu, F. Chen, S. Fan, C. Du, Q. Chen, X. Chen, and K. Yu, “Anitalker: animate vivid and diverse talking faces through identity- decoupled facial motion encoding,” inACM MM, 2024

    2024

  136. [141]

    Synctalk: The devil is in the synchronization for talking head synthesis,

    Z. Peng, W. Hu, Y. Shi, X. Zhu, X. Zhang, H. Zhao, J. He, H. Liu, and Z. Fan, “Synctalk: The devil is in the synchronization for talking head synthesis,” inCVPR, 2024

    2024

  137. [144]

    Towards the generation of synchronized and believable non-verbal facial behaviors of a talking virtual agent,

    A. Delbosc, M. Ochs, N. Sabouret, B. Ravenet, and S. Ayache, “Towards the generation of synchronized and believable non-verbal facial behaviors of a talking virtual agent,” inICMI, 2023

    2023

  138. [145]

    Corrtalk: Correlation between hierarchical speech and facial activity variances for 3d animation,

    Z. Chu, K. Guo, X. Xing, Y. Lan, B. Cai, and X. Xu, “Corrtalk: Correlation between hierarchical speech and facial activity variances for 3d animation,”IEEE TCSVT, 2024

    2024

  139. [146]

    Dual- stream diffusion net for text-to-video generation,

    B. Liu, X. Liu, A. Dai, Z. Zeng, Z. Cui, and J. Yang, “Dual- stream diffusion net for text-to-video generation,” arXiv preprint arXiv:2308.08316, 2023

    2023 arXiv

  140. [147]

    Customize-a-video: One-shot motion customization of text-to-video diffusion models,

    Y. Ren, Y. Zhou, J. Yang, J. Shi, D. Liu, F. Liu, M. Kwon, and A. Shrivastava, “Customize-a-video: One-shot motion customization of text-to-video diffusion models,”arXiv preprint arXiv:2402.14780, 2024

    2024 arXiv

  141. [148]

    Tokenflow: Consistent diffusion features for consistent video editing,

    M. Geyer, O. Bar-Tal, S. Bagon, and T. Dekel, “Tokenflow: Consistent diffusion features for consistent video editing,” inICLR, 2024

    2024

  142. [149]

    Latentman: Generating consistent ani- mated characters using image diffusion models,

    A. Eldesokey and P. Wonka, “Latentman: Generating consistent ani- mated characters using image diffusion models,” inCVPR, 2024

    2024

  143. [150]

    Zerosmooth: Training- free diffuser adaptation for high frame rate video generation,

    S.Yang, Y.Zhang, X.Cun, Y.Shan, andR. He,“Zerosmooth: Training- free diffuser adaptation for high frame rate video generation,”arXiv preprint arXiv:2406.00908, 2024. XUE et al.: HUMAN MOTION VIDEO GENERATION: A SURVEY 19

    2024 arXiv

  144. [151]

    Dancing avatar: Pose and text-guided human motion videos synthesis with image diffusion model,

    B. Qin, W. Ye, Q. Yu, S. Tang, and Y. Zhuang, “Dancing avatar: Pose and text-guided human motion videos synthesis with image diffusion model,” arXiv preprint arXiv:2308.07749, 2023

    2023 arXiv

  145. [152]

    Predicting head pose from speech with a conditional variational autoencoder,

    D. Greenwood, S. Laycock, and I. Matthews, “Predicting head pose from speech with a conditional variational autoencoder,” inISCA, 2017

    2017

  146. [153]

    Bidirectional lstm networks for improved phoneme classification and recognition,

    A. Graves, S. Fernández, and J. Schmidhuber, “Bidirectional lstm networks for improved phoneme classification and recognition,” in ICANN, 2005

    2005

  147. [154]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in NeurIPS, 2017

    2017

  148. [155]

    Audio-driven emotional video portraits,

    X. Ji, H. Zhou, K. Wang, W. Wu, C. C. Loy, X. Cao, and F. Xu, “Audio-driven emotional video portraits,” inCVPR, 2021

    2021

  149. [156]

    Styletalk: One-shot talking head generation with controllable speaking styles,

    Y. Ma, S. Wang, Z. Hu, C. Fan, T. Lv, Y. Ding, Z. Deng, and X. Yu, “Styletalk: One-shot talking head generation with controllable speaking styles,” inAAAI, 2023

    2023

  150. [157]

    Real3d-portrait: One-shot realistic 3d talking portrait synthesis,

    Z. Ye, T. Zhong, Y. Ren, J. Yang, W. Li, J. Huang, Z. Jiang, J. He, R. Huang, J. Liu, et al., “Real3d-portrait: One-shot realistic 3d talking portrait synthesis,” inICLR, 2024

    2024

  151. [158]

    Dreamtalk: When expressive talking head generation meets diffusion probabilistic models,

    Y. Ma, S. Zhang, J. Wang, X. Wang, Y. Zhang, and Z. Deng, “Dreamtalk: When expressive talking head generation meets diffusion probabilistic models,”arXiv preprint arXiv:2312.09767, 2023

    2023 arXiv

  152. [159]

    V-express: Conditional dropout for progressive training of portrait video generation,

    C. Wang, K. Tian, J. Zhang, Y. Guan, F. Luo, F. Shen, Z. Jiang, Q. Gu, X. Han, and W. Yang, “V-express: Conditional dropout for progressive training of portrait video generation,”arXiv preprint arXiv:2406.02511, 2024

    2024 arXiv

  153. [160]

    Speech driven talking face generation from a single image and an emotion condition,

    S. E. Eskimez, Y. Zhang, and Z. Duan, “Speech driven talking face generation from a single image and an emotion condition,”IEEE Trans. Multimedia, vol. 24, 2021

    2021

  154. [161]

    Sadtalker: Learning realistic 3d motion coefficients for stylized audio-driven single image talking face animation,

    W. Zhang, X. Cun, X. Wang, Y. Zhang, X. Shen, Y. Guo, Y. Shan, and F. Wang, “Sadtalker: Learning realistic 3d motion coefficients for stylized audio-driven single image talking face animation,” inCVPR, 2023

    2023

  155. [162]

    High-fidelity facial avatar reconstruction from monocular video with generative priors,

    Y. Bai, Y. Fan, X. Wang, Y. Zhang, J. Sun, C. Yuan, and Y. Shan, “High-fidelity facial avatar reconstruction from monocular video with generative priors,” inCVPR, 2023

    2023

  156. [163]

    Stylesync: High-fidelity generalized and personalized lip sync in style-based generator,

    J. Guan, Z. Zhang, H. Zhou, T. Hu, K. Wang, D. He, H. Feng, J. Liu, E. Ding, Z. Liu,et al., “Stylesync: High-fidelity generalized and personalized lip sync in style-based generator,” inCVPR, 2023

    2023

  157. [164]

    Facechain-imagineid: Freely crafting high- fidelity diverse talking faces from disentangled audio,

    C. Xu, Y. Liu, J. Xing, W. Wang, M. Sun, J. Dan, T. Huang, S. Li, Z.-Q. Cheng, Y. Tai,et al., “Facechain-imagineid: Freely crafting high- fidelity diverse talking faces from disentangled audio,” inCVPR, 2024

    2024

  158. [165]

    Emoportraits: Emotion-enhanced multi- modal one-shot head avatars,

    N. Drobyshev, A. B. Casademunt, K. Vougioukas, Z. Landgraf, S. Petridis, and M. Pantic, “Emoportraits: Emotion-enhanced multi- modal one-shot head avatars,” inCVPR, 2024

    2024

  159. [166]

    Listen, disentangle, and control: Controllable speech-driven talking head generation,

    C. Cai, G. Guo, J. Li, J. Su, C. He, J. Xiao, Y. Chen, L. Dai, and F. Zhu, “Listen, disentangle, and control: Controllable speech-driven talking head generation,”arXiv preprint arXiv:2405.07257, 2024

    2024 arXiv

  160. [167]

    Vasa-1: Lifelike audio-driven talking faces generated in real time,

    S. Xu, G. Chen, Y.-X. Guo, J. Yang, C. Li, Z. Zang, Y. Zhang, X. Tong, and B. Guo, “Vasa-1: Lifelike audio-driven talking faces generated in real time,”arXiv preprint arXiv:2404.10667, 2024

    2024 arXiv

  161. [168]

    Expressive speech-driven facial animation with controllable emotions,

    Y. Chen, J. Zhao, and W.-Q. Zhang, “Expressive speech-driven facial animation with controllable emotions,” inICMEW, 2023

    2023

  162. [169]

    Media2face: Co-speech facial animation generation with multi-modality guidance,

    Q. Zhao, P. Long, Q. Zhang, D. Qin, H. Liang, L. Zhang, Y. Zhang, J. Yu, and L. Xu, “Media2face: Co-speech facial animation generation with multi-modality guidance,” inSIGGRAPH, 2024

    2024

  163. [170]

    Audio- driven co-speech gesture video generation,

    X. Liu, Q. Wu, H. Zhou, Y. Du, W. Wu, D. Lin, and Z. Liu, “Audio- driven co-speech gesture video generation,” inNeurIPS, vol. 35, 2022

    2022

  164. [171]

    Make your actor talk: Generalizable and high-fidelity lip sync with motion and appearance disentanglement,

    R. Yu, T. He, A. Zeng, Y. Wang, J. Guo, X. Tan, C. Liu, J. Chen, and J. Bian, “Make your actor talk: Generalizable and high-fidelity lip sync with motion and appearance disentanglement,”arXiv preprint arXiv:2406.08096, 2024

    2024 arXiv

  165. [172]

    Vast: Vivify your talking avatar via zero-shot expressive facial style transfer,

    L. Chen, Z. Wu, R. Li, W. Bao, J. Ling, X. Tan, and S. Zhao, “Vast: Vivify your talking avatar via zero-shot expressive facial style transfer,” in ICCV, 2023

    2023

  166. [173]

    wav2vec 2.0: A framework for self-supervised learning of speech representations,

    A. Baevski, Y. Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” in NeurIPS, 2020

    2020

  167. [174]

    Realtalk: Real-time and realistic audio-driven face generation with 3d facial prior-guided identity alignment network,

    X. Ji, C. Lin, Z. Ding, Y. Tai, J. Yang, J. Zhu, X. Hu, J. Zhang, D. Luo, and C. Wang, “Realtalk: Real-time and realistic audio-driven face generation with 3d facial prior-guided identity alignment network,” arXiv preprint arXiv:2406.18284, 2024

    2024 arXiv

  168. [175]

    R2-talker: Realistic real-time talking head synthesis with hash grid landmarks encoding and progressive multilayer conditioning,

    Z. Ye, L. Zhang, D. Zeng, Q. Lu, and N. Jiang, “R2-talker: Realistic real-time talking head synthesis with hash grid landmarks encoding and progressive multilayer conditioning,”arXiv preprint arXiv:2312.05572, 2023

    2023 arXiv

  169. [176]

    Conditional image- to-video generation with latent flow diffusion models,

    H. Ni, C. Shi, K. Li, S. X. Huang, and M. R. Min, “Conditional image- to-video generation with latent flow diffusion models,” inCVPR, 2023

    2023

  170. [177]

    End-to-end generation of talking faces from noisy speech,

    S. E. Eskimez, R. K. Maddox, C. Xu, and Z. Duan, “End-to-end generation of talking faces from noisy speech,” inICASSP, 2020

    2020

  171. [178]

    Gaussiantalker: Real-time high-fidelity talking head synthesis with audio-driven 3d gaussian splatting,

    K. Cho, J. Lee, H. Yoon, Y. Hong, J. Ko, S. Ahn, and S. Kim, “Gaussiantalker: Real-time high-fidelity talking head synthesis with audio-driven 3d gaussian splatting,”arXiv preprint arXiv:2404.16012, 2024

    2024 arXiv

  172. [179]

    Large-scale multilingual audio visual dubbing,

    Y. Yang, B. Shillingford, Y. Assael, M. Wang, W. Liu, Y. Chen, Y. Zhang, E. Sezener, L. C. Cobo, M. Denil, Y. Aytar, and N. de Fre- itas, “Large-scale multilingual audio visual dubbing,”arXiv preprint arXiv:2011.03530, 2020

    2011 arXiv

  173. [180]

    Towards robust blind face restoration with codebook lookup transformer,

    S. Zhou, K. Chan, C. Li, and C. C. Loy, “Towards robust blind face restoration with codebook lookup transformer,” inNeurIPS, 2022

    2022

  174. [181]

    Kalman-inspired feature propagation for video face super-resolution,

    R. Feng, C. Li, and C. C. Loy, “Kalman-inspired feature propagation for video face super-resolution,” inECCV, 2024

    2024

  175. [182]

    Streamdiffusion: A pipeline-level solution for real-time interactive generation,

    A. Kodaira, C. Xu, T. Hazama, T. Yoshimoto, K. Ohno, S. Mit- suhori, S. Sugano, H. Cho, Z. Liu, and K. Keutzer, “Streamdiffusion: A pipeline-level solution for real-time interactive generation,”arXiv preprint arXiv:2312.12491, 2023

    2023 arXiv

  176. [183]

    Looking backward: Streaming video-to-video translation with feature banks,

    F. Liang, A. Kodaira, C. Xu, M. Tomizuka, K. Keutzer, and D. Mar- culescu, “Looking backward: Streaming video-to-video translation with feature banks,”arXiv preprint arXiv:2405.15757, 2024

    2024 arXiv

  177. [184]

    Adversarial diffusion distillation,

    A. Sauer, D. Lorenz, A. Blattmann, and R. Rombach, “Adversarial diffusion distillation,” inECCV, 2024

    2024

  178. [185]

    Motion consistency model: Accelerating video diffusion with disentangled motion-appearance distillation,

    Y. Zhai, K. Lin, Z. Yang, L. Li, J. Wang, C.-C. Lin, D. Doermann, J. Yuan, and L. Wang, “Motion consistency model: Accelerating video diffusion with disentangled motion-appearance distillation,” in NeurIPS, 2024

    2024

  179. [186]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium,

    M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” inNeurIPS, 2017

    2017

  180. [187]

    On aliased resizing and surprising subtleties in gan evaluation,

    G. Parmar, R. Zhang, and J.-Y. Zhu, “On aliased resizing and surprising subtleties in gan evaluation,” inCVPR, 2022

    2022

  181. [188]

    Image quality assessment: From error visibility to structural similarity,

    Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,”IEEE TIP, vol. 13, no. 4, pp. 600–612, 2004

    2004

  182. [189]

    Image quality metrics: Psnr vs. ssim,

    A. Hore and D. Ziou, “Image quality metrics: Psnr vs. ssim,” inICPR, 2010

    2010

  183. [190]

    The unreasonable effectiveness of deep features as a perceptual metric,

    R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in CVPR, 2018

    2018

  184. [191]

    A no-reference perceptual image sharpness metric based on a cumulative probability of blur detection,

    N. D. Narvekar and L. J. Karam, “A no-reference perceptual image sharpness metric based on a cumulative probability of blur detection,” in QoMEX, 2009

    2009

  185. [192]

    A no-reference image blur metric based on the cumulative probability of blur detection (cpbd),

    N. D. Narvekar and L. J. Karam, “A no-reference image blur metric based on the cumulative probability of blur detection (cpbd),”IEEE TIP, vol. 20, no. 9, pp. 2678–2683, 2011

    2011

  186. [193]

    Making a “completely blind

    A. Mittal, R. Soundararajan, and A. C. Bovik, “Making a “completely blind” image quality analyzer,”IEEE Signal Process. Lett., vol. 20, no. 3, pp. 209–212, 2013

    2013

  187. [194]

    Towards accurate generative models of video: A new metric & challenges,

    T. Unterthiner, S. Van Steenkiste, K. Kurach, R. Marinier, M. Michal- ski, and S. Gelly, “Towards accurate generative models of video: A new metric & challenges,”arXiv preprint arXiv:1812.01717, 2018

    2018 arXiv

  188. [195]

    Speech gesture generation from the trimodal context of text, audio, and speaker identity,

    Y. Yoon, B. Cha, J.-H. Lee, M. Jang, J. Lee, J. Kim, and G. Lee, “Speech gesture generation from the trimodal context of text, audio, and speaker identity,”TOG, vol. 39, no. 6, 2020

    2020

  189. [196]

    Mocogan: Decompos- ing motion and content for video generation,

    S. Tulyakov, M.-Y. Liu, X. Yang, and J. Kautz, “Mocogan: Decompos- ing motion and content for video generation,” inCVPR, 2018

    2018

  190. [197]

    Structure and content-guided video synthesis with diffusion models,

    P. Esser, J. Chiu, P. Atighehchian, J. Granskog, and A. Germanidis, “Structure and content-guided video synthesis with diffusion models,” in ICCV, 2023

    2023

  191. [198]

    Improved techniques for training single-image gans,

    T. Hinz, M. Fisher, O. Wang, and S. Wermter, “Improved techniques for training single-image gans,” inWACV, 2021

    2021

  192. [199]

    Learning hierarchical cross-modal association for co-speech gesture generation,

    X. Liu, Q. Wu, H. Zhou, Y. Xu, R. Qian, X. Lin, X. Zhou, W. Wu, B. Dai, and B. Zhou, “Learning hierarchical cross-modal association for co-speech gesture generation,” inCVPR, 2022

    2022

  193. [200]

    Audio2gestures: Generating diverse gestures from speech audio with conditional variational autoencoders,

    J. Li, D. Kang, W. Pei, X. Zhe, Y. Zhang, Z. He, and L. Bao, “Audio2gestures: Generating diverse gestures from speech audio with conditional variational autoencoders,” inICCV, 2021

    2021

  194. [201]

    Articulated human detection with flexible mixtures of parts,

    Y. Yang and D. Ramanan, “Articulated human detection with flexible mixtures of parts,”IEEE TPAMI, vol. 35, no. 12, pp. 2878–2890, 2013

    2013

  195. [202]

    First order motion model for image animation,

    A. Siarohin, S. Lathuilière, S. Tulyakov, E. Ricci, and N. Sebe, “First order motion model for image animation,” inNeurIPS, 2019

    2019

  196. [203]

    Thin-plate spline motion model for image animation,

    J. Zhao and H. Zhang, “Thin-plate spline motion model for image animation,” inCVPR, 2022. XUE et al.: HUMAN MOTION VIDEO GENERATION: A SURVEY 20

    2022

  197. [204]

    Exploring video quality assessment on user generated contents from aesthetic and technical perspectives,

    H. Wu, E. Zhang, L. Liao, C. Chen, J. Hou, A. Wang, W. Sun, Q. Yan, and W. Lin, “Exploring video quality assessment on user generated contents from aesthetic and technical perspectives,” inICCV, 2023

    2023

  198. [205]

    Vbench: Comprehensive benchmark suite for video generative models,

    Z. Huang, Y. He, J. Yu, F. Zhang, C. Si, Y. Jiang, Y. Zhang, T. Wu, Q. Jin, N. Chanpaisit,et al., “Vbench: Comprehensive benchmark suite for video generative models,” inCVPR, 2024

    2024

  199. [206]

    Evalcrafter: Benchmarking and evaluating large video generation models,

    Y. Liu, X. Cun, X. Liu, X. Wang, Y. Zhang, H. Chen, Y. Liu, T. Zeng, R. Chan, and Y. Shan, “Evalcrafter: Benchmarking and evaluating large video generation models,” inCVPR, 2024

    2024

  200. [207]

    Learning high fidelity depths of dressed humans by watching social media dance videos,

    Y. Jafarian and H. S. Park, “Learning high fidelity depths of dressed humans by watching social media dance videos,” inCVPR, 2021

    2021

  201. [209]

    Animate-x: Universal character im- age animation with enhanced motion representation,

    S. Tan, B. Gong, X. Wang, S. Zhang, D. Zheng, R. Zheng, K. Zheng, J. Chen, and M. Yang, “Animate-x: Universal character im- age animation with enhanced motion representation,”arXiv preprint arXiv:2410.10306, 2024

    2024 arXiv

  202. [210]

    Vfhq: A high- quality dataset and benchmark for video face super-resolution,

    L. Xie, X. Wang, H. Zhang, C. Dong, and Y. Shan, “Vfhq: A high- quality dataset and benchmark for video face super-resolution,” in CVPRW, 2022

    2022

  203. [211]

    Celebv-hq: A large-scale video facial attributes dataset,

    H. Zhu, W. Wu, W. Zhu, L. Jiang, S. Tang, L. Zhang, Z. Liu, and C. C. Loy, “Celebv-hq: A large-scale video facial attributes dataset,” in ECCV, 2022

    2022

  204. [212]

    Lsa-t: The first continuous argentinian sign language dataset for sign language translation,

    P. Dal Bianco, G. Ríos, F. Ronchetti, F. Quiroga, O. Stanchi, W. Haspe- rué, and A. Rosete, “Lsa-t: The first continuous argentinian sign language dataset for sign language translation,” inIBERAMIA, 2022

    2022

  205. [213]

    Celebv-text: A large-scale facial text-video dataset,

    J. Yu, H. Zhu, L. Jiang, C. C. Loy, W. Cai, and W. Wu, “Celebv-text: A large-scale facial text-video dataset,” inCVPR, 2023

    2023

  206. [214]

    Responsive listening head generation: A benchmark dataset and baseline,

    M. Zhou, Y. Bai, W. Zhang, T. Yao, T. Zhao, and T. Mei, “Responsive listening head generation: A benchmark dataset and baseline,” in ECCV, 2022

    2022

  207. [215]

    Motion representations for articulated animation,

    A. Siarohin, O. Woodford, J. Ren, M. Chai, and S. Tulyakov, “Motion representations for articulated animation,” inCVPR, 2021

    2021

  208. [216]

    Open-domain sign language translation learned from online video,

    B. Shi, D. Brentari, G. Shakhnarovich, and K. Livescu, “Open-domain sign language translation learned from online video,” inEMNLP, 2022

    2022

  209. [217]

    A large-scale multimodal dataset for continuous american sign language,

    A. Duarte, S. Palaskar, L. Ventura, D. Ghadiyaram, K. DeHaan, F. Metze, J. Torres, and X. Giro-i Nieto, “A large-scale multimodal dataset for continuous american sign language,” inCVPR, 2021

    2021

  210. [218]

    Learning to answer questions in dynamic audio-visual scenarios,

    G. Li, Y. Wei, Y. Tian, C. Xu, J.-R. Wen, and D. Hu, “Learning to answer questions in dynamic audio-visual scenarios,” inCVPR, 2022

    2022

  211. [219]

    Headset: Human emotion awareness under partial occlusions multimodal dataset,

    F. G. Lohesara, D. R. Freitas, C. Guillemot, K. Eguiazarian, and S. Knorr, “Headset: Human emotion awareness under partial occlusions multimodal dataset,”TVCG, vol. 29, no. 11, pp. 4686–4696, 2023

    2023

  212. [220]

    Tsa-net: Tube self- attention network for action quality assessment,

    S. Wang, D. Yang, P. Zhai, C. Chen, and L. Zhang, “Tsa-net: Tube self- attention network for action quality assessment,” inACM MM, 2021

    2021

  213. [221]

    Real-time high-resolution background matting,

    S. Lin, A. Ryabtsev, S. Sengupta, B. L. Curless, S. M. Seitz, and I. Kemelmacher-Shlizerman, “Real-time high-resolution background matting,” inCVPR, 2021

    2021

  214. [222]

    3dyoga90: A hierarchical video dataset for yoga pose under- standing,

    S. Kim, “3dyoga90: A hierarchical video dataset for yoga pose under- standing,” arXiv preprint arXiv:2310.10131, 2023

    2023 arXiv

  215. [223]

    Fine-grained activities of people worldwide,

    J. Byrne, G. Castanon, Z. Li, and G. Ettinger, “Fine-grained activities of people worldwide,” inWACV, 2023

    2023

  216. [224]

    Mevid: Multi-view extended videos with identities for video person re-identification,

    D. Davila, D. Du, B. Lewis, C. Funk, J. Van Pelt, R. Collins, K. Corona, M. Brown, S. McCloskey, A. Hoogs, and B. Clipp, “Mevid: Multi-view extended videos with identities for video person re-identification,” in WACV, 2023

    2023

  217. [225]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation,

    J. Z. Wu, Y. Ge, X. Wang, S. W. Lei, Y. Gu, Y. Shi, W. Hsu, Y. Shan, X. Qie, and M. Z. Shou, “Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation,” inICCV, 2023

    2023

  218. [226]

    Video-ChatGPT: Towards detailed video understanding via large vision and language models,

    M. Maaz, H. Rasheed, S. Khan, and F. Khan, “Video-ChatGPT: Towards detailed video understanding via large vision and language models,” inACL, 2024

    2024

  219. [227]

    Can i trust your answer? visually grounded video question answering,

    J. Xiao, A. Yao, Y. Li, and T.-S. Chua, “Can i trust your answer? visually grounded video question answering,” inCVPR, 2024

    2024

  220. [228]

    Brace: The breakdancing competition dataset for dance motion synthesis,

    D. Moltisanti, J. Wu, B. Dai, and C. C. Loy, “Brace: The breakdancing competition dataset for dance motion synthesis,” inECCV, 2022. VIII. Biography Section Haiwei Xue received his B.S. degree from the Guangdong University of Foreign Studies in 2022. He is currently pursuing a...

    2022

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.