Pith. sign in

REVIEW 4 major objections 6 minor 3 cited by

Animate-X++ claims to be the first universal character image animation framework that can animate generic cartoon and anthropomorphic images without strict pose alignment, while also generating text-driven dynamic backgrounds.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Animate-X++ turns cartoon images into pose-driven animations with text-controlled moving backgrounds, claiming state-of-the-art results on a new synthetic anthropomorphic benchmark.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A plausible extension of Animate-X with thorough ablations, but the headline SOTA claim on A2Bench is confounded by the rescaled-pose protocol. the 4 major comments →

arxiv 2508.09454 v1 pith:XTUWLJ32 submitted 2025-08-13 cs.CV

Animate-X++: Universal Character Image Animation with Dynamic Backgrounds

Shuai Tan , Biao Gong , Zhuoxin Liu , Yan Wang , Xi Chen , Yifan Feng , Hengshuang Zhao This is my paper
classification cs.CV
keywords character image animationanthropomorphic charactersdiffusion transformerpose indicatordynamic backgroundstext-image-to-videoidentity preservationmotion transfer
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to establish that a single diffusion-transformer model can animate not just humans but arbitrary anthropomorphic characters—cartoons, game figures, objects with faces—using a driving video's motion, without requiring the reference image and pose skeleton to be aligned. It also claims that the same model can, on text instruction, animate the background scene rather than leaving it static. The authors argue that previous models fail on such characters because they treat pose as a rigid skeleton to be replicated, so they distort the character's identity in favor of matching the pose. They introduce a Pose Indicator that extracts motion both implicitly via CLIP features of the driving video and explicitly via pose transformations simulated during training, plus a multi-task training scheme with partial parameter updates that adds a text-image-to-video objective. If true, this would broaden character animation to the non-human characters common in gaming and entertainment and make outputs noticeably more alive.

Core claim

Animate-X++ is built on a Diffusion Transformer (DiT) denoising network initialized from a large video generation model. Given a reference image and a driving video, it extracts an explicit pose sequence with DWPose and an implicit motion descriptor with a CLIP image encoder. The Implicit Pose Indicator (IPI) uses a learnable query plus pose keypoint embeddings to attend over CLIP features, capturing motion patterns and temporal relations that skeletons miss. The Explicit Pose Indicator (EPI) simulates misalignments during training by realigning the driving pose to an anchor pose from a pool and rescaling body-part lengths, so the model learns to follow approximate motion rather than exact j

What carries the argument

The Pose Indicator, consisting of the Implicit Pose Indicator (IPI) and the Explicit Pose Indicator (EPI), is the central mechanism. IPI fuses DWPose keypoint queries with a learnable query and attends over CLIP visual features of the driving video, capturing the gist of motion beyond skeleton positions. EPI perturbs the pose during training by realigning it to an anchor pose and rescaling body parts, teaching the model to be robust to shape mismatches between the reference image and the driving pose. A secondary mechanism is the multi-task, partial-parameter training strategy: on the text-image-to-video task the pose modules are frozen and only LoRA parameters are updated, allowing text-dri

Load-bearing premise

The quantitative evidence stands on A2Bench, a benchmark whose reference dance videos are themselves generated by a commercial AI pipeline and screened by hand for pose accuracy, so the measured superiority is only as strong as that synthetic ground truth is valid.

What would settle it

Run the model on the roughly 400 image-video pairs from A2Bench that were excluded because DWPose could not extract accurate poses, while giving it a human dance video as the motion source; if human raters find that identity is not preserved or motion is not followed on a substantial share of these hardest cases, the claim that strict pose alignment is unnecessary would be directly contradicted.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A single trained model can animate anthropomorphic characters from games, films, and posters without per-character retraining or pose-aligned reference images.
  • The same model can generate text-driven background dynamics, such as light changes, glowing auras, or ocean motion, concurrently with character animation.
  • Because the model is trained only on human dance videos and generic text-video pairs, it demonstrates that motion understanding from human data can transfer to non-human body plans when the motion representation is made flexible.
  • On human benchmarks such as TikTok and Fashion, the method matches or exceeds previous state-of-the-art, so the changes do not sacrifice human animation quality.
  • The new A2Bench benchmark, organized into three difficulty levels, provides a common testbed for comparing future anthropomorphic animation methods.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the central claim holds, character animation pipelines in games and short-video production could be shortened to a single reference image and any dance clip, removing the need for rigging or pose retargeting for each character.
  • The IPI's use of CLIP video features suggests that motion descriptors could be learned from other cross-modal sources, such as audio or text descriptions of motion, and then combined with pose skeletons for even less human-centric control.
  • A natural stress test is to evaluate on the roughly 400 A2Bench image-video pairs that were excluded because DWPose could not produce accurate poses; success there would directly confirm that strict pose alignment is unnecessary.
  • The A2Bench ground truth is itself generated by a commercial AI pipeline, so an independent benchmark with real recorded or hand-animated anthropomorphic characters would make the quantitative superiority claim more robust.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Animate-X++, a diffusion-transformer (DiT) model for character image animation that targets anthropomorphic characters and adds text-driven background dynamics. The method combines an Implicit Pose Indicator (IPI), which extracts motion-related information from CLIP features of the driving video, with an Explicit Pose Indicator (EPI), which trains the pose encoder on realigned and rescaled poses to tolerate body-shape mismatch. A multi-task training strategy with partial parameter updates (LoRA) is used to add a Text-Image-to-Video (TI2V) objective. The authors introduce A2Bench, a synthetic benchmark of 500 anthropomorphic character images and dance videos generated by GPT-4o and Kling AI, and report quantitative comparisons on A2Bench and on the TikTok/Fashion human benchmarks, along with ablations. The central claim is that Animate-X++ generalizes pose-driven animation to arbitrary cartoon/anthropomorphic images without strict pose alignment.

Significance. If fully supported, the contribution would be practically valuable: EPI's pose-rescaling augmentation is a sensible way to mitigate human/non-human pose mismatch, IPI's CLIP-based implicit motion summary complements DWPose skeletons, and LoRA-based multi-task training is a clean approach to adding background dynamics without degrading animation. The authors also make an effort to hold A2Bench out from training and to disclose its construction pipeline. However, the main quantitative evidence for the headline capability currently rests on a benchmark protocol that is not a valid fidelity test, and the gains on the human benchmarks are small and reported without uncertainty estimates. The dynamic-background contribution is supported only qualitatively. With the measurement issues addressed, the work has clear potential, but the evaluation needs to be repaired before the claims can be accepted.

major comments (4)
  1. [Section IV-B, Table I] The 'rescaled pose setting' is not a valid reconstruction test. Each method receives a pose p_a^b that has been realigned to a randomly chosen human body shape, while the ground-truth video is the original Kling-generated anthropomorphic character with its own non-human proportions. A model that faithfully follows p_a^b cannot simultaneously match the ground-truth frames, so PSNR/SSIM/LPIPS/FVD no longer measure fidelity to the reference video; they reward methods that ignore part of the input. Because EPI is trained on exactly such realigned/rescaled poses, this protocol aligns with the proposed method's inductive bias rather than with a neutral measure of universal capability. The claim in §IV-B that Animate-X++ 'markedly surpasses all comparative methods' is therefore unsupported by Table I. The evaluation should be redone with the original anthropomorphic pose sequence (or with a gro
  2. [Section III-E and Section IV-B, Tables II and X] A2Bench's reference videos are generated by Kling AI, and only 100 of the 500 videos are manually screened for accurate DWPose poses. The screening criteria and inter-annotator reliability are not reported, so it is unclear whether the 100-video test set is representative or whether it preferentially selects human-like characters that favor methods trained on human dance data. Additionally, the metric PSNR* is introduced in Table I with the note 'using the modified metric [84] to avoid numerical overflow,' but the modification is never defined. This makes the central numbers unreproducible. Please define PSNR*, report the full screening protocol, and validate A2Bench with human evaluation or an independently collected anthropomorphic benchmark.
  3. [Tables III and VI] On the standard human benchmarks, the reported advantage over Unianimate-DiT is small (e.g., Table VI: PSNR 31.05 vs 30.95, SSIM 0.817 vs 0.815, FVD 135.20 vs 136.10). No error bars, confidence intervals, or significance tests are provided for any metric in any table. Given the small margins and the 100-video test sets, the claim that Animate-X++ 'exceeds' state-of-the-art methods on human datasets is not statistically established. Please report per-metric uncertainty and, if possible, multiple seeds or bootstrapped intervals.
  4. [Section III-D, Fig. 16, Table IV] The text-driven background dynamics contribution is demonstrated only qualitatively. No quantitative metric is reported for prompt adherence, background motion, or the effect of the TI2V task on the generated background. The user study (Table IV) asks about identity preservation, temporal consistency, and visual quality, but not about background dynamics or text alignment. Since the title and abstract foreground dynamic backgrounds as a core contribution, please add a quantitative evaluation on held-out TI2V pairs (e.g., prompt-video alignment scores and background motion metrics) or explicitly scope this as a qualitative capability.
minor comments (6)
  1. [Tables I, II, VII] The L1 metric is defined as mean absolute difference, so lower is better, but the table headers show an upward arrow. The arrow direction should be reversed.
  2. [Tables III and VI] The row labeled 'Unianimate-DiT' is cited as [66], which is StableAnimator; the correct reference is [80]. This also affects the reference numbering in the captions.
  3. [Section V, Limitation and Future Work] The limitation section says, 'even though we replace the transformer with a more efficient Mamba model for temporal modeling,' but the method described in Section III-C and Fig. 2 uses DiT blocks with self-attention, not Mamba. This sentence appears to be inconsistent with the rest of the manuscript.
  4. [Fig. 8] The notation p_a^b is used in the comparison-setting figure but is not formally defined in the text. Please define the realignment/rescaling mapping and how the random human pose P_b is selected.
  5. [References] There are duplicate references: [3] and [83] are the same work, [50] and [91] are the same, and [62] and [84] are the same. Please consolidate them.
  6. [Table V] The rows in Table V are misaligned; for example, the '90%' row entries do not line up with the columns. Please reformat.

Circularity Check

2 steps flagged

The headline A2Bench SOTA is partially circular: its 'rescaled pose' test inputs are generated by the same realign-and-rescale procedure used to train the EPI module, and the key EPI hyperparameter λ is tuned on A2Bench itself.

specific steps
  1. other [Section III-B (Explicit Pose Indicator) and Section IV-B (Quantitative Results, rescaled pose setting)]
    "The transformed pose I p n retains the same motion as the sampled pose I p but has a body shape similar to the anchor pose I p anchor. This simulates scenarios during inference where there are body shape differences between the reference image and the driving pose, enabling the model to generalize to such cases. ... For each case in A 2Bench (i.e., a reference image I a and a pose P a), we randomly select one human’s pose image P b and align the anthropomorphic character’s pose P a to it, such that the aligned pose p a b retains the movements of P a but has the same body shape (fat/thin, tall/"

    The A2Bench evaluation inputs are generated by the exact operation EPI uses during training: realign the driving pose to a random anchor human pose, then rescale body parts. Animate-X++ is trained under this transformation with probability λ>98%, so Table I measures the model's ability to invert its own training augmentation rather than an independent external benchmark. Competing methods are not trained on this synthetic distribution, so they are systematically disadvantaged. Moreover, the ground-truth video retains the anthropomorphic body shape while the input pose is human-aligned, so no model can simultaneously match the pose and the reference; PSNR/SSIM/LPIPS/FVD then partly measure the induced pose mismatch rather than animation quality.

  2. fitted input called prediction [Section IV-C, Ablation on Explicit Pose Indicator, Table V]
    "The results presented in Tab. V suggest that a high λ performs better on A 2Bench, i.e., it performs better when the reference image and pose image are not aligned, but harms performance on the TikTok dataset, i.e., when the reference image and pose image are strictly aligned."

    The key hyperparameter λ of the EPI module is selected by monitoring A2Bench metrics in Table V, and the final model then uses a high λ (over 98%) because it maximizes A2Bench performance. Reporting the same A2Bench numbers as evidence of state-of-the-art results is therefore test-set configuration: the model has been fitted to the benchmark, so the quantitative superiority on A2Bench is not an independent prediction but a consequence of tuning on that benchmark.

full rationale

The paper's central technical derivation — Pose Indicator (IPI + EPI), DiT backbone, multi-task partial-parameter training — is not circular in the sense of assuming what it proves: IPI is a learned CLIP-conditioned query extraction, EPI is a data-augmentation scheme, and the multi-task loss is a standard combination of animation and TI2V objectives. However, the headline quantitative claim on A2Bench is partially circular. The A2Bench 'rescaled pose setting' constructs test inputs by realigning an anthropomorphic pose onto a random human body shape, which is precisely the EPI training augmentation (pose realignment + pose rescale). Thus the benchmark is aligned with the method's inductive bias, and the reported 'markedly surpasses' margins in Table I are partly by construction. Additionally, the EPI probability λ is ablated directly on A2Bench and then the same A2Bench numbers are presented as the final result, which is test-set hyperparameter selection. These issues affect the strength of the quantitative evidence, but not all evidence: the self-driven A2Bench (Table II), human TikTok/Fashion benchmarks, and qualitative demonstrations provide some independent support for the method's capability. The paper's own stated limitations (hands/faces and non-real-time speed) do not affect circularity. Overall, the central claim retains independent content, but the A2Bench-based SOTA claim is partially reduced by construction, giving a score of 6.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 1 invented entities

The central claim depends on several domain assumptions about CLIP features conveying motion, pose transformations covering cartoon body shapes, the validity of a synthetic benchmark, and human-to-character transfer. The free parameters lambda and alpha are explicitly tuned on the same benchmarks where final results are reported. No code or data are provided, so none of these assumptions can be independently checked.

free parameters (3)
  • lambda (pose transformation probability) = 0.98
    Chosen by comparing performance on the test benchmarks A2Bench and TikTok in Table V; a high lambda favors A2Bench, a lower lambda favors strictly aligned human data.
  • alpha (IPI residual weight) = 1.0
    Set empirically from the sweep in Fig. 14; controls how strongly the implicit pose indicator features are injected.
  • p_ti2v (TI2V task sampling probability) = not specified
    Described as 'a relatively low probability', set by hand to balance the animation task against background-dynamics learning without degrading pose alignment.
axioms (4)
  • domain assumption CLIP visual features encode a transferable 'gist of motion' from the driving video.
    Section III-B builds the IPI on CLIP features as keys and values, asserting they contain motion patterns and temporal relations. No external analysis or benchmark proves this; the paper only shows that removing IPI degrades results.
  • domain assumption DWPose keypoint rescaling and realignment on human videos spans the body-shape space of anthropomorphic characters.
    Section III-B assumes that randomly transforming human poses (body, leg, arm, neck lengths; adding or dropping parts) simulates the distribution of cartoon body shapes well enough for the model to generalize at inference.
  • domain assumption A2Bench videos generated by Kling AI are valid references for quantitative animation metrics.
    Section III-E constructs ground truth with a commercial image-to-video model and defends it with CLIP scores and a QWen MLLM logical check, but pixel-space metrics against these synthetic videos are not independently validated.
  • domain assumption A model trained only on human dance videos transfers to anthropomorphic characters.
    Sections I and IV claim the model can animate arbitrary X characters despite training solely on human datasets; no multi-domain character training data is used.
invented entities (1)
  • A2Bench benchmark dataset no independent evidence
    purpose: Evaluation resource for animating 500 anthropomorphic characters across three difficulty levels, with generated images and dancing videos.
    Constructed by the authors using GPT-4o prompts and Kling AI text-to-image and image-to-video; not released and not validated by external annotation or third-party evaluation. It is held out from training, so it functions as a test set, but a synthetic, author-controlled one.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Animate-X++: Universal Character Image Animation with Dynamic Backgrounds." pith.science (2026). https://pith.science/paper/XTUWLJ32

@misc{pith2026250809454,
  author       = {Pith},
  title        = {Pith review of: Animate-X++: Universal Character Image Animation with Dynamic Backgrounds},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XTUWLJ32}},
  note         = {Machine review of arXiv:2508.09454}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Character image animation, which generates high-quality videos from a reference image and target pose sequence, has seen significant progress in recent years. However, most existing methods only apply to human figures, which usually do not generalize well on anthropomorphic characters commonly used in industries like gaming and entertainment. Furthermore, previous methods could only generate videos with static backgrounds, which limits the realism of the videos. For the first challenge, our in-depth analysis suggests to attribute this limitation to their insufficient modeling of motion, which is unable to comprehend the movement pattern of the driving video, thus imposing a pose sequence rigidly onto the target character. To this end, this paper proposes Animate-X++, a universal animation framework based on DiT for various character types, including anthropomorphic characters. To enhance motion representation, we introduce the Pose Indicator, which captures comprehensive motion pattern from the driving video through both implicit and explicit manner. The former leverages CLIP visual features of a driving video to extract its gist of motion, like the overall movement pattern and temporal relations among motions, while the latter strengthens the generalization of DiT by simulating possible inputs in advance that may arise during inference. For the second challenge, we introduce a multi-task training strategy that jointly trains the animation and TI2V tasks. Combined with the proposed partial parameter training, this approach achieves not only character animation but also text-driven background dynamics, making the videos more realistic. Moreover, we introduce a new Animated Anthropomorphic Benchmark (A2Bench) to evaluate the performance of Animate-X++ on universal and widely applicable animation images. Extensive experiments demonstrate the superiority and effectiveness of Animate-X++.

Figures

Figures reproduced from arXiv: 2508.09454 by Biao Gong, Hengshuang Zhao, Shuai Tan, Xi Chen, Yan Wang, Yifan Feng, Zhuoxin Liu.

Figure 1
Figure 1. Figure 1: Animations produced by Animate-X++ which extends beyond human to anthropomorphic characters with various body structures, e.g., without limbs, from games, animations, and posters. Notably, our framework brings the background dynamics to life, such as the formation of a glowing aura, dynamic scene relighting, and the ebb and flow of ocean tides. backgrounds and thus provides no supervision for dynamic scene… view at source ↗
Figure 2
Figure 2. Figure 2: (a) The overview of our Animate-X++. Given a reference image I r , we first extract latent feature f r e via VAE encoder E. The proposed Implicit Pose Indicator (IPI) and Explicit Pose Indicator (EPI) produce motion feature fi and pose feature fe, respectively. f r e is concatenated with the noised input ϵ along the channel dimension, then further add to fe and fi. This serves as the input to the diffusion… view at source ↗
Figure 3
Figure 3. Figure 3: More example for EPI. once the reversed denoising stage is complete, the predicted clean latent is passed through the VAE decoder to reconstruct the predicted video in pixel space. B. Pose Indicator To extract motion representations, previous works typically detect the pose keypoints via DWPose [77] from the driven video I d 1:F and further visualize them as pose image I p , which are trained using self-dr… view at source ↗
Figure 5
Figure 5. Figure 5: The difference of training and inference pipeline. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Detailed pipeline for building A2Bench [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Examples from our A2Bench [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: The illustration of comparison settings. [PITH_FULL_IMAGE:figures/full_fig_p007_8.png] view at source ↗
Figure 10
Figure 10. Figure 10: In contrast, Animate-X++ ensures both identity and consistency with the reference image while generating expressive and exaggerated figure motion, rather than simply adopting quasi-static motion of the target character. The com￾parison with StableAnimator and Unianimate-DiT is displayed in [PITH_FULL_IMAGE:figures/full_fig_p009_10.png] view at source ↗
Figure 10
Figure 10. Figure 10: Qualitative comparisons with state-of-the-art methods. [PITH_FULL_IMAGE:figures/full_fig_p010_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Qualitative comparisons with Unianimate in terms of long video generation. [PITH_FULL_IMAGE:figures/full_fig_p011_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Comparison with SOTA methods. TABLE IV USER STUDY RESULTS. Method Moore-AA MimicMotion ControlNeXt MusePose Unianimate Animate-X++ Identity preservation ↑ 60.4% 14.8% 52.0% 31.3% 43.0% 98.5% Temporal consistency ↑ 19.8% 24.9% 36.9% 43.9% 81.1% 93.4% Visual quality ↑ 27.0% 17.2% 40.4% 40.3% 79.3% 95.8% our backbone for conducting the ablation studies on IPI and EPI, which allows for a more efficient valida… view at source ↗
Figure 15
Figure 15. Figure 15: Visualization of ablation study on IPI and EPI. [PITH_FULL_IMAGE:figures/full_fig_p012_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Ablation study on backbone and training strategy. [PITH_FULL_IMAGE:figures/full_fig_p012_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Visualization of the robustness of Animate-X++. Text Level 1Level 2Level 3 [PITH_FULL_IMAGE:figures/full_fig_p014_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Difficulty levels in A2Bench. variations, our method consistently produces satisfactory re￾sults, showcasing its robustness to the input reference images. On the other hand, as illustrated in [PITH_FULL_IMAGE:figures/full_fig_p014_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Prompts, generated images by T2I in A2Bench, and logical answers from QWen. 1 category. These characters are generally simpler to drive, produce fewer artifacts, and have better motion consistency. In contrast, characters that maintain more distinct structural features from humans, such as dragons and ducks in the third row of [PITH_FULL_IMAGE:figures/full_fig_p015_19.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. CoMoVi: Co-Generation of 3D Human Motions and Realistic Videos

    cs.CV 2026-01 unverdicted novelty 7.0

    CoMoVi co-generates 3D human motions and 2D videos synchronously in a single diffusion denoising loop using 3D-to-2D projection and dual-branch diffusion with 3D-2D cross attentions.

  2. Towards 3D-Aware Video Diffusion Models: Render-Free Human Motion Control with Mesh Tokenization

    cs.CV 2026-06 unverdicted novelty 6.0

    Introduces mesh tokenization to condition DiT-based video diffusion models directly on 3D human meshes for motion control without 2D rendering.

  3. EDTalk++: Full Disentanglement for Controllable Talking Head Synthesis

    cs.CV 2025-08 conditional novelty 5.0

    EDTalk++ disentangles talking-head video into four orthogonal motion banks (mouth, pose, eyes, expression) and drives them from either video or audio inputs.

Reference graph

Works this paper leans on

103 extracted references · 34 canonical work pages · cited by 3 Pith papers · 2 internal anchors

  1. [1]

    Animate-x: Universal character image anima- tion with enhanced motion representation,

    S. Tan, B. Gong, X. Wang, S. Zhang, D. Zheng, R. Zheng, K. Zheng, J. Chen, and M. Yang, “Animate-x: Universal character image anima- tion with enhanced motion representation,” inICLR, 2025

    2025

  2. [2]

    Pose guided human video generation,

    C. Yang, Z. Wang, X. Zhu, C. Huang, J. Shi, and D. Lin, “Pose guided human video generation,” inECCV, 2018

    2018

  3. [4]

    Animate any- one: Consistent and controllable image-to-video synthesis for character animation,

    L. Hu, X. Gao, P. Zhang, K. Sun, B. Zhang, and L. Bo, “Animate any- one: Consistent and controllable image-to-video synthesis for character animation,”arXiv:2311.17117, 2023

    Pith/arXiv arXiv 2023

  4. [5]

    Magicanimate: Temporally consistent human image animation using diffusion model,

    Z. Xu, J. Zhang, J. H. Liew, H. Yan, J.-W. Liu, C. Zhang, J. Feng, and M. Z. Shou, “Magicanimate: Temporally consistent human image animation using diffusion model,”arXiv:2311.16498, 2023

    Pith/arXiv arXiv 2023

  5. [6]

    Magicdance: Realistic human dance video generation with motions & facial expressions transfer,

    D. Chang, Y . Shi, Q. Gao, J. Fu, H. Xu, G. Song, Q. Yan, X. Yang, and M. Soleymani, “Magicdance: Realistic human dance video generation with motions & facial expressions transfer,”arXiv:2311.12052, 2023

    Pith/arXiv arXiv 2023

  6. [7]

    Text2human: Text-driven controllable human image generation,

    Y . Jiang, S. Yang, H. Qiu, W. Wu, C. C. Loy, and Z. Liu, “Text2human: Text-driven controllable human image generation,”ACM Transactions on Graphics, 2022

    2022

  7. [8]

    Generative adversarial nets,

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y . Bengio, “Generative adversarial nets,” NeurIPS, 2014

    2014

  8. [9]

    Bidirectionally deformable motion modulation for video-based human pose transfer,

    W.-Y . Yu, L.-M. Po, R. C. Cheung, Y . Zhao, Y . Xue, and K. Li, “Bidirectionally deformable motion modulation for video-based human pose transfer,” inICCV, 2023

    2023

  9. [11]

    Imagen video: High definition video generation with diffusion models,

    J. Ho, W. Chan, C. Saharia, J. Whang, R. Gao, A. Gritsenko, D. P. Kingma, B. Poole, M. Norouzi, D. J. Fleetet al., “Imagen video: High definition video generation with diffusion models,”arXiv:2210.02303, 2022

    Pith/arXiv arXiv 2022

  10. [12]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,”NeurIPS, vol. 33, 2020

    2020

  11. [13]

    An- imatediff: Animate your personalized text-to-image diffusion models without specific tuning,

    Y . Guo, C. Yang, A. Rao, Y . Wang, Y . Qiao, D. Lin, and B. Dai, “An- imatediff: Animate your personalized text-to-image diffusion models without specific tuning,”arXiv:2307.04725, 2023

    Pith/arXiv arXiv 2023

  12. [14]

    High- resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” inCVPR, 2022

    2022

  13. [15]

    Wan: Open and advanced large-scale video generative mod- els,

    T. Wan, “Wan: Open and advanced large-scale video generative mod- els,”arXiv:2503.20314, 2025

    Pith/arXiv arXiv 2025

  14. [16]

    Scalable diffusion models with transformers,

    W. Peebles and S. Xie, “Scalable diffusion models with transformers,” inICCV, 2023

    2023

  15. [17]

    Denoising diffusion implicit mod- els,

    J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit mod- els,” inICLR, 2021

    2021

  16. [18]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models,

    A. Q. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. Mc- grew, I. Sutskever, and M. Chen, “Glide: Towards photorealistic image generation and editing with text-guided diffusion models,” inICML, 2022

    2022

  17. [19]

    Hierarchical text-conditional image generation with clip latents,

    A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierarchical text-conditional image generation with clip latents,” arXiv:2204.06125, 2022

    Pith/arXiv arXiv 2022

  18. [20]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models,

    C. Mou, X. Wang, L. Xie, J. Zhang, Z. Qi, Y . Shan, and X. Qie, “T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models,”arXiv:2302.08453, 2023

    Pith/arXiv arXiv 2023

  19. [21]

    Composer: Creative and controllable image synthesis with composable conditions,

    L. Huang, D. Chen, Y . Liu, Y . Shen, D. Zhao, and J. Zhou, “Composer: Creative and controllable image synthesis with composable conditions,” ICML, 2023

    2023

  20. [22]

    Adding conditional control to text-to-image diffusion models,

    L. Zhang, A. Rao, and M. Agrawala, “Adding conditional control to text-to-image diffusion models,” inICCV, 2023

    2023

  21. [23]

    Survey on leveraging pre-trained generative adversarial networks for image editing and restoration,

    M. Liu, Y . Wei, X. Wu, W. Zuo, and L. Zhang, “Survey on leveraging pre-trained generative adversarial networks for image editing and restoration,”Science China Information Sciences, 2023

    2023

  22. [24]

    Photorealistic text-to-image diffusion models with deep language understanding,

    C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimanset al., “Photorealistic text-to-image diffusion models with deep language understanding,”NeurIPS, 2022

    2022

  23. [25]

    Make-a-video: Text-to-video generation without text-video data,

    U. Singer, A. Polyak, T. Hayes, X. Yin, J. An, S. Zhang, Q. Hu, H. Yang, O. Ashual, O. Gafniet al., “Make-a-video: Text-to-video generation without text-video data,”ICLR, 2023

    2023

  24. [26]

    Modelscope text-to-video technical report,

    J. Wang, H. Yuan, D. Chen, Y . Zhang, X. Wang, and S. Zhang, “Modelscope text-to-video technical report,”arXiv:2308.06571, 2023

    Pith/arXiv arXiv 2023

  25. [27]

    A recipe for scaling up text-to-video generation with text-free videos,

    X. Wang, S. Zhang, H. Yuan, Z. Qing, B. Gong, Y . Zhang, Y . Shen, C. Gao, and N. Sang, “A recipe for scaling up text-to-video generation with text-free videos,” inCVPR, 2024

    2024

  26. [28]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation,

    J. Z. Wu, Y . Ge, X. Wang, S. W. Lei, Y . Gu, Y . Shi, W. Hsu, Y . Shan, X. Qie, and M. Z. Shou, “Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation,” inICCV, 2023

    2023

  27. [29]

    Stablevideo: Text-driven consistency-aware diffusion video editing,

    W. Chai, X. Guo, G. Wang, and Y . Lu, “Stablevideo: Text-driven consistency-aware diffusion video editing,” inICCV, 2023

    2023

  28. [30]

    Pix2video: Video editing using image diffusion,

    D. Ceylan, C.-H. P. Huang, and N. J. Mitra, “Pix2video: Video editing using image diffusion,” inICCV, 2023

    2023

  29. [31]

    Mag- icvideo: Efficient video generation with latent diffusion models,

    D. Zhou, W. Wang, H. Yan, W. Lv, Y . Zhu, and J. Feng, “Mag- icvideo: Efficient video generation with latent diffusion models,” arXiv:2211.11018, 2022

    Pith/arXiv arXiv 2022

  30. [32]

    Latent-shift: Latent diffusion with temporal shift for efficient text-to- video generation,

    J. An, S. Zhang, H. Yang, S. Gupta, J.-B. Huang, J. Luo, and X. Yin, “Latent-shift: Latent diffusion with temporal shift for efficient text-to- video generation,”arXiv:2304.08477, 2023

    Pith/arXiv arXiv 2023

  31. [33]

    Simda: Simple diffusion adapter for efficient video generation,

    Z. Xing, Q. Dai, H. Hu, Z. Wu, and Y .-G. Jiang, “Simda: Simple diffusion adapter for efficient video generation,”arXiv:2308.09710, 2023. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 17

    Pith/arXiv arXiv 2023

  32. [34]

    Hierarchical spatio-temporal decoupling for text-to-video generation,

    Z. Qing, S. Zhang, J. Wang, X. Wang, Y . Wei, Y . Zhang, C. Gao, and N. Sang, “Hierarchical spatio-temporal decoupling for text-to-video generation,”arXiv:2312.04483, 2023

    Pith/arXiv arXiv 2023

  33. [35]

    Instructvideo: Instructing video diffusion models with human feedback,

    H. Yuan, S. Zhang, X. Wang, Y . Wei, T. Feng, Y . Pan, Y . Zhang, Z. Liu, S. Albanie, and D. Ni, “Instructvideo: Instructing video diffusion models with human feedback,”arXiv:2312.12490, 2023

    Pith/arXiv arXiv 2023

  34. [36]

    Style2talker: High-resolution talking head generation with emotion style and art style,

    S. Tan, B. Ji, and Y . Pan, “Style2talker: High-resolution talking head generation with emotion style and art style,” inAAAI, 2024

    2024

  35. [37]

    Check locate rectify: A training-free layout calibration system for text-to- image generation,

    B. Gong, S. Huang, Y . Feng, S. Zhang, Y . Li, and Y . Liu, “Check locate rectify: A training-free layout calibration system for text-to- image generation,” inCVPR, 2024

    2024

  36. [38]

    Dreamvideo: Composing your dream videos with customized subject and motion,

    Y . Wei, S. Zhang, Z. Qing, H. Yuan, Z. Liu, Y . Liu, Y . Zhang, J. Zhou, and H. Shan, “Dreamvideo: Composing your dream videos with customized subject and motion,” inCVPR, 2024

    2024

  37. [39]

    Dreamvideo-2: Zero-shot subject-driven video customization with precise motion control,

    Y . Wei, S. Zhang, H. Yuan, X. Wang, H. Qiu, R. Zhao, Y . Feng, F. Liu, Z. Huang, J. Yeet al., “Dreamvideo-2: Zero-shot subject-driven video customization with precise motion control,”arXiv:2410.13830, 2024

    Pith/arXiv arXiv 2024

  38. [40]

    Mimir: Improving video diffusion models for precise text understanding,

    S. Tan, B. Gong, Y . Feng, K. Zheng, D. Zheng, S. Shi, Y . Shen, J. Chen, and M. Yang, “Mimir: Improving video diffusion models for precise text understanding,”arXiv:2412.03085, 2024

    Pith/arXiv arXiv 2024

  39. [41]

    MotionStone: Decoupled Motion Intensity Modulation with Diffusion Transformer for Image-to-Video Generation

    S. Shi, B. Gong, X. Chen, D. Zheng, S. Tan, Z. Yang, Y . Li, J. He, K. Zheng, J. Chenet al., “Motionstone: Decoupled motion intensity modulation with diffusion transformer for image-to-video generation,” arXiv:2412.05848, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  40. [42]

    Synmotion: Semantic-visual adaptation for motion customized video generation,

    S. Tan, B. Gong, Y . Wei, S. Zhang, Z. Liu, D. Zheng, J. Chen, Y . Wang, H. Ouyang, K. Zheng, and Y . Shen, “Synmotion: Semantic-visual adaptation for motion customized video generation,”arXiv preprint arXiv:2506.23690, 2025

    Pith/arXiv arXiv 2025

  41. [43]

    Mimicmotion: High-quality human motion video generation with confidence-aware pose guidance,

    Y . Zhang, J. Gu, L.-W. Wang, H. Wang, J. Cheng, Y . Zhu, and F. Zou, “Mimicmotion: High-quality human motion video generation with confidence-aware pose guidance,”arXiv:2406.19680, 2024

    Pith/arXiv arXiv 2024

  42. [44]

    Magicpose: Realistic human poses and facial expressions retargeting with identity-aware diffusion,

    D. Chang, Y . Shi, Q. Gao, H. Xu, J. Fu, G. Song, Q. Yan, Y . Zhu, X. Yang, and M. Soleymani, “Magicpose: Realistic human poses and facial expressions retargeting with identity-aware diffusion,” inICML, 2023

    work page 2023

  43. [45]

    Dancetogether! identity-preserving multi- person interactive video generation,

    J. Chen, M. Chen, J. Xu, X. Li, J. Dong, M. Sun, P. Jiang, H. Li, Y . Yang, H. Zhaoet al., “Dancetogether! identity-preserving multi- person interactive video generation,”arXiv:2505.18078, 2025

    Pith/arXiv arXiv 2025

  44. [46]

    Flexiact: Towards flexible action control in heterogeneous scenarios,

    S. Zhang, J. Zhuang, Z. Zhang, Y . Shan, and Y . Tang, “Flexiact: Towards flexible action control in heterogeneous scenarios,” inPro- ceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers, 2025

    work page 2025

  45. [47]

    Stableanimator++: Overcoming pose misalignment and face distortion for human image animation,

    S. Tu, Z. Xing, X. Han, Z.-Q. Cheng, Q. Dai, C. Luo, Z. Wu, and Y .-G. Jiang, “Stableanimator++: Overcoming pose misalignment and face distortion for human image animation,”arXiv:2507.15064, 2025

    Pith/arXiv arXiv 2025

  46. [48]

    EvAnimate: Event-conditioned Image-to-Video Generation for Human Animation

    Q. Qu, M. Li, X. Chen, and T. Liu, “Evanimate: Event-conditioned image-to-video generation for human animation,”arXiv:2503.18552, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  47. [49]

    Dense intrinsic appearance flow for human pose transfer,

    Y . Li, C. Huang, and C. C. Loy, “Dense intrinsic appearance flow for human pose transfer,” inCVPR, 2019

    work page 2019

  48. [53]

    Edtalk: Efficient disentanglement for emotional talking head synthesis,

    S. Tan, B. Ji, M. Bi, and Y . Pan, “Edtalk: Efficient disentanglement for emotional talking head synthesis,” inECCV. Springer, 2024

    work page 2024

  49. [54]

    Latent image animator: Learning to animate images via latent space navigation,

    Y . Wang, D. Yang, F. Bremond, and A. Dantcheva, “Latent image animator: Learning to animate images via latent space navigation,” arXiv:2203.09043, 2022

    Pith/arXiv arXiv 2022

  50. [55]

    Flowvqtalker: High-quality emotional talking face generation through normalizing flow and quantization,

    S. Tan, B. Ji, and Y . Pan, “Flowvqtalker: High-quality emotional talking face generation through normalizing flow and quantization,” inCVPR, 2024

    work page 2024

  51. [56]

    Say anything with any style,

    S. Tan, B. Ji, Y . Ding, and Y . Pan, “Say anything with any style,” in AAAI, 2024

    work page 2024

  52. [57]

    Emmn: Emotional motion memory network for audio-driven emotional talking face generation,

    S. Tan, B. Ji, and Y . Pan, “Emmn: Emotional motion memory network for audio-driven emotional talking face generation,” inICCV, 2023

    work page 2023

  53. [58]

    Expressive talking avatars,

    Y . Pan, S. Tan, S. Cheng, Q. Lin, Z. Zeng, and K. Mitchell, “Expressive talking avatars,”IEEE Transactions on Visualization and Computer Graphics, 2024

    work page 2024

  54. [59]

    Fixtalk: Taming identity leakage for high-quality talking head generation in extreme cases,

    S. Tan, B. Gong, B. Ji, and Y . Pan, “Fixtalk: Taming identity leakage for high-quality talking head generation in extreme cases,” inICCV, 2025

    work page 2025

  55. [60]

    Advancing pose-guided image synthesis with progressive conditional diffusion models,

    F. Shen, H. Ye, J. Zhang, C. Wang, X. Han, and Y . Wei, “Advancing pose-guided image synthesis with progressive conditional diffusion models,” inICLR, 2024. [Online]. Available: https: //openreview.net/forum?id=rHzapPnCgT

    work page 2024

  56. [61]

    Champ: Controllable and consistent human image animation with 3d parametric guidance,

    S. Zhu, J. L. Chen, Z. Dai, Y . Xu, X. Cao, Y . Yao, H. Zhu, and S. Zhu, “Champ: Controllable and consistent human image animation with 3d parametric guidance,” inECCV, 2024

    work page 2024

  57. [62]

    Disco: Disentangled control for referring human dance generation in real world,

    T. Wang, L. Li, K. Lin, C.-C. Lin, Z. Yang, H. Zhang, Z. Liu, and L. Wang, “Disco: Disentangled control for referring human dance generation in real world,”arXiv e-prints, 2023

    work page 2023

  58. [63]

    Adding conditional control to text-to-image diffusion models,

    L. Zhang, A. Rao, and M. Agrawala, “Adding conditional control to text-to-image diffusion models,” 2023

    2023

  59. [64]

    Magicanimate: Temporally consistent human image animation using diffusion model,

    Z. Xu, J. Zhang, J. H. Liew, H. Yan, J.-W. Liu, C. Zhang, J. Feng, and M. Z. Shou, “Magicanimate: Temporally consistent human image animation using diffusion model,” inarXiv, 2023

    work page 2023

  60. [65]

    Attention is all you need,

    A. Vaswani, “Attention is all you need,”NeurIPS, 2017

    2017

  61. [66]

    Stableanimator: High-quality identity-preserving human image anima- tion,

    S. Tu, Z. Xing, X. Han, Z.-Q. Cheng, Q. Dai, C. Luo, and Z. Wu, “Stableanimator: High-quality identity-preserving human image anima- tion,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025

    work page 2025

  62. [67]

    Vision mamba: Efficient visual representation learning with bidirectional state space model,

    L. Zhu, B. Liao, Q. Zhang, X. Wang, W. Liu, and X. Wang, “Vision mamba: Efficient visual representation learning with bidirectional state space model,”arXiv:2401.09417, 2024

    Pith/arXiv arXiv 2024

  63. [68]

    Plainmamba: Improving non-hierarchical mamba in visual recognition,

    C. Yang, Z. Chen, M. Espinosa, L. Ericsson, Z. Wang, J. Liu, and E. J. Crowley, “Plainmamba: Improving non-hierarchical mamba in visual recognition,”arXiv:2403.17695, 2024

    Pith/arXiv arXiv 2024

  64. [69]

    Vmamba: Visual state space model,

    Y . Liu, Y . Tian, Y . Zhao, H. Yu, L. Xie, Y . Wang, Q. Ye, and Y . Liu, “Vmamba: Visual state space model,”arXiv:2401.10166, 2024

    Pith/arXiv arXiv 2024

  65. [70]

    Video mamba suite: State space model as a versatile alternative for video understanding,

    G. Chen, Y . Huang, J. Xu, B. Pei, Z. Chen, Z. Li, J. Wang, K. Li, T. Lu, and L. Wang, “Video mamba suite: State space model as a versatile alternative for video understanding,”arXiv:2403.09626, 2024

    Pith/arXiv arXiv 2024

  66. [71]

    Videomamba: State space model for efficient video understanding,

    K. Li, X. Li, Y . Wang, Y . He, Y . Wang, L. Wang, and Y . Qiao, “Videomamba: State space model for efficient video understanding,” arXiv:2403.06977, 2024

    Pith/arXiv arXiv 2024

  67. [72]

    Mamba: Linear-time sequence modeling with selective state spaces,

    A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,”arXiv:2312.00752, 2023

    Pith/arXiv arXiv 2023

  68. [73]

    Efficiently modeling long sequences with structured state spaces,

    A. Gu, K. Goel, and C. R ´e, “Efficiently modeling long sequences with structured state spaces,”arXiv:2111.00396, 2021

    Pith/arXiv arXiv 2021

  69. [74]

    Unianimate: Taming unified video diffusion models for consistent human image animation,

    X. Wang, S. Zhang, C. Gao, J. Wang, X. Zhou, Y . Zhang, L. Yan, and N. Sang, “Unianimate: Taming unified video diffusion models for consistent human image animation,”arXiv:2406.01188, 2024

    Pith/arXiv arXiv 2024

  70. [75]

    Hypermotion: Dit-based pose-guided human image animation of complex motions,

    S. Xu, S. Zheng, Z. Wang, H. Yu, J. Chen, H. Zhang, B. Li, and P.-T. Jiang, “Hypermotion: Dit-based pose-guided human image animation of complex motions,”arXiv:2505.22977, 2025

    arXiv 2025

  71. [76]

    Auto-encoding variational bayes,

    D. P. Kingma, “Auto-encoding variational bayes,”arXiv:1312.6114, 2013

    Pith/arXiv arXiv 2013

  72. [77]

    Effective whole-body pose estimation with two-stages distillation,

    Z. Yang, A. Zeng, C. Yuan, and Y . Li, “Effective whole-body pose estimation with two-stages distillation,” inICCV, 2023

    work page 2023

  73. [78]

    Openflamingo: An open-source framework for training large autoregressive vision- language models,

    A. Awadalla, I. Gao, J. Gardner, J. Hessel, Y . Hanafy, W. Zhu, K. Marathe, Y . Bitton, S. Gadre, S. Sagawaet al., “Openflamingo: An open-source framework for training large autoregressive vision- language models,”arXiv:2308.01390, 2023

    Pith/arXiv arXiv 2023

  74. [79]

    Perceiver: General perception with iterative attention,

    A. Jaegle, F. Gimeno, A. Brock, O. Vinyals, A. Zisserman, and J. Carreira, “Perceiver: General perception with iterative attention,” in ICML. PMLR, 2021

    work page 2021

  75. [80]

    Unianimate-dit: Human image animation with large-scale video diffusion transformer,

    X. Wang, S. Zhang, L. Tang, Y . Zhang, C. Gao, Y . Wang, and N. Sang, “Unianimate-dit: Human image animation with large-scale video diffusion transformer,”arXiv:2504.11289, 2025

    arXiv 2025

  76. [81]

    Roformer: En- hanced transformer with rotary position embedding,

    J. Su, M. Ahmed, Y . Lu, S. Pan, W. Bo, and Y . Liu, “Roformer: En- hanced transformer with rotary position embedding,”Neurocomputing, 2024

    work page 2024

  77. [82]

    Learning high fidelity depths of dressed humans by watching social media dance videos,

    Y . Jafarian and H. S. Park, “Learning high fidelity depths of dressed humans by watching social media dance videos,” inCVPR, 2021

    work page 2021

  78. [83]

    Dwnet: Dense warp-based network for pose-guided human video generation,

    P. Zablotskaia, A. Siarohin, B. Zhao, and L. Sigal, “Dwnet: Dense warp-based network for pose-guided human video generation,” arXiv:1910.09139, 2019

    Pith/arXiv arXiv 1910

  79. [84]

    Disco: Disentangled control for referring human dance generation in real world,

    T. Wang, L. Li, K. Lin, C.-C. Lin, Z. Yang, H. Zhang, Z. Liu, and L. Wang, “Disco: Disentangled control for referring human dance generation in real world,” inICLR, 2024

    work page 2024

  80. [85]

    Moore-AnimateAnyone,

    M. T. Corporation, “Moore-AnimateAnyone,” 2024. [Online]. Available: https://github.com/MooreThreads/Moore-AnimateAnyone

    work page 2024

Showing first 80 references.

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.