Pith. sign in

REVIEW 5 major objections 5 minor 4 cited by

InfinityHuman claims a two-stage, pose-guided pipeline keeps identity, color, hands, and lip-sync stable in minute-scale audio-driven human videos, with state-of-the-art results on EMTD and HDTF.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A coarse-to-fine audio-driven animation framework that uses pose-guided refinement and hand-specific reward learning to generate long, identity-stable talking videos.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A solid systems paper with real EMTD gains, but the SOTA claim is too broad and the pose-stability premise needs direct measurement. the 5 major comments →

arxiv 2508.20210 v1 pith:T4MDMYDQ submitted 2025-08-27 cs.CV

InfinityHuman: Towards Long-Term Audio-Driven Human

Xiaodi Li , Pan Xie , Yi Ren , Qijun Gan , Chen Zhang , Fangyuan Kong , Xiang Yin , Bingyue Peng
show 1 more author
Zehuan Yuan
This is my paper
classification cs.CV
keywords audio-driven human animationlong-form video generationpose-guided refinerdiffusion transformeridentity preservationlip-synchand gesture generationreward fine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

InfinityHuman is a two-stage system for turning one photo and an audio track into a long, high-resolution video of a person speaking and gesturing. The paper argues that the usual recipe—extending a video model autoregressively by feeding back overlapping motion frames—accumulates errors and destroys identity, color, and scene stability, and that hands are especially badly rendered. Its fix is to first synthesize a low-resolution, audio-aligned coarse video, then refine it into 720p using the extracted pose sequence plus the first frame as a visual anchor, on the theory that pose is structurally decoupled from appearance and therefore does not degrade over time. A separate hand-specific reward, trained on 10,000 hand images, pushes the model toward anatomically plausible, audio-aligned gestures. On EMTD and HDTF the authors report state-of-the-art FID, FVD, identity similarity, hand keypoint confidence, and lip-sync scores.

Core claim

The central claim is that long-form audio-driven human animation can be made visually stable by splitting the problem into a coarse motion stage and a pose-guided refinement stage. Specifically, InfinityHuman first produces a low-resolution video whose motion is synchronized with audio, then estimates pose maps from that low-resolution video and feeds them, together with the reference image as a prefix latent, into a high-resolution diffusion refiner. Because pose encodes structure rather than appearance, the authors argue, it survives long generation runs without the drift that corrupts color and identity; the first frame anchors appearance. The paper further claims that adding a hand-speci

What carries the argument

The load-bearing object is the pose sequence P, an 8-channel pixel-level tensor whose first seven channels encode human keypoints and whose last channel encodes up to 20 background keypoints, extracted from the low-resolution video by the Sapiens pose estimator. The refiner (PG-Refiner) tokenizes P along temporal and spatial dimensions, projects it into the DiT latent space, and adds it to the high-resolution latent; the first reference frame is kept noise-free as a prefix latent and its noise prediction is masked out of the loss. This combination is what the paper credits for correcting accumulated errors, preserving lip-sync, and reducing artifacts. The second mechanism is the hand-specifi

Load-bearing premise

The whole refinement stage trusts pose maps extracted from the low-resolution video: if those poses drift or distort as the video gets longer, the refiner is conditioning on wrong structure, and the paper does not measure pose error over time.

What would settle it

Measure pose-estimation error (e.g., keypoint PCK or average keypoint distance) between poses extracted from V_lr and ground-truth poses at 10s, 20s, 30s, 40s, 50s, and 60s on EMTD. If error grows with duration, or if replacing extracted poses with ground-truth poses substantially improves FSIM, FaceSIM, or HKC, then the paper's premise that pose sequences resist temporal degradation is the load-bearing and vulnerable point.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Long-video generation can be decomposed so that most denoising happens at low resolution, with a separate high-resolution refinement stage, making minute-scale 720p video practical; the appendix reports refiner distillation to a one-step model with comparable FID and FVD.
  • Because pose is the structural signal and appearance is anchored by the first frame, identity and color drift are bounded by the anchor rather than by accumulated frame-to-frame errors, which is why the method maintains FaceSIM and FID over 40–60 second clips.
  • Hand quality need not be modeled architecturally; an image-level reward applied to sampled video frames is enough to raise hand keypoint confidence and reduce visible distortions, including semantic gestures like a two-finger sign during the word 'twenty'.
  • Chunked continuation with overlapping latents (first m latents carried from the previous chunk) plus pose guidance produces smooth motion transitions without retraining on curated long-video datasets.
  • On both a long upper-body benchmark (EMTD) and a talking-head benchmark (HDTF), the method beats current full-body and portrait baselines on the reported metrics, and in short clips it also outperforms a 15-second model (OmniHuman) on all reported metrics.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper assumes pose estimated from V_lr is reliable, but never reports pose error over time; a direct test would be to compare the refiner fed with extracted poses versus ground-truth poses. If the gap is large, pose quality is the true bottleneck; if small, the claimed stability comes mostly from the reference anchor.
  • The hand reward samples a single frame per video, so temporal consistency of hands is only indirectly optimized; a video-level reward or a keypoint-trajectory consistency term would be a natural next step and could improve the reported HKV.
  • The long-form stability table shows FID and Sync staying nearly flat while FVD fluctuates; this suggests the stability claim is mainly about identity, color, and appearance, not about smoothness of dynamics, and duration-wise motion quality deserves its own metric.
  • The appendix's multi-person extension (silent audio for a non-speaking character, bounding-box gated attention) is a lightweight workaround; combining it with multiple pose streams in the refiner could be a testable path toward true multi-character interaction.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. InfinityHuman proposes a two-stage coarse-to-fine framework for long audio-driven full-body animation. A low-resolution audio-to-video (LR-A2V) stage generates 360p motion, and a pose-guided refiner (PG-Refiner) upscales to high resolution using pose sequences extracted from the LR video, the initial reference frame as a visual anchor, and degraded LR latents. A hand-specific reward fine-tuning step is added to improve hand realism. The paper reports quantitative results on EMTD and HDTF, ablations, a user study, and long-form stability analysis, claiming state-of-the-art performance in video quality, identity preservation, hand accuracy, and lip-sync.

Significance. If the central claims hold, the paper offers a practical recipe for minute-scale, high-resolution audio-driven animation: the pose-guided refiner is a reasonable mechanism for reducing drift while preserving motion, and the hand-reward stage addresses a known weak point of full-body animation. The paper includes extensive experiments, a large training corpus, and an ablation study indicating that pose conditioning is important. The main strengths are the industrial-scale training setup, the explicit coarse-to-fine design, and the attempt to measure long-form stability. However, the reported numbers do not uniformly support the headline 'state-of-the-art' claim, and several load-bearing assumptions are not directly verified.

major comments (5)
  1. [Abstract; Table 1] The abstract and Section 4.2 claim 'state-of-the-art performance in video quality, identity preservation, hand accuracy, and lip-sync.' Table 1 does not support this broadly. On HDTF, the proposed method's FaceSIM (0.89) is below SadTalker (0.93) and Hallo3 (0.91), and its Sync-C (8.59) is below SadTalker (8.87) and OmniAvatar (8.81). On EMTD, the HKV value (16.0) is worse than Hallo3 (6.3) if lower is better, and worse than HyAva (29.2) and OmniAvatar (28.7) if higher is better; the table does not specify the direction for HKV. The claims should be restricted to the metrics for which the paper actually reports best results, or the table should be corrected and discussed.
  2. [§3.2, eq. (5)-(6)] The pose-stability premise is load-bearing but unverified. The paper states that pose sequences 'remain highly stable with minimal error accumulation in long-duration generation tasks' and then extracts poses from the progressively degraded low-resolution video V_lr using Sapiens. No pose error or drift measurement is reported as a function of duration, and no comparison is made between poses extracted from V_lr and poses from a clean reference. The ablation 'w/o pose cond' (Table 2) removes pose entirely; it does not test whether the pose condition is accurate. If Sapiens degrades on V_lr (e.g., due to color shifts or distortions shown in Fig. 2), the refiner may be guided by erroneous structure, and the claimed advantage over direct LR-to-HR refinement collapses. Please add a direct evaluation of pose estimation accuracy on V_lr as a function of duration and demonstrate that pose error
  3. [§3.3, eq. (9)] The hand-specific reward mechanism is described as improving 'the realism, stability, and audio synchronization of hand motion,' but eq. (9) optimizes only a randomly selected single decoded frame using an image-level reward model. An image-level, single-frame reward cannot directly enforce temporal stability or audio synchronization. Moreover, the paper does not specify how the reward gradient is propagated through the 3D VAE and the diffusion model (e.g., policy gradient, differentiable rendering, or score distillation), which is essential for reproducibility. Please clarify the optimization procedure and report temporal or audio-sync hand metrics (e.g., temporal keypoint consistency, hand-audio alignment) beyond HKC and HKV.
  4. [Table 2] The ablation results contain an unexplained anomaly: removing the hand reward ('w/o hand refl') improves FID from 91.74 to 86.32 while HKC drops by only 0.02. This contradicts the impression that the hand reward improves overall video quality. The paper should discuss this trade-off explicitly and, if the claim is only about hand quality, separate the video-quality metrics from the hand-specific metrics. Otherwise, the ablation provides evidence that the hand reward may actually hurt image-level quality.
  5. [§6.7, Table 6] The long-form stability analysis reports FSIM decreasing from 0.8357 at 10s to 0.8057 at 50s and HKC decreasing from 0.9224 to 0.9059. The paper labels this 'minimal degradation,' but there is no baseline comparison on the same duration bins. Without showing the trajectories of previous methods over identical 10s intervals, the central claim that InfinityHuman 'corrects accumulated errors' and maintains stability better than alternatives is not established. Please include baseline long-form stability curves or otherwise quantify the improvement over existing methods.
minor comments (5)
  1. [Throughout] There are several typos: 'vanilly' (Introduction), 'Sapines' (Section 4.1), 'refl' for 'reward' in Figure 5 and Table 2, and the Hallo3 reference is duplicated (Cui et al. 2024a and 2024b).
  2. [Table 1] The HKV column lacks an arrow indicating whether lower or higher is better. Please add the direction and define the metric in Section 4.2.
  3. [§3.2] The notation for the pose tensor and pose tokens is confusing: P is defined as ∈ R^{(4f+1)×4h×4w×8}, but P' is ∈ R^{(f+1)×h×w×(64×8)}. Please clarify the temporal downsampling factor and the channel-to-patch reshaping.
  4. [§4.2] The evaluation metrics are reported as point estimates without confidence intervals or the number of videos used for each metric. Please state the sample size and, where feasible, report statistical significance.
  5. [§6.6] The user study uses 'six expert raters' and reports percentages, but no inter-rater agreement (e.g., Cohen's kappa) or statistical test is provided.

Circularity Check

0 steps flagged

No circularity: InfinityHuman's claims rest on external benchmarks and separated optimization/evaluation; pose-stability is an empirical premise, not a circular reduction.

full rationale

The central derivation chain is an empirical coarse-to-fine pipeline, not a set of equations that reduce predictions to fitted inputs. The LR-A2V stage (Eqs. 1–3) regresses flow-matching velocities; the pose-guided refiner (Eqs. 5–8) is a diffusion refiner conditioned on low-resolution latents, pose maps, and the reference frame. No equation defines the output in terms of the evaluation metrics (FID, FVD, FSIM, HKC, Sync-C/D), and the SOTA claim is validated on external benchmarks (EMTD, HDTF) against published baselines. The pose condition is extracted from the low-resolution video V_lr, but the paper treats pose stability as an assumption and tests the refiner's contribution via ablations (Table 2); whether Sapiens remains accurate on degraded V_lr is a robustness/correctness concern, not a definitional equivalence. The hand-specific reward model (Eq. 9) is trained with human-annotated hand data, while the reported hand metric HKC comes from Sapiens keypoint confidence, so optimization and evaluation use different signals. SyncNet is used for training-data filtering and for lip-sync evaluation, but filtering is applied to the training corpus, not to the held-out test sets, so evaluation remains externally grounded. Self-citations to Humandit and Goku-I2V concern training recipes or base models and are not invoked as proof of the paper's central empirical claims. No step exhibits the required pattern of a fitted parameter renamed as a prediction or a result equivalent to its input by construction.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

All central-claim dependencies are listed. The paper contributes no new physical entities; its novelties are learned modules. Main free hyperparameters are reported but several are empirically tuned (alpha, T, CFG, steps), and the key chunk overlap m is not specified. The axioms are standard generative-model assumptions plus the unverified premise that pose extraction from low-res video is stable enough to guide refinement.

free parameters (4)
  • alpha_deg (degradation noise strength) = 0.7
    Chosen empirically in Sec. 6.2 to balance visual realism and temporal stability in the pose-guided refiner (Eq. 5).
  • T (hand-quality reward threshold) = 0.4
    Set empirically in Sec. 6.2 as the aesthetic threshold in the hand reward loss (Eq. 9).
  • CFG weights and denoising steps = audio/text CFG 6.5, pose CFG 1.5; 30 and 20 steps
    Inference-time hyperparameters in Sec. 4.1; selected for quality and not derived from first principles.
  • m (number of clean prefix or anchor latents) = not specified
    The chunk overlap length m in Eq. 6 and inference continuation is load-bearing for identity anchoring but its value is never reported.
axioms (5)
  • standard math Flow matching and DiT training is a valid generative objective for video latents.
    Used as the training objective for both LR-A2V and the refiner (Eq. 3, Eq. 7).
  • domain assumption Sapiens pose estimation returns keypoints accurate enough to guide high-resolution refinement, including background keypoints.
    Pose maps P are extracted from low-res generated frames and treated as ground-truth structure in Sec. 3.2.
  • domain assumption Pretrained Goku-I2V provides a suitable base model for both stages.
    Sec. 4.1 states both models start from Goku-I2V; the paper does not ablate the choice of base model.
  • domain assumption The 7,700h and 1,800h curated single-person video datasets are sufficiently representative for training long video generation.
    Sec. 4.1 describes filtering; no analysis of distribution or failure modes is given beyond a stated limitation in Sec. 5.
  • domain assumption The hand reward model fine-tuned from MPS is a valid proxy for hand realism and transfers from single frames to video.
    Sec. 3.3 and Sec. 6.2; training uses randomly sampled single frames, so frame-level reward is assumed to transfer to video motion.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of InfinityHuman: Towards Long-Term Audio-Driven Human." pith.science (2026). https://pith.science/paper/T4MDMYDQ

@misc{pith2026250820210,
  author       = {Pith},
  title        = {Pith review of: InfinityHuman: Towards Long-Term Audio-Driven Human},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/T4MDMYDQ}},
  note         = {Machine review of arXiv:2508.20210}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Audio-driven human animation has attracted wide attention thanks to its practical applications. However, critical challenges remain in generating high-resolution, long-duration videos with consistent appearance and natural hand motions. Existing methods extend videos using overlapping motion frames but suffer from error accumulation, leading to identity drift, color shifts, and scene instability. Additionally, hand movements are poorly modeled, resulting in noticeable distortions and misalignment with the audio. In this work, we propose InfinityHuman, a coarse-to-fine framework that first generates audio-synchronized representations, then progressively refines them into high-resolution, long-duration videos using a pose-guided refiner. Since pose sequences are decoupled from appearance and resist temporal degradation, our pose-guided refiner employs stable poses and the initial frame as a visual anchor to reduce drift and improve lip synchronization. Moreover, to enhance semantic accuracy and gesture realism, we introduce a hand-specific reward mechanism trained with high-quality hand motion data. Experiments on the EMTD and HDTF datasets show that InfinityHuman achieves state-of-the-art performance in video quality, identity preservation, hand accuracy, and lip-sync. Ablation studies further confirm the effectiveness of each module. Code will be made public.

Figures

Figures reproduced from arXiv: 2508.20210 by Bingyue Peng, Chen Zhang, Fangyuan Kong, Pan Xie, Qijun Gan, Xiang Yin, Xiaodi Li, Yi Ren, Zehuan Yuan.

Figure 1
Figure 1. Figure 1: InfinityHuman is an audio-driven full-body animation framework that synthesizes long-duration videos with (a) temporally consistent visual appearance, (b) expressive and style-rich hand gestures, (c) dynamic human-object interactions, and (d) emotion-controllable, audio-aligned full-body motions. Abstract Audio-driven human animation has attracted wide attention thanks to its practical applications. Howeve… view at source ↗
Figure 2
Figure 2. Figure 2: Progressive Degradation in Long Video Animation by Previous Methods. Existing methods suffer from cumulative errors leading to pronounced identity drift (facial inconsistencies), color shifts (hair, clothing), scene instability (background fluctuations), and hand motion artifacts. These challenges underscore the necessity of InfinityHuman’s pose-guided refiner and hand-specific optimization for producing h… view at source ↗
Figure 3
Figure 3. Figure 3: InfinityHuman Pipeline. The pipeline generates high-resolution (HR) audio-driven full-body videos through a two-stage coarse-to-fine process. First, a speech-aligned low-resolution (LR) video is generated using multimodal conditioning (text and audio) and DiT blocks. In the second stage, a pose-guided refiner utilizes pose guidance, LR latents, and reference images to restore degraded details, enhancing id… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative Results of Audio-Driven Animation Methods on EMTD. Yellow and blue boxes highlight hand distortions and face ID mismatches, respectively. The results demonstrate the superiority of InfinityHuman in maintaining identity consistency, lip-sync accuracy, and visual fidelity during long-duration generation. Please zoom in for details. (§4.2) To train pose-guided refiner, we also use Goku-I2V as pre￾… view at source ↗
Figure 5
Figure 5. Figure 5: Visualization of Ablation Study. Demonstrating the effects of key components on animation quality. information or low-resolution latent features after training leads to color shifts and structural degradation in long-term video generation. This suggests that both inputs serve as essential references: the pose offers accurate structural con￾straints, while the low-resolution latent helps preserve overall se… view at source ↗
Figure 6
Figure 6. Figure 6: Qualitative comparison between OmniHuman and [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Illustration of multi-person alternating speaking. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Visual comparison between the original and dis [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: When the character says "twenty", the hand per￾forms a two-finger gesture, demonstrating alignment between speech and hand gesture. 6.6 User Study Results To evaluate the perceptual quality of the generated videos, six expert raters assessed 78 videos from two methods: Infin￾ityHuman and Omnihuman (Lin et al. 2025). The evaluation covered overall video quality (high-standard and non-high￾standard), complia… view at source ↗
Figure 10
Figure 10. Figure 10: Additional qualitative results showcasing the robustness and generality of our method across diverse scenarios. [PITH_FULL_IMAGE:figures/full_fig_p013_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. ViBES: A Conversational Agent with Behaviorally-Intelligent 3D Virtual Body

    cs.CV 2025-12 unverdicted novelty 7.0

    ViBES introduces a speech-language-behavior model using modality-specific transformer experts that jointly generates dialogue and 3D body actions, showing gains over separate co-speech and text-to-motion baselines on ...

  2. SyncCache: Exploiting Asymmetric Dynamics for Fast Audio-Driven Portrait Animation

    cs.CV 2026-06 unverdicted novelty 6.0

    SyncCache accelerates DiT-based audio-driven portrait animation up to 4.12x via spatially-asymmetric probing and modality-decoupled caching while preserving near-lossless quality and audio sync.

  3. Generate Your Talking Avatar from Video Reference

    cs.CV 2026-04 unverdicted novelty 6.0

    TAVR generates high-fidelity talking avatars from cross-scene video references via token selection and three-stage training (same-scene pretraining, cross-scene fine-tuning, identity RL), outperforming baselines on a ...

  4. EchoTorrent: Towards Swift, Sustained, and Streaming Multi-Modal Video Generation

    cs.CV 2026-02 unverdicted novelty 4.0

    EchoTorrent combines multi-teacher distillation, adaptive CFG calibration, hybrid long-tail forcing, and VAE decoder refinement to enable few-pass autoregressive streaming video generation with improved temporal consi...

Reference graph

Works this paper leans on

47 extracted references · 18 canonical work pages · cited by 4 Pith papers

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    ai, S.; Teng, H.; Jia, H.; Sun, L.; Li, L.; Li, M.; Tang, M.; Han, S.; Zhang, T.; Zhang, W. Q.; Luo, W.; Kang, X.; Sun, Y.; Cao, Y.; Huang, Y.; Lin, Y.; Fang, Y.; Tao, Z.; Zhang, Z.; Wang, Z.; Liu, Z.; Shi, D.; Su, G.; Sun, H.; Pan, H.; Wang, J.; Sheng, J.; Cui, M.; Hu, M.; Yan, M.; Yin, S.; Zhang, S.; Liu, T.; Yin, X.; Yang, X.; Song, X.; Hu, X.; Zhang, ...

    Pith/arXiv arXiv 2025

  4. [4]

    Bao, F.; Xiang, C.; Yue, G.; He, G.; Zhu, H.; Zheng, K.; Zhao, M.; Liu, S.; Wang, Y.; and Zhu, J. 2024. Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Models. arXiv preprint arXiv:2405.04233

    Pith/arXiv arXiv 2024

  5. [5]

    Brandon, C. 2024. PySceneDetect. https://github.com/Breakthrough/PySceneDetect/

    work page 2024

  6. [6]

    Chen, S.; Ge, C.; Zhang, Y.; Zhang, Y.; Zhu, F.; Yang, H.; Hao, H.; Wu, H.; Lai, Z.; Hu, Y.; et al. 2025 a . Goku: Flow based video generative foundation models. In Proceedings of the Computer Vision and Pattern Recognition Conference, 23516--23527

    work page 2025

  7. [7]

    E.; Fang, Y.; Lee, H.-Y.; Ren, J.; Yang, M.-H.; et al

    Chen, T.-S.; Siarohin, A.; Menapace, W.; Deyneka, E.; Chao, H.-w.; Jeon, B. E.; Fang, Y.; Lee, H.-Y.; Ren, J.; Yang, M.-H.; et al. 2024. Panda-70m: Captioning 70m videos with multiple cross-modality teachers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13320--13331

    work page 2024

  8. [8]

    Chen, Y.; Liang, S.; Zhou, Z.; Huang, Z.; Ma, Y.; Tang, J.; Lin, Q.; Zhou, Y.; and Lu, Q. 2025 b . HunyuanVideo-Avatar: High-Fidelity Audio-Driven Human Animation for Multiple Characters. arXiv:2505.20156

    Pith/arXiv arXiv 2025

  9. [9]

    Chen, Z.; Cao, J.; Chen, Z.; Li, Y.; and Ma, C. 2025 c . Echomimic: Lifelike audio-driven portrait animations through editable landmark conditions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, 2403--2410

    work page 2025

  10. [10]

    S.; and Zisserman, A

    Chung, J. S.; and Zisserman, A. 2017. Out of time: automated lip sync in the wild. In Computer Vision--ACCV 2016 Workshops: ACCV 2016 International Workshops, Taipei, Taiwan, November 20-24, 2016, Revised Selected Papers, Part II 13, 251--263. Springer

    work page 2017

  11. [12]

    Cui, J.; Li, H.; Zhan, Y.; Shang, H.; Cheng, K.; Ma, Y.; Mu, S.; Zhou, H.; Wang, J.; and Zhu, S. 2024 b . Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer. arXiv preprint arXiv:2412.00733

    Pith/arXiv arXiv 2024

  12. [13]

    Gan, Q.; Ren, Y.; Zhang, C.; Ye, Z.; Xie, P.; Yin, X.; Yuan, Z.; Peng, B.; and Zhu, J. 2025 a . Humandit: Pose-guided diffusion transformer for long-form human motion video generation. arXiv preprint arXiv:2502.04847

    Pith/arXiv arXiv 2025

  13. [14]

    Gan, Q.; Yang, R.; Zhu, J.; Xue, S.; and Hoi, S. 2025 b . OmniAvatar: Efficient Audio-Driven Avatar Video Generation with Adaptive Body Animation. arXiv preprint arXiv:2506.18866

    Pith/arXiv arXiv 2025

  14. [15]

    Henschel, R.; Khachatryan, L.; Hayrapetyan, D.; Poghosyan, H.; Tadevosyan, V.; Wang, Z.; Navasardyan, S.; and Shi, H. 2024. StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text. arXiv preprint arXiv:2403.14773

    Pith/arXiv arXiv 2024

  15. [16]

    Heusel, M.; Ramsauer, H.; Unterthiner, T.; Nessler, B.; and Hochreiter, S. 2017. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30

    2017

  16. [17]

    Ho, J.; and Salimans, T. 2022. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598

    Pith/arXiv arXiv 2022

  17. [18]

    Hogue, S.; Zhang, C.; Daruger, H.; Tian, Y.; and Guo, X. 2024. DiffTED: One-shot Audio-driven TED Talk Video Generation with Diffusion-based Co-speech Gestures. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1922--1931

    work page 2024

  18. [19]

    Hu, L.; Gao, X.; Zhang, P.; Sun, K.; Zhang, B.; and Bo, L. 2023. Animate anyone: Consistent and controllable image-to-video synthesis for character animation. arXiv preprint arXiv:2311.17117

    Pith/arXiv arXiv 2023

  19. [20]

    Huang, J.; Dong, X.; Song, W.; Chong, Z.; Tang, Z.; Zhou, J.; Cheng, Y.; Chen, L.; Li, H.; Yan, Y.; et al. 2024. Consistentid: Portrait generation with multimodal fine-grained identity preserving. arXiv preprint arXiv:2404.16771

    Pith/arXiv arXiv 2024

  20. [21]

    Jiang, J.; Liang, C.; Yang, J.; Lin, G.; Zhong, T.; and Zheng, Y. 2024. Loopy: Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependency. arXiv preprint arXiv:2409.02634

    Pith/arXiv arXiv 2024

  21. [22]

    Jocher, G.; Qiu, J.; and Chaurasia, A. 2023. Ultralytics YOLO. https://github.com/ultralytics/ultralytics

    work page 2023

  22. [23]

    Khirodkar, R.; Bagautdinov, T.; Martinez, J.; Zhaoen, S.; James, A.; Selednik, P.; Anderson, S.; and Saito, S. 2024. Sapiens: Foundation for Human Vision Models. arXiv preprint arXiv:2408.12569

    Pith/arXiv arXiv 2024

  23. [24]

    Kong, Z.; Gao, F.; Zhang, Y.; Kang, Z.; Wei, X.; Cai, X.; Chen, G.; and Luo, W. 2025. Let Them Talk: Audio-Driven Multi-Person Conversational Video Generation. arXiv preprint arXiv:2505.22647

    Pith/arXiv arXiv 2025

  24. [25]

    Lin, G.; Jiang, J.; Liang, C.; Zhong, T.; Yang, J.; and Zheng, Y. 2024. CyberHost: Taming Audio-driven Avatar Diffusion Model with Region Codebook Attention. arXiv preprint arXiv:2409.01876

    Pith/arXiv arXiv 2024

  25. [26]

    Lin, G.; Jiang, J.; Yang, J.; Zheng, Z.; and Liang, C. 2025. OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Models. arXiv preprint arXiv:2502.01061

    Pith/arXiv arXiv 2025

  26. [27]

    T.; Ben-Hamu, H.; Nickel, M.; and Le, M

    Lipman, Y.; Chen, R. T.; Ben-Hamu, H.; Nickel, M.; and Le, M. 2022. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747

    Pith/arXiv arXiv 2022

  27. [28]

    Meng, R.; Zhang, X.; Li, Y.; and Ma, C. 2024. EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation. arXiv preprint arXiv:2411.10061

    arXiv 2024

  28. [29]

    Peebles, W.; and Xie, S. 2023. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, 4195--4205

    work page 2023

  29. [30]

    Qiu, H.; Xia, M.; Zhang, Y.; He, Y.; Wang, X.; Shan, Y.; and Liu, Z. 2023. Freenoise: Tuning-free longer video diffusion via noise rescheduling. arXiv preprint arXiv:2310.15169

    Pith/arXiv arXiv 2023

  30. [31]

    Ren, Y.; Xia, X.; Lu, Y.; Zhang, J.; Wu, J.; Xie, P.; Wang, X.; and Xiao, X. 2024. Hyper-sd: Trajectory segmented consistency model for efficient image synthesis. Advances in Neural Information Processing Systems, 37: 117340--117362

    work page 2024

  31. [32]

    Sauer, A.; Boesel, F.; Dockhorn, T.; Blattmann, A.; Esser, P.; and Rombach, R. 2024. Fast high-resolution image synthesis with latent adversarial diffusion distillation. In SIGGRAPH Asia 2024 Conference Papers, 1--11

    work page 2024

  32. [33]

    Unterthiner, T.; Van Steenkiste, S.; Kurach, K.; Marinier, R.; Michalski, M.; and Gelly, S. 2018. Towards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717

    Pith/arXiv arXiv 2018

  33. [35]

    Wang, C.; Tian, K.; Zhang, J.; Guan, Y.; Luo, F.; Shen, F.; Jiang, Z.; Gu, Q.; Han, X.; and Yang, W. 2024 b . V-express: Conditional dropout for progressive training of portrait video generation. arXiv preprint arXiv:2406.02511

    Pith/arXiv arXiv 2024

  34. [36]

    Wang, F.-Y.; Chen, W.; Song, G.; Ye, H.-J.; Liu, Y.; and Li, H. 2023. Gen-l-video: Multi-text to long video generation via temporal co-denoising. arXiv preprint arXiv:2305.18264

    Pith/arXiv arXiv 2023

  35. [37]

    Wang, F.-Y.; Huang, Z.; Bergman, A.; Shen, D.; Gao, P.; Lingelbach, M.; Sun, K.; Bian, W.; Song, G.; Liu, Y.; et al. 2024 c . Phased consistency models. Advances in neural information processing systems, 37: 83951--84009

    work page 2024

  36. [38]

    Wang, M.; Wang, Q.; Jiang, F.; Fan, Y.; Zhang, Y.; Qi, Y.; Zhao, K.; and Xu, M. 2025. FantasyTalking: Realistic Talking Portrait Generation via Coherent Motion Synthesis. arXiv preprint arXiv:2504.04842

    Pith/arXiv arXiv 2025

  37. [39]

    Wei, H.; Yang, Z.; and Wang, Z. 2024. Aniportrait: Audio-driven synthesis of photorealistic portrait animation. arXiv preprint arXiv:2403.17694

    Pith/arXiv arXiv 2024

  38. [40]

    Wu, H.; Zhang, Z.; Zhang, W.; Chen, C.; Liao, L.; Li, C.; Gao, Y.; Wang, A.; Zhang, E.; Sun, W.; et al. 2023. Q-align: Teaching lmms for visual scoring via discrete text-defined levels. arXiv preprint arXiv:2312.17090

    Pith/arXiv arXiv 2023

  39. [41]

    Xie, L.; Li, Y.; Du, S.; Xia, M.; Wang, X.; Yu, F.; Chen, Z.; Wan, P.; Zhou, J.; and Dong, C. 2025. SimpleGVR: A Simple Baseline for Latent-Cascaded Video Super-Resolution. arXiv preprint arXiv:2506.19838

    arXiv 2025

  40. [42]

    Xu, M.; Li, H.; Su, Q.; Shang, H.; Zhang, L.; Liu, C.; Wang, J.; Van Gool, L.; Yao, Y.; and Zhu, S. 2024. Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animation. arXiv preprint arXiv:2406.08801

    Pith/arXiv arXiv 2024

  41. [43]

    T.; Durand, F.; Shechtman, E.; and Huang, X

    Yin, T.; Zhang, Q.; Zhang, R.; Freeman, W. T.; Durand, F.; Shechtman, E.; and Huang, X. 2025. From Slow Bidirectional to Fast Autoregressive Video Diffusion Models. In CVPR

    work page 2025

  42. [44]

    Yuan, S.; Huang, J.; He, X.; Ge, Y.; Shi, Y.; Chen, L.; Luo, J.; and Yuan, L. 2025. Identity-preserving text-to-video generation by frequency decomposition. In Proceedings of the Computer Vision and Pattern Recognition Conference, 12978--12988

    work page 2025

  43. [45]

    Zeng, Y.; Wei, G.; Zheng, J.; Zou, J.; Wei, Y.; Zhang, Y.; and Li, H. 2024. Make pixels dance: High-dynamic video generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8850--8860

    work page 2024

  44. [46]

    Zhang, L.; and Agrawala, M. 2025. Packing Input Frame Contexts in Next-Frame Prediction Models for Video Generation. Arxiv

    work page 2025

  45. [47]

    Zhang, S.; Wang, B.; Wu, J.; Li, Y.; Gao, T.; Zhang, D.; and Wang, Z. 2024. Learning multi-dimensional human preference for text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8018--8027

    work page 2024

  46. [48]

    Zhang, W.; Cun, X.; Wang, X.; Zhang, Y.; Shen, X.; Guo, Y.; Shan, Y.; and Wang, F. 2023. Sadtalker: Learning realistic 3d motion coefficients for stylized audio-driven single image talking face animation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 8652--8661

    work page 2023

  47. [49]

    Zhang, Z.; Li, L.; Ding, Y.; and Fan, C. 2021. Flow-guided one-shot talking face generation with a high-resolution audio-visual dataset. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 3661--3670

    work page 2021

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.