Pith. sign in

REVIEW 4 major objections 7 minor 32 references

Identity-Preserving Video Dubbing Using Motion Warping

T0 review · 4 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read IPTalker synthesizes dubbed video by aligning driving audio with reference mouth images in a transformer, then warping those images to the target mouth shape, and the paper shows this preserves the speaker's identity better than existing…

desk verdict The architecture is a plausible incremental step, but the SOTA claim is not supported because Table I compares multi-reference IPTalker against one-shot baselines and the paper never isolates the reference budget. read the letter →

arxiv 2501.04586 v2 pith:GSZ3B5EN submitted 2025-01-08 cs.CV

Runzhen Liu , Qinjie Lin , Yunfei Liu , Lijian Lin , Ye Zhu , Yu Li , Chuhua Xian , Fa-Ting Hong This is my paper · ORCID
classification cs.CV
keywords videodubbingidentitypreservationmotionwarpingaudio-visualalignmenttalkingheadgenerationlipsynchronizationtransformerattentionimageinpainting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes IPTalker, a video dubbing framework that keeps the reference speaker's identity while matching lip movements to a new audio track. Instead of generating mouth pixels from scratch, it first learns which reference mouth image best corresponds to the driving audio, then spatially warps that image to the target mouth configuration, and finally inpaints any occluded region. The authors claim this approach produces high-resolution, temporally consistent dubbed videos with realistic mouth textures, and they report it outperforms prior open dubbing methods on both the HDTF and VFHQ datasets. The reason this matters is that previous dubbing models tend to trade lip-sync accuracy against identity fidelity, and IPTalker is designed to preserve both.

What carries the argument

The key machinery is the Audio-Visual Alignment Unit (AVAU), a transformer block that treats audio and reference mouth embeddings as queries and keys to learn which reference mouth image best matches the target lip shape, alongside the warping module that uses the resulting correspondence embedding to modulate an encoder-decoder via AdaIN and produce a motion flow for spatial deformation. The SPADE decoder then inpaints the masked mouth region, and a Gaussian-smoothed convex-hull mask blends the result back into the frame. Together these components replace direct pixel generation from audio with a deform-and-inpaint scheme that preserves the reference texture.

What would settle it

Run IPTalker with exactly one reference image on the same HDTF/VFHQ test videos used in Table I, measuring SSIM, PSNR, LPIPS, and LSE-C/LSE-D; if the metrics fall to the level of DINet or below, the claimed superiority is an artifact of the number of reference images rather than the alignment-warping mechanism.

Watch

Extended reading notes

Core claim

The paper's central claim is that explicit audio-visual correspondence learning, followed by motion warping of reference mouth images, is sufficient to achieve state-of-the-art video dubbing with strong identity retention. Given a source video and driving audio, IPTalker extracts embeddings from N reference lower-face images and the audio, uses stacked Audio-Visual Alignment Units and a cross-modalities encoder to produce an identity-audio correspondence embedding, then predicts a motion flow that warps reference features to the audio-driven mouth shape. A SPADE-based inpainting decoder fills the masked mouth region while a Gaussian-smoothed convex-hull mask pastes the generated face back onto the original frame. On quantitative metrics SSIM, PSNR, LPIPS, LSE-C, and LSE-D, and in qualitative comparisons, the paper reports IPTalker achieves the best visual quality among compared methods on both HDTF and VFHQ, with best lip-sync on HDTF and slightly lower LSE-D on VFHQ attributed to training on front-facing HDTF data only. The paper also acknowledges that because deformation is confined to the mouth region, the generated mouth can appear to 'float' relative to surrounding facial features when angle or position is slightly off.

Load-bearing premise

The evaluation assumes that giving IPTalker N reference mouth images while giving each baseline only one source image is a fair comparison for the state-of-the-art claim; with a single reference image, IPTalker's reported identity-preservation and quality margins could shrink or disappear.

Editorial extensions

If this is right

  • If IPTalker's claim holds, video dubbing systems can keep the original speaker's teeth, tongue, lip color, and skin texture instead of synthesizing generic mouth interiors.
  • The warping-based pipeline should yield temporally stable output because most pixels in consecutive generated frames come from the same reference images rather than being regenerated per frame.
  • Identity fine-tuning on a short segment of a target video further improves visual quality, suggesting a practical workflow of per-identity adaptation for production use.
  • The alignment-by-correspondence design also provides a principled way to fuse audio with multiple reference views, which could extend to multi-view talking-head synthesis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: The paper compares IPTalker, which uses N reference mouth images, against one-shot baselines that see a single source image; this asymmetry may inflate identity-preservation margins, and a fair head-to-head would restrict IPTalker to one reference.
  • Editorial: The same deform-and-inpaint recipe could be applied to other audio-driven portrait editing tasks, such as singing voice dubbing or emotion-transferred speech, where reference texture fidelity matters.
  • Editorial: Since the method only warps the mouth region, its success suggests that audio mainly constrains lower-face geometry while other identity attributes can be carried over spatially; testing on profile views may expose where warping fails.
  • Editorial: A natural testable extension is to report IPTalker's performance with N=1, N=2, and N=5 references on the same test set to quantify how much of the reported advantage comes from the number of references.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper presents IPTalker, a video dubbing framework composed of an audio-visual alignment module (transformer-based), a warping module using AdaIN, and an inpainting module with SPADE. The alignment module takes N reference mouth images and the driving audio to produce an identity-audio embedding; the warping module produces a flow to deform reference features; the inpainting module fills the masked mouth region. The model is trained with perception, GAN, and SyncNet-based lip-sync losses. Experiments on HDTF and VFHQ compare against recent one-shot methods (DreamTalk, MakeItTalk, Audio2Head, SadTalker, Hallo, AniPortrait, DINet) and report superior SSIM, PSNR, LPIPS, LSE-C, LSE-D, plus ablations and identity fine-tuning. The paper claims a new state of the art for identity-preserving video dubbing.

Significance. If the quantitative results are taken at face value, IPTalker would be a strong identity-preserving video dubbing method, and the component-wise ablations suggest the alignment, SPADE, and cross-modalities encoder each contribute. The paper explicitly includes a Limitations section and reports an identity fine-tuning study, which is transparent. However, the significance is conditional on a fair evaluation against matched reference budgets and independent lip-sync metrics; the current comparison does not establish the stated state of the art.

major comments (4)
  1. [Sec. IV-D, Table I; Sec. III-B, Eq. (1); Sec. III-C, Eq. (4)] The central SOTA claim is not supported by a controlled comparison. IPTalker's alignment module takes N reference mouth images (Eq. 1) and the warping module concatenates N reference features (Eq. 4), whereas every baseline in Table I is one-shot, receiving a single source image. The value of N is never stated, and the paper does not say whether the reference frames are drawn from the same test video whose frames are being reconstructed. If they are, the large SSIM/PSNR/LPIPS margins partly measure access to ground-truth-adjacent appearance at inference time. The authors should either evaluate IPTalker with N=1, or add multi-reference baselines and an ablation over N.
  2. [Sec. III-E, Eq. (12) vs Sec. IV-B] The lip-sync evaluation is circular with the training objective. The model is trained with L_sync = E[(SyncNet(A,I_O)-1)^2], and the reported LSE-C/LSE-D metrics are derived from the same SyncNet (the paper says 'inspired by [1]', which uses SyncNet). Any improvement in LSE-C/LSE-D may be an artifact of directly optimizing the same network. To support the lip-sync claim, the authors should evaluate with at least one independent audio-visual synchronization model or a human study.
  3. [Sec. IV-D, Table II] The identity fine-tuning experiment shows that a few hundred steps of training on a split of the test video raises SSIM from 0.6332 to 0.6724 and PSNR from 30.89 to 34.08. This indicates the reported metrics are highly sensitive to access to the target identity's own video. The paper must clarify whether Table I uses the fine-tuned or non-fine-tuned model; if fine-tuned, the comparison with non-fine-tuned baselines is unfair, and if not, the fine-tuning results reveal that the base model's advantage may shrink when the reference budget is matched.
  4. [Sec. IV-A, Sec. IV-D] The experimental report lacks crucial details: the number N of reference images, training hyperparameters (epochs, learning rate, batch size, number of AVAU layers), and any error bars or significance tests over the 30 (HDTF) and 20 (VFHQ) test videos. Given the large variance typical of talking-head generation, the reported point estimates without variance do not support the 'consistently outperforms' claim.
minor comments (7)
  1. [Sec. III-C] The reference to 'Figure ??' is unresolved; the figure for the warping network architecture is missing.
  2. [Sec. III-E, Eq. (8)] The summation index i in the perception loss is overloaded with the number of reference images N; using L or a layer index would avoid confusion.
  3. [Sec. IV-B] Typos: 'btween' should be 'between', and 'pixl-level' should be 'pixel-level'.
  4. [Conclusion] The phrase 'conpensate the occlusion regions' should be 'compensate for the occlusion regions'.
  5. [Table I] Ground Truth PSNR is listed as N/A; the authors should explain, as PSNR can be computed for a perfect reconstruction.
  6. [Sec. IV-D] The sentence that each test video is 'divided into two segments on average' is ambiguous; please specify the exact split.
  7. [Limitations] The Limitations paragraph raises a 'floating mouth' issue that is not analyzed in the experiments; consider discussing this in the main evaluation.

Circularity Check

1 steps flagged · score 4.0 of 10

Lip-sync metrics overlap with training loss: LSE-C/LSE-D use the same SyncNet that Eq. 12 optimizes, so the claimed lip-sync advantage partially reduces to the training objective.

  1. fitted input called prediction [Section III-E (Eq. 12) and Section IV-D (Table I / LSE-C, LSE-D)]
    "we incorporate a lip-sync loss [1]. The lip-sync loss is defined as: Lsync = E[(SyncNet(A, IO) − 1)^2] ... For audio-visual synchronization, inspired by [1], we calculate Lip Sync Error Confidence (LSE-C) and Lip Sync Error Distance (LSE-D)."

    Eq. 12 trains IPTalker to drive the pretrained SyncNet output toward 1, i.e., to maximize SyncNet's lip-sync confidence and minimize its audio-visual distance. The quantitative lip-sync claim in Table I is then measured by LSE-C/LSE-D, which are exactly the confidence and distance produced by that same SyncNet (via [1]). Thus the reported lip-sync 'state of the art' numbers are not an independent test: they evaluate the model on the same function it was explicitly optimized against. Baselines that do not use the SyncNet loss are measured on IPTalker's training objective. This makes the lip-sync portion of the central claim partially circular; the SSIM/PSNR visual metrics and LPIPS (though LPIPS shares VGG with the perception loss) provide independent content, so the circularity is partial.

full rationale

The only identifiable circular element is the SyncNet overlap between the training loss (Eq. 12) and the lip-sync evaluation metrics (LSE-C/LSE-D, computed via [1] with the same SyncNet). Because the model is explicitly trained to make SyncNet(A, IO) approach 1, reporting LSE-C/LSE-D as evidence of lip-sync superiority is partially reporting success on the training objective; this affects the lip-sync component of the SOTA claim. The SSIM and PSNR metrics are independent of the losses, and LPIPS, while sharing the VGG backbone with the perception loss, is a standard external metric, so the visual-quality portion of the claim retains independent content. The paper does not rely on self-citation for any load-bearing premise. The comparison in Table I does not hold the reference-image budget constant (IPTalker uses N reference images while baselines are one-shot), and the identity fine-tuning experiment uses a split of the test videos, but these are evaluation fairness concerns rather than derivation circularity; they do not make the method's output equivalent to its inputs by construction. Overall the circularity is partial and confined to the lip-sync measurement.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard deep learning assumptions (pretrained SyncNet, VGG, Mediapipe) and on the paper's own design choice that attention-based alignment improves identity preservation. The free parameters are mostly training hyperparameters; the most concerning is the selection of fine-tuning steps based on test-set performance.

free parameters (4)
  • Lambda_p (perception loss weight) = 10
    Hand-chosen loss weight in Eq. 13; not ablated.
  • Lambda_sync (lip-sync loss weight) = 0.1
    Hand-chosen loss weight in Eq. 13; not ablated.
  • Number of reference images N = not specified
    The method uses N reference mouth images; N is a model hyperparameter that strongly affects available texture, but its value or effect is not reported or ablated.
  • Identity fine-tuning steps = 200 (optimal on test set)
    The optimal fine-tuning step count is selected by evaluating on the test videos (Table II), which is fitting to the test set.
assumptions (5)
  • domain assumption SyncNet provides a valid lip-sync score and is used both as a loss (Eq. 12) and as the evaluation metric (LSE-C/LSE-D).
    The paper assumes SyncNet's confidence is a faithful measure of synchronization, and then optimizes it, which can circularly inflate the metric.
  • standard math VGG-19 features are a valid perceptual similarity measure for identity and texture preservation.
    Used in the perception loss (Eq. 8), following common practice in image generation.
  • domain assumption Mediapipe facial landmarks and convex hulls reliably define the mouth region for masking.
    The mask is computed from landmarks as in Figure 4; accuracy of Mediapipe is assumed.
  • ad hoc to paper Attention between audio and reference mouth embeddings captures the 'interplay' needed for identity-aware dubbing.
    This is the paper's core design assumption (Section III-B); there is no theoretical or empirical proof that attention-based alignment is necessary beyond the ablation, which shows a small gain.
  • domain assumption A motion flow can deform reference mouth pixels into target mouth shapes while preserving texture.
    This is the premise of the warping module; the paper relies on prior optical-flow-based warping literature.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Identity-Preserving Video Dubbing Using Motion Warping." pith.science (2026). https://pith.science/paper/GSZ3B5EN

@misc{pith2026250104586,
  author       = {Pith},
  title        = {Pith review of: Identity-Preserving Video Dubbing Using Motion Warping},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GSZ3B5EN}},
  note         = {Machine review of arXiv:2501.04586}
}
read the original abstract

Video dubbing aims to synthesize realistic, lip-synced videos from a reference video and a driving audio signal. Although existing methods can accurately generate mouth shapes driven by audio, they often fail to preserve identity-specific features, largely because they do not effectively capture the nuanced interplay between audio cues and the visual attributes of reference identity . As a result, the generated outputs frequently lack fidelity in reproducing the unique textural and structural details of the reference identity. To address these limitations, we propose IPTalker, a novel and robust framework for video dubbing that achieves seamless alignment between driving audio and reference identity while ensuring both lip-sync accuracy and high-fidelity identity preservation. At the core of IPTalker is a transformer-based alignment mechanism designed to dynamically capture and model the correspondence between audio features and reference images, thereby enabling precise, identity-aware audio-visual integration. Building on this alignment, a motion warping strategy further refines the results by spatially deforming reference images to match the target audio-driven configuration. A dedicated refinement process then mitigates occlusion artifacts and enhances the preservation of fine-grained textures, such as mouth details and skin features. Extensive qualitative and quantitative evaluations demonstrate that IPTalker consistently outperforms existing approaches in terms of realism, lip synchronization, and identity retention, establishing a new state of the art for high-quality, identity-consistent video dubbing.

Figures

Figures reproduced from arXiv: 2501.04586 by the authors.

Figure 1
Figure 1. Our proposed IPTalker generates dubbed videos using a few reference [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The framework of our method consists of three components: [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Illustration of the Audio-Visual Alignment Unit (AVAU) and the cross-modalities encoder [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: We obtain a precise mask by calculating the convex hull of the lower [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: We paste the generated face onto the original frame using a Gaussian [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Synthetic results of our method. IPTalker generates high-resolution images with precise mouth shapes and realistic faces while effectively preserving [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Qualitative comparisons with state-of-the-art works. The results demonstrate that our model can generate high-resolution images with precise mouth [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Qualitative results of the ablation study. We conduct ablation studies [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 21 canonical work pages

  1. [1]

    A lip sync expert is all you need for speech to lip generation in the wild,

    K. Prajwal, R. Mukhopadhyay, V . P. Namboodiri, and C. Jawahar, “A lip sync expert is all you need for speech to lip generation in the wild,” in ACMMM, 2020

    work page 2020

  2. [2]

    Identity- preserving talking face generation with landmark and appearance priors,

    W. Zhong, C. Fang, Y . Cai, P. Wei, G. Zhao, L. Lin, and G. Li, “Identity- preserving talking face generation with landmark and appearance priors,” in CVPR, 2023

    work page 2023

  3. [3]

    Makelttalk: speaker-aware talking-head animation,

    Y . Zhou, X. Han, E. Shechtman, J. Echevarria, E. Kalogerakis, and D. Li, “Makelttalk: speaker-aware talking-head animation,” TOG, 2020

    work page 2020

  4. [4]

    Towards realistic visual dubbing with heteroge- neous sources,

    T. Xie, L. Liao, C. Bi, B. Tang, X. Yin, J. Yang, M. Wang, J. Yao, Y . Zhang, and Z. Ma, “Towards realistic visual dubbing with heteroge- neous sources,” in ACMMM, 2021

    work page 2021

  5. [5]

    Towards automatic face-to-face translation,

    P. KR, R. Mukhopadhyay, J. Philip, A. Jha, V . Namboodiri, and C. Jawahar, “Towards automatic face-to-face translation,” in ACMMM, 2019

    work page 2019

  6. [6]

    Semantic-aware implicit neural audio-driven video portrait generation,

    X. Liu, Y . Xu, Q. Wu, H. Zhou, W. Wu, and B. Zhou, “Semantic-aware implicit neural audio-driven video portrait generation,” in ECCV, 2022

    work page 2022

  7. [7]

    Learning dynamic facial radiance fields for few-shot talking head synthesis,

    S. Shen, W. Li, Z. Zhu, Y . Duan, J. Zhou, and J. Lu, “Learning dynamic facial radiance fields for few-shot talking head synthesis,” in ECCV, 2022

    work page 2022

  8. [8]

    Ad-nerf: Audio driven neural radiance fields for talking head synthesis,

    Y . Guo, K. Chen, S. Liang, Y .-J. Liu, H. Bao, and J. Zhang, “Ad-nerf: Audio driven neural radiance fields for talking head synthesis,” in ICCV, 2021

    work page 2021

Show all 32 references
  1. [9]

    Expressive talking head generation with granular audio-visual control,

    B. Liang, Y . Pan, Z. Guo, H. Zhou, Z. Hong, X. Han, J. Han, J. Liu, E. Ding, and J. Wang, “Expressive talking head generation with granular audio-visual control,” in CVPR, 2022

    2022

  2. [10]

    Vfhq: A high-quality dataset and benchmark for video face super-resolution,

    L. Xie, X. Wang, H. Zhang, C. Dong, and Y . Shan, “Vfhq: A high-quality dataset and benchmark for video face super-resolution,” in CVPR, 2022

    2022

  3. [11]

    Flow-guided one-shot talking face generation with a high-resolution audio-visual dataset,

    Z. Zhang, L. Li, Y . Ding, and C. Fan, “Flow-guided one-shot talking face generation with a high-resolution audio-visual dataset,” in CVPR, 2021

    2021

  4. [12]

    A morphable model for the synthesis of 3d faces,

    V . Blanz, T. Vetter, and A. Rockwood, “A morphable model for the synthesis of 3d faces,” acm Si , pp. 187–194, 2002

    2002

  5. [13]

    Styletalk: One-shot talking head generation with controllable speaking styles,

    Y . Ma, S. Wang, Z. Hu, C. Fan, T. Lv, Y . Ding, Z. Deng, and X. Yu, “Styletalk: One-shot talking head generation with controllable speaking styles,” in AAAI, 2023

    2023

  6. [14]

    Nerf: Representing scenes as neural radiance fields for view synthesis,

    B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,” Communications of the ACM , vol. 65, no. 1, pp. 99–106, 2021

    2021

  7. [15]

    Attention is all you need,

    A. Vaswani, “Attention is all you need,” NeurIPS, 2017

    2017

  8. [16]

    Dinet: Deformation inpainting network for realistic face visually dubbing on high resolution video,

    Z. Zhang, Z. Hu, W. Deng, C. Fan, T. Lv, and Y . Ding, “Dinet: Deformation inpainting network for realistic face visually dubbing on high resolution video,” in AAAI, 2023

    2023

  9. [17]

    Adaptive affine transformation: A simple and effective operation for spatial misaligned image generation,

    Z. Zhang and Y . Ding, “Adaptive affine transformation: A simple and effective operation for spatial misaligned image generation,” in ACMMM, 2022

    2022

  10. [18]

    Pirenderer: Controllable portrait image generation via semantic neural rendering,

    Y . Ren, G. Li, Y . Chen, T. H. Li, and S. Liu, “Pirenderer: Controllable portrait image generation via semantic neural rendering,” in ICCV, 2021

    2021

  11. [19]

    Arbitrary style transfer in real-time with adaptive instance normalization,

    X. Huang and S. Belongie, “Arbitrary style transfer in real-time with adaptive instance normalization,” in ICCV, 2017

    2017

  12. [20]

    First order motion model for image animation,

    A. Siarohin, S. Lathuili `ere, S. Tulyakov, E. Ricci, and N. Sebe, “First order motion model for image animation,” NeurIPS, 2019

    2019

  13. [21]

    Mediapipe: A framework for building perception pipelines,

    C. Lugaresi, J. Tang, H. Nash, C. McClanahan, E. Uboweja, M. Hays, F. Zhang, C.-L. Chang, M. G. Yong, J. Lee et al. , “Mediapipe: A framework for building perception pipelines,” arXiv preprint arXiv:1906.08172, 2019

    1906 arXiv

  14. [22]

    Semantic image synthesis with spatially-adaptive normalization,

    T. Park, M.-Y . Liu, T.-C. Wang, and J.-Y . Zhu, “Semantic image synthesis with spatially-adaptive normalization,” in CVPR, 2019

    2019

  15. [23]

    Perceptual losses for real-time style transfer and super-resolution,

    J. Johnson, A. Alahi, and L. Fei-Fei, “Perceptual losses for real-time style transfer and super-resolution,” in ECCV, 2016

    2016

  16. [24]

    Least squares generative adversarial networks,

    X. Mao, Q. Li, H. Xie, R. Y . Lau, Z. Wang, and S. Paul Smolley, “Least squares generative adversarial networks,” in ICCV, 2017

    2017

  17. [25]

    Very deep convolutional networks for large-scale image recognition,

    K. Simonyan, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 , 2014

    2014 arXiv

  18. [26]

    Dreamtalk: When expressive talking head generation meets diffusion probabilistic models,

    Y . Ma, S. Zhang, J. Wang, X. Wang, Y . Zhang, and Z. Deng, “Dreamtalk: When expressive talking head generation meets diffusion probabilistic models,” arXiv preprint arXiv:2312.09767 , 2023

    2023 arXiv

  19. [27]

    Audio2head: Audio- driven one-shot talking-head generation with natural head motion,

    S. Wang, L. Li, Y . Ding, C. Fan, and X. Yu, “Audio2head: Audio- driven one-shot talking-head generation with natural head motion,”arXiv preprint arXiv:2107.09293, 2021

    2021 arXiv

  20. [28]

    Sadtalker: Learning realistic 3d motion coefficients for stylized audio-driven single image talking face animation,

    W. Zhang, X. Cun, X. Wang, Y . Zhang, X. Shen, Y . Guo, Y . Shan, and F. Wang, “Sadtalker: Learning realistic 3d motion coefficients for stylized audio-driven single image talking face animation,” in CVPR, 2023

    2023

  21. [29]

    Hallo: Hierarchical audio-driven visual synthesis for portrait image animation,

    M. Xu, H. Li, Q. Su, H. Shang, L. Zhang, C. Liu, J. Wang, L. Van Gool, Y . Yao, and S. Zhu, “Hallo: Hierarchical audio-driven visual synthesis for portrait image animation,” arXiv preprint arXiv:2406.08801 , 2024

    2024 arXiv

  22. [30]

    Aniportrait: Audio-driven synthesis of photorealistic portrait animation,

    H. Wei, Z. Yang, and Z. Wang, “Aniportrait: Audio-driven synthesis of photorealistic portrait animation,” arXiv preprint arXiv:2403.17694 , 2024

    2024 arXiv

  23. [31]

    Image quality assessment: Form error visibility to structural similarity,

    Z. Wang, “Image quality assessment: Form error visibility to structural similarity,” TIP, vol. 13, no. 4, pp. 604–606, 2004

    2004

  24. [32]

    The unreasonable effectiveness of deep features as a perceptual metric,

    R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in CVPR, 2018

    2018

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.