Pith. sign in

REVIEW 4 major objections 5 minor 35 references

IF-MDM: Implicit Face Motion Diffusion Model for High-Fidelity Realtime Talking Head Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Talking-head video can be generated in real time from one portrait and speech by making motion itself a compact, decoder-aware code that a diffusion model learns to generate.

desk verdict The core idea is solid and the speed is real, but the paper's own lip-sync numbers contradict its 'superior performance' claim and the main evaluation setup is partially undisclosed. read the letter →

arxiv 2412.04000 v2 pith:I24DISZL submitted 2024-12-05 cs.CV

Sejong Yang , Seoung Wug Oh , Yang Zhou , Seon Joo Kim This is my paper · ORCID
classification cs.CV
keywords talkingheadgenerationdiffusionmodelimplicitmotionrepresentationvideolipsyncclassifier-freeguidancetransferreal-timesynthesis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that talking-head video can be generated in real time from one portrait and speech by making the motion itself a compact, identity-aware code rather than a 3D model or landmark set. The system first learns a 20-D per-frame implicit motion vector through self-supervised inter-frame reconstruction, then trains a lightweight diffusion transformer to generate those motion codes from audio. The paper reports 512x512 video at 30-45 fps, with better image quality, identity preservation, temporal consistency, and lip sync than a full-image video-diffusion baseline, and without the floating-head harmonization artifacts of explicit-model methods. It also introduces the motion sequence's mean and standard deviation as inference-time knobs that trade expressiveness against visual quality and lip-sync accuracy. The authors are candid that lip sync still trails the explicit 3DMM baseline, pointing to the compact motion bottleneck as the limiting factor.

What carries the argument

The load-bearing object is the implicit motion sequence $M=\{m_f\}_{f=1}^{N_f}$, a per-frame 20-D vector extracted from the visual encoder's bottleneck via a linear decomposition layer in stage 1 and generated by a diffusion transformer in stage 2. The sequence mean $m_\mu$ and standard deviation $m_\sigma$ are injected as extra conditions during classifier-free guidance, giving the model global cues about overall motion dynamics and giving users a single knob for pose/translation (mean) and expression intensity (std). The 'decoder-aware' property -- the motion codes are learned jointly with the generator that turns them back into frames -- is what lets such a compact code carry enough appearance information for high-fidelity reconstruction.

What would settle it

Take the frozen stage-1 encoder and decoder, feed ground-truth motion codes (not audio-generated) into the decoder for HDTF test videos, and measure LSE-D/LSE-C on the mouth region. If this replay trails Real3DPortrait by the same margin as full IF-MDM, the 20-D bottleneck is what limits lip sync; if the replay closes the gap, the loss is in the audio-to-motion generation instead.

Watch

Extended reading notes

Core claim

The central claim is that a highly compressed, decoder-aware implicit motion representation can serve as the generative target of a diffusion model for talking heads. Instead of mapping audio to 3D morphable model parameters or facial landmarks, the model regresses sequences of 20-dimensional vectors $m_f\in[-1,1]^{20}$ that carry meaning only through the generator they were trained with. The paper argues that because these codes are appearance-aware and directly decodable, they avoid error-prone warping, rendering, and background/torso harmonization, while the small state space lets a diffusion transformer generate motion at interactive rates. The paper further claims that feeding the motion sequence's mean $m_\mu$ and standard deviation $m_\sigma$ as additional classifier-free conditions compensates for the lack of spatial disentanglement in implicit codes, enabling audio-conditioned learning and user control over pose stability versus expression intensity.

Load-bearing premise

The load-bearing premise is that the 20-D per-frame motion code learned by reconstructing neighboring frames retains enough fine-grained mouth detail for lip sync; if it discards lip information, the audio-conditioned diffusion generator cannot recover it.

Editorial extensions

If this is right

  • A single portrait plus audio can drive a talking head at 512x512 in real time (30-45 fps depending on diffusion steps), making interactive avatars practical.
  • Replacing explicit face models with decoder-aware implicit motion avoids the torso and background harmonization artifacts that make rendered faces look like floating heads.
  • Users can tune the motion mean and standard deviation to shift between stable, lip-sync-accurate dubbing and more animated, expressive avatars without retraining.
  • Because appearance and motion are disentangled, new identities can be animated from one image, avoiding the per-person retraining needed by neural-rendering avatars.
  • Increasing the number of diffusion steps improves FID and identity preservation at the cost of speed, providing a quality-latency trade-off for deployment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct follow-up test the paper does not run: widen the stage-1 bottleneck or add a lip-region reconstruction weight, and check whether LSE-D/C improve while 45 fps is retained; this would localize the lip-sync ceiling.
  • The motion-statistics conditioning is a generic global control signal, so the same trick could help audio-conditioned generation of other implicit motion representations, such as whole-body avatars, where no per-part spatial supervision exists.
  • Because the motion code is decoder-aware, the audio-conditioned generator could in principle be reused with a different renderer, such as a compact avatar codec, without retraining the motion model; the paper does not test this.
  • The reported trade-off between motion standard deviation and LSE-D suggests a deployment policy -- low standard deviation for dubbing, high for expressive characters -- that the paper stops short of prescribing.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes IF-MDM, a two-stage talking-head generation framework. Stage 1 learns a 20-D per-frame implicit motion representation through inter-frame reconstruction, and Stage 2 trains a diffusion transformer to generate motion sequences conditioned on audio, with the motion mean and standard deviation used as additional guidance. At inference, motion degree can be controlled by adjusting those statistics. The authors claim real-time 512x512 generation at up to 45 FPS and 'superior performance' over both explicit face models and video diffusion models. The experiments compare against Real3DPortrait and AniPortrait on HDTF, reporting FID, CSIM, video temporal consistency, and lip-sync metrics, plus ablations on diffusion steps and motion statistics.

Significance. The paper's core idea is valuable: replacing explicit face models with a compact, appearance-aware implicit motion code allows a lightweight diffusion model to run in real time, and the motion-statistics conditioning provides a simple mechanism for controlling expressiveness. If the performance were validated as claimed, the work would be a meaningful contribution to efficient talking-head generation. The paper ships with a code-release promise and the method achieves impressive inference speeds (30.9 FPS in Table 1, up to 45.75 FPS in Table 2). However, the significance is currently undermined by multiple inconsistencies in the reported numbers and by a central claim that is contradicted by the paper's own lip-sync metrics.

major comments (4)
  1. [Abstract and Section 4.3 (Table 1)] The abstract and Section 5 claim 'superior performance over existing diffusion and explicit face models.' Yet Table 1 shows IF-MDM is markedly worse than Real3DPortrait on lip sync: LSE-D 11.04 vs. 8.23 and LSE-C 3.88 vs. 6.58. Section 4.3 itself admits 'relatively lower lip sync quality' than explicit face models. The central claim of superiority is therefore not supported by the evidence. The authors should either substantially temper the claim (e.g., claim superiority only over video diffusion models in quality and over all baselines in speed) or provide a principled multi-metric aggregation justifying the word 'superior.'
  2. [Tables 1 and 3] There is a direct numerical inconsistency between the main results and the ablation. Table 1 reports IF-MDM with FID 42.84 and LSE-D 11.04. Table 3's row with mμ = m' and no motion std reports the same FID 42.84 but LSE-D 12.79. If these are the same configuration, one of the numbers is wrong; if they are different configurations, the settings for Table 1 are undisclosed. The evaluation protocol for the headline numbers must be specified exactly, including which mμ and mσ values are used at inference.
  3. [Section 3.3 and Section 4.5 (Table 3)] The motion statistics mμ and mσ are computed from the target motion sequence during training, but at inference they are set by the user. The paper never states which values were used for the main comparison in Table 1. If the quantities are taken from the ground-truth sequence during evaluation, the comparison is unfair because baselines do not have access to such target statistics; if they are user-set, the exact values are essential for reproducibility. This issue is load-bearing because the motion statistics are the mechanism the authors credit for overcoming the implicit motion's difficulty with subtle lip movements.
  4. [Section 3.3 and Table 3] The paper acknowledges in Section 3.3 that implicit motion 'lacks spatial disentanglement, making it challenging... particularly for subtle lip movements.' The lip-sync results are consistent with that limitation: even the best LSE-D in Table 3 (9.78 with mσ=0.3, no mμ) does not reach Real3DPortrait's 8.23, and the default configuration (mμ=m') gives 12.79. These numbers show that the motion-statistics guidance does not recover the fine articulatory detail lost in Stage 1. This structural limitation directly undermines the claim of 'high-fidelity' talking head generation and should be addressed head-on, for example by adding a lip-sync loss or a more detailed motion representation for the mouth region.
minor comments (5)
  1. [Table 2] The column header 'LSE-D↑' is mislabeled: LSE-D is a distance metric where lower is better, as indicated in Table 1. The arrow should be '↓'.
  2. [Section 4.4, Table 2] The text says 'increasing the number of diffusion steps generally improves visual quality, as evidenced by lower FID and higher CSIM.' However, CSIM is not monotonic in Table 2: it is 0.981 at 50 steps, 0.984 at 100, then 0.982, 0.981, and 0.980 at 200, 500, and 1000 steps. The claim should be restricted to FID, or the wording should be made approximate.
  3. [Section 5] The conclusion states 'Our model achieves real-time performance, generating 512x512 resolution videos at 45 fps.' This is imprecise: Table 1 reports 30.90 fps, and 45 FPS is only achieved with 50 diffusion steps, which comes with degraded FID (50.22 vs. 42.35). The 'up to' qualifier should be carried into the conclusion.
  4. [Section 3.1, Eq. (3)] The classifier-free guidance equation is written as ϵθ(xt, t, c + s·ĉ), but the standard formulation is ϵθ(xt, t, c) + s·(ϵθ(xt, t, c) − ϵθ(xt, t, ∅)). The notation in Eq. (3) is nonstandard and likely a typo, since the text describes adding a noise component. This should be corrected for clarity.
  5. [General] The reference list contains a duplicated entry for [6] and [7], both for Ho and Salimans 'Classifier-free diffusion guidance' (NeurIPS Workshop 2021 and 2022). These should be merged or correctly differentiated.

Circularity Check

0 steps flagged · score 0.0 of 10

No material circularity found: the two-stage pipeline is a conventional conditional generation framework, and the motion-statistics conditioning is user-supplied at inference rather than a predicted quantity.

full rationale

We walked the paper's claimed derivation chain. Stage 1 learns a compact 20-D implicit motion code through inter-frame reconstruction, explicitly building on LIA [26] (an independent prior work, not by the present authors). Stage 2 trains a diffusion transformer to generate that motion sequence conditioned on audio and on motion statistics (mean m_mu and standard deviation m_sigma). The key potential circularity concern is the motion statistics: they are computed from the target motion sequence during training, but at inference they are set by the user, inherited from the input image m', or taken from a previously generated frame. They are conditional inputs, not outputs that the model derives from its own generation. Thus the model is not predicting its own conditioning statistics, and the reported controllability is a standard conditional-generation design rather than an equation-level reduction. The paper contains one self-citation ([9], a prior motion-transfer paper by two of the same authors), but it is used only as background among several motion-transfer references and is not load-bearing for any central claim. The architecture is otherwise based on LIA [26] and DiT [14], both external works. Evaluations use external baselines (Real3DPortrait, AniPortrait), external datasets (HDTF, CelebV-Text), and standard metrics (FID, CSIM, LSE-D/C). The fact that the paper's lip-sync metrics are worse than Real3DPortrait and that Table 1 values do not exactly match any single ablation row is an empirical/reporting issue, not circularity. No step in the derivation is equivalent by construction to its own input, so the appropriate finding is no significant circularity.

Assumptions & free parameters 6 free parameters · 6 assumptions · 2 invented entities

The central performance and controllability claims rest on the 20-D implicit motion representation and the motion statistics knobs. The dimensionality is a chosen bottleneck with no demonstrated guarantee that lip details survive; the motion statistics are user-set at inference and their default values are not reported. The paper also omits guidance scale and loss weights. These are the parameters the reader would need to reproduce the headline numbers.

free parameters (6)
  • motion vector dimension = 20
    Chosen bottleneck; compactness drives speed but no analysis of whether 20 dims retain lip detail; central to the method.
  • motion standard deviation at inference (m_sigma) = unspecified; ablation uses 0.3, 0.6, 0.9
    Control knob for motion intensity; reported quality in Table 1 depends on the unstated default value; Table 3 shows FID varies from 46 to 103 with this knob.
  • motion mean strategy at inference (m_mu) = m' (input image motion) for default; m_hat_{n-1} alternative
    Choice changes FID from 28.83 to 46.21 (Table 3), so the headline numbers depend on this user-set quantity.
  • number of diffusion steps = 100 (default); 50 for max FPS claim
    Speed-quality tradeoff shown in Table 2; 'up to 45 fps' is with 50 steps and worse FID.
  • classifier-free guidance scale s = not stated
    Guidance scale strongly affects sample quality and lip sync in diffusion models; omitted.
  • stage 1/2 loss weights and training schedules = not stated
    Needed for replication; no training hyperparameters given.
assumptions (6)
  • domain assumption The LIA-style inter-frame reconstruction with a 20-D bottleneck learns a face motion representation that is appearance-independent and audio-predictable.
    Invoked in Section 3.2 as the basis for separating appearance and motion; if false, the diffusion generator cannot produce identity-preserving speech motion.
  • domain assumption Wav2Vec 2.0 speech embeddings are a sufficient condition for lip and expression motion.
    Section 4.1: speech vectors come from Wav2Vec; no analysis of what information is lost.
  • ad hoc to paper Motion statistics (mean and std over the generated sequence) can be set by the user without access to the target sequence and still yield natural motion.
    Section 3.3/3.4: this is a design choice; default value is not specified, and the claimed controllability relies on it.
  • ad hoc to paper tanh normalization of motion vectors bounds m_sigma in [0,1] and preserves useful variation.
    Section 3.4: normalization ensures predictable behavior of m_sigma; no justification that tanh is appropriate.
  • standard math Diffusion and DiT training assumptions (Gaussian noise, classifier-free guidance, learned beta schedule) behave as in prior work.
    Sections 3.1 and 3.3 borrow standard results from Ho et al. and Peebles & Xie.
  • domain assumption The HDTF test split of approximately 60 videos and the metrics FID, CSIM, VideoScore-TC, LSE-D, LSE-C are accepted measures of talking-head quality.
    Section 4.2 establishes the evaluation protocol; the paper does not validate metric reliability for this task.
invented entities (2)
  • implicit motion vector (20-D per frame)
    purpose: Compact appearance-aware motion representation used to drive the decoder
    Learned latent with no external or interpretable handle; existence is only evidenced by the system's own reconstruction and downstream metrics.
  • motion statistics conditioning (m_mu, m_sigma)
    purpose: Conditional guidance to improve lip alignment and to control motion degree
    Summary statistics of the latent motion sequence; at inference they are user-set values, not predicted from audio, and no default is specified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of IF-MDM: Implicit Face Motion Diffusion Model for High-Fidelity Realtime Talking Head Generation." pith.science (2026). https://pith.science/paper/I24DISZL

@misc{pith2026241204000,
  author       = {Pith},
  title        = {Pith review of: IF-MDM: Implicit Face Motion Diffusion Model for High-Fidelity Realtime Talking Head Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I24DISZL}},
  note         = {Machine review of arXiv:2412.04000}
}
read the original abstract

We introduce a novel approach for high-resolution talking head generation from a single image and audio input. Prior methods using explicit face models, like 3D morphable models (3DMM) and facial landmarks, often fall short in generating high-fidelity videos due to their lack of appearance-aware motion representation. While generative approaches such as video diffusion models achieve high video quality, their slow processing speeds limit practical application. Our proposed model, Implicit Face Motion Diffusion Model (IF-MDM), employs implicit motion to encode human faces into appearance-aware compressed facial latents, enhancing video generation. Although implicit motion lacks the spatial disentanglement of explicit models, which complicates alignment with subtle lip movements, we introduce motion statistics to help capture fine-grained motion information. Additionally, our model provides motion controllability to optimize the trade-off between motion intensity and visual quality during inference. IF-MDM supports real-time generation of 512x512 resolution videos at up to 45 frames per second (fps). Extensive evaluations demonstrate its superior performance over existing diffusion and explicit face models. The code will be released publicly, available alongside supplementary materials. The video results can be found on https://bit.ly/ifmdm_supplementary.

Figures

Figures reproduced from arXiv: 2412.04000 by the authors.

Figure 1
Figure 1. The main difference of Implicit Face Motion Diffusion [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The inference pipeline of the Implicit Face Motion Diffusion Model (IF-MDM). [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The training pipeline of our framework and the detailed architecture of the implicit motion generator. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The detailed architecture of the stage 1 models. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: The qualitative results of HDTF datasets with baselines. The output of Real3DPortrait exhibits good lipsync quality, yet gives [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 7
Figure 7. Figure 7: The examples with motion standard [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 6
Figure 6. Figure 6: The qualitative results of the talking head generation [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 31 canonical work pages

  1. [1]

    wav2vec 2.0: A framework for self-supervised learning of speech representations

    Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in Neural In- formation Processing Systems (NeuIPS) , 33:12449–12460,

    work page

  2. [2]

    Gaus- siantalker: Real-time high-fidelity talking head synthesis with audio-driven 3d gaussian splatting

    Kyusun Cho, Joungbin Lee, Heeji Yoon, Yeobin Hong, Jaehoon Ko, Sangjun Ahn, and Seungryong Kim. Gaus- siantalker: Real-time high-fidelity talking head synthesis with audio-driven 3d gaussian splatting. arXiv preprint arXiv:2404.16012, 2024. 2

    arXiv 2024

  3. [3]

    Out of time: auto- mated lip sync in the wild

    Joon Son Chung and Andrew Zisserman. Out of time: auto- mated lip sync in the wild. In Asian Conference on Com- puter Vision Workshops(ACCV - Workshops) , pages 251–

    work page

  4. [4]

    Generative adversarial nets

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in Neural Information Processing Systems (NeuIPS), 27, 2014. 1

    work page 2014

  5. [5]

    Ad-nerf: Audio driven neural ra- diance fields for talking head synthesis

    Yudong Guo, Keyu Chen, Sen Liang, Yong-Jin Liu, Hujun Bao, and Juyong Zhang. Ad-nerf: Audio driven neural ra- diance fields for talking head synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 5784–5794, 2021. 2

    work page 2021

  6. [6]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. Advances in Neural Information Processing Sys- tems Workshop (NeuIPS Workshop), 2021. 3

    work page 2021

  7. [7]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. Advances in Neural Information Processing Sys- tems Workshops (NeuIPS Workshops), 2022. 2

    work page 2022

  8. [8]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. Advances in Neural Information Processing Systems (NeuIPS), 33:6840–6851, 2020. 1, 3

    work page 2020

Show all 35 references
  1. [9]

    Cross-identity motion transfer for arbitrary objects through pose-attentive video reassembling

    Subin Jeon, Seonghyeon Nam, Seoung Wug Oh, and Seon Joo Kim. Cross-identity motion transfer for arbitrary objects through pose-attentive video reassembling. In Pro- ceedings of Proceedings of European Conference on Com- puter Vision (ECCV), pages 292–308. Springer, 2020. 2, 3

    2020

  2. [10]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. Inter- national Conference for Learning Representations (ICLR) ,

  3. [11]

    Neural human performer: Learning generalizable ra- diance fields for human performance rendering

    Youngjoong Kwon, Dahun Kim, Duygu Ceylan, and Henry Fuchs. Neural human performer: Learning generalizable ra- diance fields for human performance rendering. Advances in Neural Information Processing Systems (NeuIPS) , 34: 24741–24752, 2021. 3

    2021

  4. [12]

    Expressive talking head generation with granular audio-visual control

    Borong Liang, Yan Pan, Zhizhi Guo, Hang Zhou, Zhibin Hong, Xiaoguang Han, Junyu Han, Jingtuo Liu, Errui Ding, and Jingdong Wang. Expressive talking head generation with granular audio-visual control. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco...

    2022

  5. [13]

    Semantic-aware implicit neural audio- driven video portrait generation

    Xian Liu, Yinghao Xu, Qianyi Wu, Hang Zhou, Wayne Wu, and Bolei Zhou. Semantic-aware implicit neural audio- driven video portrait generation. In Proceedings of Proceed- ings of European Conference on Computer Vision (ECCV) , pages 106–125. Springer, 2022. 1, 2

    2022

  6. [14]

    Scalable diffusion mod- els with transformers

    William Peebles and Saining Xie. Scalable diffusion mod- els with transformers. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision (ICCV) , pages 4195–4205, 2023. 2, 5

    2023

  7. [15]

    A lip sync expert is all you need for speech to lip generation in the wild

    KR Prajwal, Rudrabha Mukhopadhyay, Vinay P Nambood- iri, and CV Jawahar. A lip sync expert is all you need for speech to lip generation in the wild. In ACM International Conference on Multimedia (ACMM), pages 484–492, 2020. 2, 8

    2020

  8. [16]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684–10695, 2022. 1

    2022

  9. [17]

    Relightable gaussian codec avatars

    Shunsuke Saito, Gabriel Schwartz, Tomas Simon, Junxuan Li, and Giljoo Nam. Relightable gaussian codec avatars. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 130–141, 2024. 3

    2024

  10. [18]

    Difftalk: Crafting diffusion models for generalized audio-driven portraits animation

    Shuai Shen, Wenliang Zhao, Zibin Meng, Wanhua Li, Zheng Zhu, Jie Zhou, and Jiwen Lu. Difftalk: Crafting diffusion models for generalized audio-driven portraits animation. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), 2023. 1, 2

    2023

  11. [19]

    First order motion model for im- age animation

    Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe. First order motion model for im- age animation. Advances in Neural Information Processing Systems (NeuIPS), 32, 2019. 2, 3

    2019

  12. [20]

    Animating arbitrary objects via deep motion transfer

    Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe. Animating arbitrary objects via deep motion transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2377–2386, 2019

    2019

  13. [21]

    Motion representations for artic- ulated animation

    Aliaksandr Siarohin, Oliver J Woodford, Jian Ren, Menglei Chai, and Sergey Tulyakov. Motion representations for artic- ulated animation. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR) , pages 13653–13662, 2021. 2, 3

    2021

  14. [22]

    Diffused heads: Diffusion models beat gans on talking-face genera- tion

    Michał Stypułkowski, Konstantinos V ougioukas, Sen He, Maciej Zi˛ eba, Stavros Petridis, and Maja Pantic. Diffused heads: Diffusion models beat gans on talking-face genera- tion. In IEEE Winter Conference on Applications of Com- puter Vision (WACV), pages 5091–5100, 2024. 1, 2

    2024

  15. [23]

    A-nerf: Articulated neural radiance fields for learn- ing human shape, appearance, and pose

    Shih-Yang Su, Frank Yu, Michael Zollhöfer, and Helge Rhodin. A-nerf: Articulated neural radiance fields for learn- ing human shape, appearance, and pose. Advances in Neural Information Processing Systems (NeuIPS), 34:12278–12291,

  16. [24]

    Real-time neural radiance talking portrait synthesis via audio-spatial decomposition

    Jiaxiang Tang, Kaisiyuan Wang, Hang Zhou, Xiaokang Chen, Dongliang He, Tianshu Hu, Jingtuo Liu, Gang Zeng, and Jingdong Wang. Real-time neural radiance talking portrait synthesis via audio-spatial decomposition. arXiv preprint arXiv:2211.12368, 2022. 2

    2022 arXiv

  17. [25]

    To- wards real-world blind face restoration with generative facial 10 prior

    Xintao Wang, Yu Li, Honglun Zhang, and Ying Shan. To- wards real-world blind face restoration with generative facial 10 prior. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 9168– 9178, 2021. 6

    2021

  18. [26]

    Latent image animator: Learning to animate im- ages via latent space navigation

    Yaohui Wang, Di Yang, Francois Bremond, and Antitza Dantcheva. Latent image animator: Learning to animate im- ages via latent space navigation. International Conference for Learning Representations (ICLR), 2022. 2, 3, 4

    2022

  19. [27]

    Aniportrait: Audio-driven synthesis of photorealistic portrait animation

    Huawei Wei, Zejun Yang, and Zhisheng Wang. Aniportrait: Audio-driven synthesis of photorealistic portrait animation. arXiv preprint arXiv:2403.17694, 2024. 1, 2, 6, 7

    2024 arXiv

  20. [28]

    Deep 3d portrait from a single image

    Sicheng Xu, Jiaolong Yang, Dong Chen, Fang Wen, Yu Deng, Yunde Jia, and Xin Tong. Deep 3d portrait from a single image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7710–7720, 2020. 2

    2020

  21. [29]

    Geneface: Generalized and high-fidelity audio-driven 3d talking face synthesis

    Zhenhui Ye, Ziyue Jiang, Yi Ren, Jinglin Liu, Jinzheng He, and Zhou Zhao. Geneface: Generalized and high-fidelity audio-driven 3d talking face synthesis. International Con- ference for Learning Representations (ICLR), 2023. 1, 2

    2023

  22. [30]

    Real3d-portrait: One-shot realistic 3d talk- ing portrait synthesis

    Zhenhui Ye, Tianyun Zhong, Yi Ren, Jiaqi Yang, Weichuang Li, Jiawei Huang, Ziyue Jiang, Jinzheng He, Rongjie Huang, Jinglin Liu, et al. Real3d-portrait: One-shot realistic 3d talk- ing portrait synthesis. International Conference for Learning Representations (ICLR), 2024. 1, 2, 6, 7

    2024

  23. [31]

    CelebV-Text: A large-scale facial text-video dataset

    Jianhui Yu, Hao Zhu, Liming Jiang, Chen Change Loy, Wei- dong Cai, and Wayne Wu. CelebV-Text: A large-scale facial text-video dataset. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR) ,

  24. [32]

    Flow-guided one-shot talking face generation with a high-resolution audio-visual dataset

    Zhimeng Zhang, Lincheng Li, Yu Ding, and Changjie Fan. Flow-guided one-shot talking face generation with a high-resolution audio-visual dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3661–3670, 2021. 6

    2021

  25. [33]

    Identity- preserving talking face generation with landmark and ap- pearance priors

    Weizhi Zhong, Chaowei Fang, Yinqi Cai, Pengxu Wei, Gangming Zhao, Liang Lin, and Guanbin Li. Identity- preserving talking face generation with landmark and ap- pearance priors. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 9729–...

    2023

  26. [34]

    Pose-controllable talking face generation by implicitly modularized audio-visual rep- resentation

    Hang Zhou, Yasheng Sun, Wayne Wu, Chen Change Loy, Xiaogang Wang, and Ziwei Liu. Pose-controllable talking face generation by implicitly modularized audio-visual rep- resentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), page...

    2021

  27. [35]

    Makelttalk: speaker-aware talking-head animation

    Yang Zhou, Xintong Han, Eli Shechtman, Jose Echevar- ria, Evangelos Kalogerakis, and Dingzeyu Li. Makelttalk: speaker-aware talking-head animation. ACM Transactions On Graphics (TOG), 39(6):1–15, 2020. 1, 2 11

    2020

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.