Pith. sign in

REVIEW 3 major objections 6 minor 87 references

STARCaster claims that free-viewpoint talking portraits—faces that speak in sync with audio while rotating to new camera angles—can be generated by a single 2D video diffusion model, with no explicit 3D reconstruction.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 16:26 UTC pith:4SRPIE3J

load-bearing objection A credible, well-engineered talking-portrait synthesis paper whose core novelty — composing independently trained audio and camera streams at inference — is exactly where the evidence is thinnest. the 3 major comments →

arxiv 2512.13247 v2 pith:4SRPIE3J submitted 2025-12-15 cs.CV

STARCaster: Spatio-Temporal AutoRegressive Video Diffusion for Identity- and View-Aware Talking Portraits

Foivos Paraperas Papantoniou , Stathis Galanakis , Rolandos Alexandros Potamias , Bernhard Kainz , Stefanos Zafeiriou This is my paper
classification cs.CV
keywords talking portrait generationvideo diffusionnovel view synthesisidentity embeddingaudio-driven animationautoregressive generationself-forcing traininglip-reading supervision
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper claims that free-viewpoint talking portraits—faces that both speak in sync with audio and rotate to new camera angles—can be generated by a single 2D video diffusion model, without reconstructing a 3D head. The authors argue that explicit 3D inversion (tri-planes, NeRFs) is unnecessary and error-prone; instead, view control can be learned as a spatial video-generation task from synthetic multi-view trajectories. Their model, STARCaster, extends an identity-consistent image diffusion backbone into an autoregressive video model with separate attention streams for identity, audio, and camera. A decoupled training schedule—first audio-driven motion with lip-reading supervision, then self-forcing long-sequence autoregression, then camera-only fine-tuning—lets the model compose all three at inference. If correct, this yields rotatable, audio-synchronized portraits from a single image or even just an identity embedding, with more motion diversity than reference-conditioned baselines.

Core claim

STARCaster claims 3D awareness for talking portraits can be manufactured inside the 2D video domain instead of reconstructed explicitly. The authors extend the identity-consistent face diffusion model Arc2Face into an autoregressive video model, adding temporal transformers and a decoupled multi-source cross-attention with parallel streams for identity, audio, and camera. Training runs in three stages: audio-driven motion with lip-reading supervision; reference-conditioned generation with self-forcing autoregression; and view synthesis by fine-tuning only the camera branch on pseudo multi-view trajectories from synthetic 3D heads. At inference the audio and camera streams compose to yield ro

What carries the argument

Three mechanisms carry the argument. (1) Decoupled multi-source cross-attention: identity, audio, and camera each have separate key/value projections that share the same query, preserving the frozen Arc2Face identity attention while adding new conditioning. (2) Self-forcing autoregressive training: each segment is conditioned on the model's own previously generated frames rather than ground truth, reducing exposure bias and the static 'copy-paste' look of typical autoregressive methods. (3) Temporal-to-spatial adaptation: view control is recast as video generation, with camera parameters fed through an MLP and fine-tuning on pseudo multi-view trajectory clips rendered from synthetic 3D heads

Load-bearing premise

The load-bearing premise is that a model trained in separate stages—audio-driven motion first, camera-only view synthesis later—can compose both streams at inference without joint audio-plus-camera training, and still produce speech-synced motion that stays identity-consistent as the viewpoint rotates.

What would settle it

Run the trained model with a fixed audio track while sweeping camera azimuth across the supported range (about ±70 degrees), and measure lip-sync error and identity similarity per view. If lip-sync or identity degrades sharply at yaw angles beyond the frontal range, while the same sweep without audio yields clean rotation, the decoupled training premise fails. A second check: evaluate on a held-out real multi-view face dataset whose identities match neither the synthetic heads nor the in-the-wild training frames; poor view accuracy there would indicate the synthetic-to-real transfer does not g

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A single model, not a pipeline of 3D fitting plus rendering, can produce free-viewpoint talking portraits from one image and an audio track.
  • Identity-only conditioning (an embedding rather than a reference image) supports subject-consistent yet reference-free generation, enabling portraits in novel poses and contexts.
  • View consistency can be learned from synthetic multi-view heads and transfers to real in-the-wild identities, avoiding per-subject 3D optimization.
  • Self-forcing autoregression increases motion diversity and reduces the static facial animations common in long-sequence talking-head generation.
  • Composing the audio and camera streams at inference yields audio-driven animations along arbitrary smooth camera trajectories, including views never seen in training.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If decoupled streams truly interfere minimally, the same recipe could add other controls (expression, lighting, background) without retraining the whole model—each new stream would only need its own cross-attention projections.
  • The synthetic-to-real transfer of view knowledge suggests the limiting factor for free-viewpoint avatars is camera-aware video priors, not 3D reconstruction fidelity; a testable extension is whether pseudo-multi-view data from parametric head models can scale up to replace multi-view capture datasets.
  • The identity-embedding generation mode separates identity from appearance, hinting that control could extend beyond faces to full-body talking agents if a suitable identity prior exists.
  • The lip-reading loss could serve as a general, annotation-free alignment regularizer for audio-conditioned video generation models beyond talking heads.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper presents STARCaster, a video diffusion model for talking portraits built on the Arc2Face identity-aware backbone. It extends the 2D UNet with temporal transformers, a decoupled multi-source cross-attention mechanism over identity, audio, and camera embeddings, and a reference network for appearance conditioning. Training proceeds in three stages: audio-driven motion learning with lip-reading supervision, autoregressive self-forcing for longer-term coherence, and temporal-to-spatial adaptation on synthetic multi-view renderings. The authors claim a unified framework for speech-driven animation, reference-based animation, and novel-view synthesis without explicit 3D representations, reporting state-of-the-art results on TH-1KH, Hallo3, and NeRSemble benchmarks.

Significance. If the claims hold, the paper offers a meaningful step toward unified 2D diffusion-based talking portraits with view control, avoiding per-subject 3D fitting. The proposed self-forcing scheme, lip-reading perceptual loss, and decoupled multi-source conditioning are sensible and clearly motivated. The ablations in Table 3 indicate that both the lip-reading loss and self-forcing contribute to the reported quality. However, the evaluation has significant gaps: the headline capability of combining audio and camera control is never jointly trained or directly validated, the quantitative comparisons lack uncertainty quantification, and the identity metric is aligned with the conditioning signal. These gaps currently prevent the paper from fully supporting its central claim.

major comments (3)
  1. [Sec. 3.4 / Eq. (2)] The most load-bearing issue is the decoupled training of the audio and camera streams. Stages 1–2 train the ID/audio streams without a camera branch; stage 3 'deactivates the audio stream' and fine-tunes only the camera attention. Inference composes them by interpolating camera trajectories and 'animat[ing] them via the audio stream.' Since Eq. (2) sums the ID, audio, and camera attention outputs, the model has never processed audio and camera conditioning in the same forward pass during training. Table 2 reports audio+view results on NeRSemble, but only against 3D-inversion baselines; there is no ablation with the camera or audio stream disabled, no joint fine-tuning variant, and no lip-sync (LSE-C/D) breakdown under camera motion. This is a correctable gap, but it is the central support for the paper's most distinctive capability. I request: (i) a joint fine-tuning stage (even brief) o
  2. [Tables 1–3, Sec. 4] The quantitative comparisons report single point estimates without variance, confidence intervals, or significance tests. Several margins are very small: Table 1 TH-1KH LSE-C is 5.493 (Ours) vs 5.482 (FLOAT), Hallo3 LSE-C is 6.292 vs 6.230 (V-Express), and FID differences are similarly narrow. The user study (20 participants, 15 sets) reports only overall percentages without confidence intervals or pairwise significance. The claims that the method 'consistently surpass[es] prior approaches' are not statistically supported. Please provide bootstrap confidence intervals or multiple-seed results, paired significance tests (e.g., Wilcoxon), and more detail on the user study (ties, per-participant variance, significance).
  3. [Fig. 5, Sec. 4.1] The identity-similarity evaluation in Fig. 5 uses ArcFace cosine similarity between the reference and generated frames. The model is conditioned on the ArcFace embedding of the reference (or on a reference image via Arc2Face, which itself is trained with ArcFace). Thus the metric is partially aligned with the conditioning signal and measures conditioning fidelity more than independent identity preservation. This circularity should be acknowledged, and ideally an independent identity metric (e.g., a different face-recognition embedding or a human identity-judgment test) should be reported before claiming strong identity consistency.
minor comments (6)
  1. [Fig. 6, Sec. 4.1] The text says the user study shows STARCaster 'preferred in the majority of cases,' but the reported percentages are Ours 31%, Hallo3 26%, EchoMimic 23%, FLOAT 20% — a plurality, not a majority. Please rephrase and provide confidence intervals or significance testing for the preferences.
  2. [Sec. 3.4] The phrase 'camera attention layers replacing the audio-specific ones' creates ambiguity about whether the audio projection weights are preserved. Since inference later uses the audio stream, please clarify the exact parameter state after stage 3.
  3. [Table 1, Sec. 4.1] The 'ID-Driven' row reports only LSE and Pose Std, not FID/FVD, yet the text calls this 'state-of-the-art performance.' No ID-driven baselines are compared. Either add such comparisons or temper the claim to 'competitive lip-sync and motion diversity.'
  4. [Table 2, Sec. 4.2] Please specify the number of videos per identity ('100 identities' but '300 view-conditioned animations') and the exact crop/alignment used for PSNR/SSIM/LPIPS, since these metrics are spatially sensitive.
  5. [Eq. (2), Sec. 3.1] In Eq. (2), the unsubscripted K, V in Attention_id are not explicitly defined; state that they correspond to the identity embedding c_id projections and similarly for the audio/camera streams.
  6. [References] References [1–3] are bare URLs with no title/author/year; please reformat them consistently.

Circularity Check

0 steps flagged

No significant circularity: STARCaster's contributions are empirical architectural and training choices, not derivations that reduce to their inputs.

full rationale

No equation-level or construction-level circularity is present. STARCaster extends Arc2Face as a pretrained external backbone; although this is a self-citation, Arc2Face is a published model with public code/weights and is not the paper's claimed result, so it provides independent support. The decoupled training protocol (audio streams in stages 1–2, camera stream in stage 3, inference combining them) creates a genuine train/inference composition gap, but that is a generalization risk, not a definitional equivalence: Eq. (2) is additive multi-source attention, and no parameter is fitted to the target metric then reported as a prediction. The ArcFace cosine-similarity metric in Fig. 5 is aligned with the ArcFace-based identity conditioning, so it is not a fully independent identity-preservation test; this weakens that particular piece of evidence but does not make the model's output equal to its input by construction. The remaining evidence (FID/FVD, LSE, pose diversity, NeRSemble comparisons, ablations of lip-reading loss and self-forcing) is externally benchmarked and does not reduce to the conditioning inputs. No circular step meets the quoted-equivalence bar.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The paper's central claim rests primarily on domain assumptions about the transferability of pre-trained feature extractors and synthetic data, rather than on fitted physical parameters or newly postulated entities. Hand-set hyperparameters that materially affect behavior are listed as free parameters.

free parameters (6)
  • Self-forcing window F = 2
    Number of autoregressive segments used during self-forcing training; authors state larger values are expected to improve performance but exceed GPU memory (Supp. B).
  • Context frames n = 2
    Number of previous frames used as context in autoregressive training; chosen without reported search (Sec. 3.2).
  • Segment length N = 16
    Number of frames generated per segment; used throughout training (Sec. 3.4).
  • LoRA rank = 64
    Rank of LoRA modules in self-attention for reference conditioning (Supp. B).
  • Classifier-free guidance scale = 3
    CFG scale used at inference for all stages (Supp. B).
  • Diffusion/lip-reading loss weights = 1.0 each
    Lip-reading and diffusion losses weighted equally; lip-reading loss activated after 100K iterations (Supp. B).
axioms (5)
  • domain assumption Arc2Face's identity conditioning (ArcFace embedding mapped into CLIP space) provides a faithful ID prior that transfers to video generation.
    STARCaster inherits identity consistency from a frozen Arc2Face encoder; if this prior is weak for unseen identities, ID-consistency claims fail (Sec. 3.1).
  • domain assumption Wav2Vec2 frame-level audio features contain sufficient phonetic content for lip-sync.
    Audio conditioning relies on off-the-shelf wav2vec2 features; if they lack articulatory detail, the lip-reading loss cannot fully compensate (Sec. 3.1).
  • domain assumption The pre-trained lip-reading network trained on LRS3 provides a meaningful perceptual loss for articulation in generated videos.
    The lip-reading MSE loss assumes that the network's features are sensitive to mouth shape and transfer to synthetic images (Sec. 3.3).
  • domain assumption Pseudo multi-view videos rendered from synthetic 3D heads (SphereHead) are representative enough to teach real-image novel-view synthesis.
    Stage 3 uses only synthetic heads for view training; generalization to real portraits depends on this distribution match (Sec. 3.4 and Supp. A).
  • domain assumption Frozen 2D UNet weights remain valid when temporal transformer blocks are inserted.
    The 2D UNet is frozen and only temporal transformers are trained; if network inflation degrades the image prior, identity fidelity is lost (Sec. 3.1).

pith-pipeline@v1.3.0-alltime-deepseek · 17753 in / 13681 out tokens · 117345 ms · 2026-08-03T16:26:59.123249+00:00 · methodology

0 comments
read the original abstract

This paper presents STARCaster, an identity-aware spatio-temporal video diffusion model that addresses both speech-driven portrait animation and dynamic viewpoint control, given an identity embedding or reference image, within a unified framework. Existing 2D speech-to-video diffusion models depend heavily on reference guidance, leading to limited motion diversity. At the same time, 3D-aware animation typically relies on inversion through pretrained tri-plane generators, which often leads to imperfect reconstructions and identity drift. We rethink reference- and geometry-based paradigms in two ways. First, we deviate from strict reference conditioning at pretraining by introducing softer identity constraints. Second, we address 3D awareness implicitly within the 2D video domain by leveraging the inherent multi-view nature of video data. STARCaster adopts a compositional approach progressing from ID-aware motion modeling, to audio-visual synchronization via lip reading-based supervision, and finally to novel view animation through temporal-to-spatial adaptation. To overcome the scarcity of 4D audio-visual data, we propose a decoupled learning approach in which view consistency and temporal coherence are trained independently. Comprehensive evaluations demonstrate that STARCaster generalizes effectively across tasks and identities, consistently surpassing prior approaches in different benchmarks.

Figures

Figures reproduced from arXiv: 2512.13247 by Bernhard Kainz, Foivos Paraperas Papantoniou, Rolandos Alexandros Potamias, Stathis Galanakis, Stefanos Zafeiriou.

Figure 1
Figure 1. Figure 1: STARCaster is a spatio-temporal autoregressive model that unifies speech-driven portrait animation and continuous view syn [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Novel talking instances of arbitrary subjects generated by STARCaster, conditioned on identity features and driving audio. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of STARCaster. We extend an ID-aware backbone into a spatio-temporal autoregressive video diffusion model, which [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Visual comparison with recent talking portrait methods [ [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Distribution (and mean) of cosine similarity scores between ref￾erence ID and generated frames. 31.0% 26.0% 23.0% 20.0% User Study - Motion Naturalness Ours Hallo3 EchoMimic FLOAT [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Audio-driven portrait animations from novel viewpoints [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Audio-driven animations of reference portraits. The [PITH_FULL_IMAGE:figures/full_fig_p008_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Effect of video data cleaning. (Left) Example frames discarded from the talking video datasets due to hands or multiple subjects. (Right) A model trained on the unfiltered data exhibits significant artifacts in audio-driven animations of reference por￾traits. B. Implementation Details Model Architecture. Our method builds on Arc2Face [55], which employs a fine-tuned UNet and an identity encoder derived fro… view at source ↗
Figure 10
Figure 10. Figure 10: Example synthetic trajectories from our dataset used for spatial fine-tuning. Each row shows a subsampled version of a smooth [PITH_FULL_IMAGE:figures/full_fig_p014_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Visual comparison with 3D-aware methods [ [PITH_FULL_IMAGE:figures/full_fig_p014_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Novel animations generated by our model, conditioned on the input identity and a driving audio. [PITH_FULL_IMAGE:figures/full_fig_p015_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Novel animations generated by our model, conditioned on the input identity and a driving audio (cont.). [PITH_FULL_IMAGE:figures/full_fig_p016_13.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

87 extracted references · 28 linked inside Pith

  1. [4]

    Deep audio-visual speech recognition.IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12):8717–8727, 2018

    Triantafyllos Afouras, Joon Son Chung, Andrew Senior, Oriol Vinyals, and Andrew Zisserman. Deep audio-visual speech recognition.IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12):8717–8727, 2018. 6

    2018

  2. [5]

    Lrs3-ted: a large-scale dataset for visual speech recog- nition.arXiv preprint arXiv:1809.00496, 2018

    Triantafyllos Afouras, Joon Son Chung, and Andrew Zisser- man. Lrs3-ted: a large-scale dataset for visual speech recog- nition.arXiv preprint arXiv:1809.00496, 2018. 6

    Pith/arXiv arXiv 2018

  3. [6]

    Rignerf: Fully controllable neu- ral 3d portraits

    ShahRukh Athar, Zexiang Xu, Kalyan Sunkavalli, Eli Shechtman, and Zhixin Shu. Rignerf: Fully controllable neu- ral 3d portraits. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 3

    2022

  4. [7]

    Id-to-3d: Expressive id-guided 3d heads via score distillation sampling.Advances in Neural Informa- tion Processing Systems, 2024

    Francesca Babiloni, Alexandros Lattas, Jiankang Deng, and Stefanos Zafeiriou. Id-to-3d: Expressive id-guided 3d heads via score distillation sampling.Advances in Neural Informa- tion Processing Systems, 2024. 3

    2024

  5. [8]

    Wav2vec 2.0: A framework for self- supervised learning of speech representations.Advances in neural information processing systems, 2020

    Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. Wav2vec 2.0: A framework for self- supervised learning of speech representations.Advances in neural information processing systems, 2020. 4, 13

    2020

  6. [9]

    Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023. 2

    Pith/arXiv arXiv 2023

  7. [10]

    Align your latents: High-resolution video synthesis with la- tent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dock- horn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with la- tent diffusion models. InIEEE Conference on Computer Vi- sion and Pattern Recognition (CVPR), 2023. 2, 4

    2023

  8. [11]

    Masactrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing

    Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xi- aohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing. InProceedings of the IEEE/CVF International Con- ference on Computer Vision (ICCV), 2023. 5

    2023

  9. [12]

    X-dyna: Ex- pressive dynamic human image animation

    Di Chang, Hongyi Xu, You Xie, Yipeng Gao, Zhengfei Kuang, Shengqu Cai, Chenxu Zhang, Guoxian Song, Chao Wang, Yichun Shi, Zeyuan Chen, Shijie Zhou, Linjie Luo, Gordon Wetzstein, and Mohammad Soleymani. X-dyna: Ex- pressive dynamic human image animation. InProceedings of the Computer Vision and Pattern Recognition Conference (CVPR), 2025. 3

    2025

  10. [13]

    Echomimic: Lifelike audio-driven portrait animations through editable landmark conditions

    Zhiyuan Chen, Jiajiong Cao, Zhiquan Chen, Yuming Li, and Chenguang Ma. Echomimic: Lifelike audio-driven portrait animations through editable landmark conditions. InPro- ceedings of the AAAI Conference on Artificial Intelligence,

  11. [14]

    Hallo2: Long-duration and high-resolution audio-driven portrait im- age animation.arXiv preprint arXiv:2410.07718, 2024

    Jiahao Cui, Hui Li, Yao Yao, Hao Zhu, Hanlin Shang, Kaihui Cheng, Hang Zhou, Siyu Zhu, and Jingdong Wang. Hallo2: Long-duration and high-resolution audio-driven portrait im- age animation.arXiv preprint arXiv:2410.07718, 2024

    Pith/arXiv arXiv 2024

  12. [15]

    Hallo3: Highly dynamic and realistic portrait image animation with video diffusion transformer

    Jiahao Cui, Hui Li, Yun Zhan, Hanlin Shang, Kaihui Cheng, Yuqi Ma, Shan Mu, Hang Zhou, Jingdong Wang, and Siyu Zhu. Hallo3: Highly dynamic and realistic portrait image animation with video diffusion transformer. InProceedings of the Computer Vision and Pattern Recognition Conference (CVPR), 2025. 3, 6, 7, 8

    2025

  13. [16]

    Arcface: Additive angular margin loss for deep face recognition

    Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 4690–4699, 2019. 4

    2019

  14. [17]

    Portrait4d-v2: Pseudo multi-view data creates better 4d head synthesizer

    Yu Deng, Duomin Wang, and Baoyuan Wang. Portrait4d-v2: Pseudo multi-view data creates better 4d head synthesizer. In European Conference on Computer Vision, 2024. 4, 8, 13, 14

    2024

  15. [18]

    Black, and Timo Bolkart

    Yao Feng, Haiwen Feng, Michael J. Black, and Timo Bolkart. Learning an animatable detailed 3D face model from in-the-wild images.ACM Transactions on Graphics, (Proc. SIGGRAPH), 2021. 6

    2021

  16. [19]

    Visual speech-aware perceptual 3d fa- cial expression reconstruction from videos.arXiv preprint arXiv:2207.11094, 2022

    Panagiotis P Filntisis, George Retsinas, Foivos Paraperas- Papantoniou, Athanasios Katsamanis, Anastasios Roussos, and Petros Maragos. Visual speech-aware perceptual 3d fa- cial expression reconstruction from videos.arXiv preprint arXiv:2207.11094, 2022. 6

    Pith/arXiv arXiv 2022

  17. [20]

    Dynamic neural radiance fields for monocular 4d facial avatar reconstruction

    Guy Gafni, Justus Thies, Michael Zollhofer, and Matthias Niessner. Dynamic neural radiance fields for monocular 4d facial avatar reconstruction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021. 3

    2021

  18. [21]

    Srinivasan, Jonathan T

    Ruiqi Gao*, Aleksander Holynski*, Philipp Henzler, Arthur Brussee, Ricardo Martin-Brualla, Pratul P. Srinivasan, Jonathan T. Barron, and Ben Poole*. Cat3d: Create any- thing in 3d with multi-view diffusion models.Advances in Neural Information Processing Systems, 2024. 2

    2024

  19. [22]

    Animateme: 4d facial expressions via diffusion 9 models

    Dimitrios Gerogiannis, Foivos Paraperas Papantoniou, Rolandos Alexandros Potamias, Alexandros Lattas, Stylianos Moschoglou, Stylianos Ploumpis, and Stefanos Zafeiriou. Animateme: 4d facial expressions via diffusion 9 models. InEuropean Conference on Computer Vision, pages 270–287. Springer, 2024. 3

    2024

  20. [23]

    Arc2avatar: Generating expressive 3d avatars from a single image via id guidance

    Dimitrios Gerogiannis, Foivos Paraperas Papantoniou, Rolandos Alexandros Potamias, Alexandros Lattas, and Ste- fanos Zafeiriou. Arc2avatar: Generating expressive 3d avatars from a single image via id guidance. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025. 2, 3

    2025

  21. [24]

    Generative adversarial nets.Advances in neural information processing systems, 2014

    Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets.Advances in neural information processing systems, 2014. 2

    2014

  22. [25]

    Neural head avatars from monocular rgb videos

    Philip-William Grassal, Malte Prinzler, Titus Leistner, Carsten Rother, Matthias Nießner, and Justus Thies. Neural head avatars from monocular rgb videos. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 3

    2022

  23. [26]

    Sparsectrl: Adding sparse con- trols to text-to-video diffusion models.arXiv preprint arXiv:2311.16933, 2023

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Sparsectrl: Adding sparse con- trols to text-to-video diffusion models.arXiv preprint arXiv:2311.16933, 2023. 3

    Pith/arXiv arXiv 2023

  24. [27]

    Animatediff: Animate your personalized text-to- image diffusion models without specific tuning.Interna- tional Conference on Learning Representations, 2024

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to- image diffusion models without specific tuning.Interna- tional Conference on Learning Representations, 2024. 2, 4

    2024

  25. [28]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium.Advances in neural information processing systems,

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium.Advances in neural information processing systems,

  26. [29]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. InAdvances in Neural Informa- tion Processing Systems, 2020. 2

    2020

  27. [30]

    Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2021

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2021. 5

    Pith/arXiv arXiv 2021

  28. [31]

    Animate anyone: Consistent and controllable image-to-video synthesis for character animation.arXiv preprint arXiv:2311.17117, 2023

    Li Hu, Xin Gao, Peng Zhang, Ke Sun, Bang Zhang, and Liefeng Bo. Animate anyone: Consistent and controllable image-to-video synthesis for character animation.arXiv preprint arXiv:2311.17117, 2023. 3

    Pith/arXiv arXiv 2023

  29. [32]

    Self forcing: Bridging the train- test gap in autoregressive video diffusion.arXiv preprint arXiv:2506.08009, 2025

    Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train- test gap in autoregressive video diffusion.arXiv preprint arXiv:2506.08009, 2025. 5

    Pith/arXiv arXiv 2025

  30. [33]

    Loopy: Taming audio-driven portrait avatar with long-term motion dependency.arXiv preprint arXiv:2409.02634, 2024

    Jianwen Jiang, Chao Liang, Jiaqi Yang, Gaojie Lin, Tianyun Zhong, and Yanbo Zheng. Loopy: Taming audio-driven portrait avatar with long-term motion dependency.arXiv preprint arXiv:2409.02634, 2024. 3

    Pith/arXiv arXiv 2024

  31. [34]

    A style-based generator architecture for generative adversarial networks

    Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. InProceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, 2019. 13

    2019

  32. [35]

    3d gaussian splatting for real-time radiance field rendering.ACM Trans

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM Trans. Graph., 2023. 2, 3

    2023

  33. [36]

    Float: Generative motion latent flow matching for audio-driven talking portrait

    Taekyung Ki, Dongchan Min, and Gyeongsu Chae. Float: Generative motion latent flow matching for audio-driven talking portrait. InProceedings of the IEEE/CVF Interna- tional Conference on Computer Vision (ICCV), 2025. 3, 6, 7

    2025

  34. [37]

    Nersemble: Multi-view ra- diance field reconstruction of human heads.ACM Trans

    Tobias Kirschstein, Shenhan Qian, Simon Giebenhain, Tim Walter, and Matthias Nießner. Nersemble: Multi-view ra- diance field reconstruction of human heads.ACM Trans. Graph., 2023. 7, 8, 14

    2023

  35. [38]

    Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024. 3

    Pith/arXiv arXiv 2024

  36. [39]

    Spherehead: stable 3d full-head synthesis with spherical tri-plane representation

    Heyuan Li, Ce Chen, Tianhao Shi, Yuda Qiu, Sizhe An, Guanying Chen, and Xiaoguang Han. Spherehead: stable 3d full-head synthesis with spherical tri-plane representation. In European Conference on Computer Vision, 2024. 6, 13

    2024

  37. [40]

    Tianye Li, Timo Bolkart, Michael. J. Black, Hao Li, and Javier Romero. Learning a model of facial shape and ex- pression from 4D scans.ACM Transactions on Graphics, (Proc. SIGGRAPH Asia), 36(6):194:1–194:17, 2017. 6

    2017

  38. [41]

    One-shot high-fidelity talking- head synthesis with deformable neural radiance field

    Weichuang Li, Longhao Zhang, Dong Wang, Bin Zhao, Zhi- gang Wang, Mulin Chen, Bang Zhang, Zhongjian Wang, Liefeng Bo, and Xuelong Li. One-shot high-fidelity talking- head synthesis with deformable neural radiance field. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 3

    2023

  39. [42]

    Generalizable one-shot 3d neural head avatar.Advances in Neural Information Processing Systems,

    Xueting Li, Shalini De Mello, Sifei Liu, Koki Nagano, Umar Iqbal, and Jan Kautz. Generalizable one-shot 3d neural head avatar.Advances in Neural Information Processing Systems,

  40. [43]

    Im-portrait: Learning 3d-aware video diffusion for photorealistic talking heads from monoc- ular videos

    Yuan Li, Ziqian Bai, Feitong Tan, Zhaopeng Cui, Sean Fanello, and Yinda Zhang. Im-portrait: Learning 3d-aware video diffusion for photorealistic talking heads from monoc- ular videos. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025. 4

    2025

  41. [44]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. InEu- ropean conference on computer vision, 2024. 13

    2024

  42. [45]

    Anitalker: animate vivid and di- verse talking faces through identity-decoupled facial motion encoding

    Tao Liu, Feilong Chen, Shuai Fan, Chenpeng Du, Qi Chen, Xie Chen, and Kai Yu. Anitalker: animate vivid and di- verse talking faces through identity-decoupled facial motion encoding. InProceedings of the 32nd ACM International Conference on Multimedia, 2024. 3, 6, 7

    2024

  43. [46]

    Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017. 13

    Pith/arXiv arXiv 2017

  44. [47]

    Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps.Advances in Neural Information Processing Systems, 35:5775–5787,

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps.Advances in Neural Information Processing Systems, 35:5775–5787,

  45. [48]

    Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models.arXiv preprint arXiv:2211.01095, 2022

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongx- uan Li, and Jun Zhu. Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models.arXiv preprint arXiv:2211.01095, 2022. 13 10

    Pith/arXiv arXiv 2022

  46. [49]

    Visual Speech Recognition for Multiple Languages in the Wild.Na- ture Machine Intelligence, 4:930–939, 2022

    Pingchuan Ma, Stavros Petridis, and Maja Pantic. Visual Speech Recognition for Multiple Languages in the Wild.Na- ture Machine Intelligence, 4:930–939, 2022. 6

    2022

  47. [50]

    Follow your pose: Pose-guided text-to-video generation using pose-free videos

    Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Ying Shan, Xiu Li, and Qifeng Chen. Follow your pose: Pose-guided text-to-video generation using pose-free videos. arXiv preprint arXiv:2304.01186, 2023. 3

    Pith/arXiv arXiv 2023

  48. [51]

    Otavatar: One-shot talking face avatar with control- lable tri-plane rendering

    Zhiyuan Ma, Xiangyu Zhu, Guo-Jun Qi, Zhen Lei, and Lei Zhang. Otavatar: One-shot talking face avatar with control- lable tri-plane rendering. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 3

    2023

  49. [52]

    Isambard-ai: a leadership-class supercomputer opti- mised specifically for artificial intelligence

    Simon McIntosh-Smith, Sadaf Alam, and Christopher Woods. Isambard-ai: a leadership-class supercomputer opti- mised specifically for artificial intelligence. InProceedings of the Cray User Group, pages 44–54. 2024. 9

    2024

  50. [53]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis.Communications of the ACM, 2021

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis.Communications of the ACM, 2021. 2, 3

    2021

  51. [54]

    Elucidating the exposure bias in diffusion models.arXiv preprint arXiv:2308.15321, 2023

    Mang Ning, Mingxiao Li, Jianlin Su, Albert Ali Salah, and Itir Onal Ertugrul. Elucidating the exposure bias in diffusion models.arXiv preprint arXiv:2308.15321, 2023. 5

    Pith/arXiv arXiv 2023

  52. [55]

    Arc2face: A foundation model for id-consistent human faces

    Foivos Paraperas Papantoniou, Alexandros Lattas, Stylianos Moschoglou, Jiankang Deng, Bernhard Kainz, and Stefanos Zafeiriou. Arc2face: A foundation model for id-consistent human faces. InProceedings of the European Conference on Computer Vision (ECCV), 2024. 4, 13

    2024

  53. [56]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF Interna- tional Conference on Computer Vision (ICCV), 2023. 3

    2023

  54. [57]

    Movie gen: A cast of media foundation models.arXiv preprint arXiv:2410.13720,

    Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih- Yao Ma, Ching-Yao Chuang, et al. Movie gen: A cast of media foundation models.arXiv preprint arXiv:2410.13720,

    Pith/arXiv arXiv

  55. [58]

    A lip sync expert is all you need for speech to lip generation in the wild

    KR Prajwal, Rudrabha Mukhopadhyay, Vinay P Nambood- iri, and CV Jawahar. A lip sync expert is all you need for speech to lip generation in the wild. InProceedings of the 28th ACM international conference on multimedia, 2020. 2, 3, 6

    2020

  56. [59]

    Gaus- sianavatars: Photorealistic head avatars with rigged 3d gaus- sians

    Shenhan Qian, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Simon Giebenhain, and Matthias Nießner. Gaus- sianavatars: Photorealistic head avatars with rigged 3d gaus- sians. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), 2024. 2, 3

    2024

  57. [60]

    Generalization in generation: A closer look at exposure bias.arXiv preprint arXiv:1910.00292, 2019

    Florian Schmidt. Generalization in generation: A closer look at exposure bias.arXiv preprint arXiv:1910.00292, 2019. 5

    Pith/arXiv arXiv 1910

  58. [61]

    Wav2vec: Unsupervised pre-training for speech recognition.arXiv preprint arXiv:1904.05862, 2019

    Steffen Schneider, Alexei Baevski, Ronan Collobert, and Michael Auli. Wav2vec: Unsupervised pre-training for speech recognition.arXiv preprint arXiv:1904.05862, 2019. 4, 13

    Pith/arXiv arXiv 1904

  59. [62]

    Difftalk: Crafting diffusion models for generalized audio-driven portraits animation

    Shuai Shen, Wenliang Zhao, Zibin Meng, Wanhua Li, Zheng Zhu, Jie Zhou, and Jiwen Lu. Difftalk: Crafting diffusion models for generalized audio-driven portraits animation. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), 2023. 3

    2023

  60. [63]

    Zero123++: a single image to consistent multi-view dif- fusion base model.arXiv preprint arXiv:2310.15110, 2023

    Ruoxi Shi, Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, and Hao Su. Zero123++: a single image to consistent multi-view dif- fusion base model.arXiv preprint arXiv:2310.15110, 2023. 2

    Pith/arXiv arXiv 2023

  61. [64]

    Make-a-video: Text-to-video generation without text-video data.arXiv preprint arXiv:2209.14792,

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data.arXiv preprint arXiv:2209.14792,

    Pith/arXiv arXiv

  62. [65]

    Felix Taubner, Ruihang Zhang, Mathieu Tuli, and David B. Lindell. Cap4d: Creating animatable 4d portrait avatars with morphable multi-view diffusion models. InProceedings of the Computer Vision and Pattern Recognition Conference (CVPR), 2025. 3

    2025

  63. [66]

    Emo: Emote portrait alive - generating expressive portrait videos with audio2video diffusion model under weak conditions

    Linrui Tian, Qi Wang, Bang Zhang, and Liefeng Bo. Emo: Emote portrait alive - generating expressive portrait videos with audio2video diffusion model under weak conditions. In European Conference on Computer Vision, 2024. 2, 3

    2024

  64. [67]

    To- wards accurate generative models of video: A new metric & challenges.arXiv preprint arXiv:1812.01717, 2018

    Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. To- wards accurate generative models of video: A new metric & challenges.arXiv preprint arXiv:1812.01717, 2018. 6

    Pith/arXiv arXiv 2018

  65. [68]

    Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianx- iao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jin- gren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng, Shiwei Zhang, Siyang Sun, Tao Fan...

    Pith/arXiv arXiv 2025

  66. [69]

    V-express: Conditional dropout for progres- sive training of portrait video generation.arXiv preprint arXiv:2406.02511, 2024

    Cong Wang, Kuan Tian, Jun Zhang, Yonghang Guan, Feng Luo, Fei Shen, Zhiwei Jiang, Qing Gu, Xiao Han, and Wei Yang. V-express: Conditional dropout for progres- sive training of portrait video generation.arXiv preprint arXiv:2406.02511, 2024. 3, 6, 7

    Pith/arXiv arXiv 2024

  67. [70]

    Disco: Disentangled control for referring human dance generation in real world.arXiv preprint arXiv:2307.00040, 2023

    Tan Wang, Linjie Li, Kevin Lin, Chung-Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, and Li- juan Wang. Disco: Disentangled control for referring human dance generation in real world.arXiv preprint arXiv:2307.00040, 2023. 3

    Pith/arXiv arXiv 2023

  68. [71]

    One-shot free-view neural talking-head synthesis for video conferenc- ing

    Ting-Chun Wang, Arun Mallya, and Ming-Yu Liu. One-shot free-view neural talking-head synthesis for video conferenc- ing. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), 2021. 6, 7, 8

    2021

  69. [72]

    Videocomposer: Compositional video synthesis 11 with motion controllability.Advances in Neural Information Processing Systems, 36:7594–7611, 2023

    Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Ji- uniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jin- gren Zhou. Videocomposer: Compositional video synthesis 11 with motion controllability.Advances in Neural Information Processing Systems, 36:7594–7611, 2023. 3

    2023

  70. [73]

    Aniportrait: Audio-driven synthesis of photorealistic portrait animation

    Huawei Wei, Zejun Yang, and Zhisheng Wang. Aniportrait: Audio-driven synthesis of photorealistic portrait animation. arXiv preprint arXiv:2403.17694, 2024. 3, 4, 6, 7

    Pith/arXiv arXiv 2024

  71. [74]

    Vfhq: A high-quality dataset and bench- mark for video face super-resolution

    Liangbin Xie, Xintao Wang, Honglun Zhang, Chao Dong, and Ying Shan. Vfhq: A high-quality dataset and bench- mark for video face super-resolution. InThe IEEE Con- ference on Computer Vision and Pattern Recognition Work- shops (CVPRW), 2022. 6, 13

    2022

  72. [75]

    Hallo: Hierarchical audio-driven visual synthesis for portrait image animation.arXiv preprint arXiv:2406.08801, 2024

    Mingwang Xu, Hui Li, Qingkun Su, Hanlin Shang, Liwei Zhang, Ce Liu, Jingdong Wang, Yao Yao, and Siyu Zhu. Hallo: Hierarchical audio-driven visual synthesis for portrait image animation.arXiv preprint arXiv:2406.08801, 2024. 2, 3, 4, 6

    Pith/arXiv arXiv 2024

  73. [76]

    Magicanimate: Temporally consistent human im- age animation using diffusion model

    Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. Magicanimate: Temporally consistent human im- age animation using diffusion model. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 3

    2024

  74. [77]

    Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiao- han Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024. 3

    Pith/arXiv arXiv 2024

  75. [78]

    Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models.arXiv preprint arXiv:2308.06721,

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models.arXiv preprint arXiv:2308.06721,

    Pith/arXiv arXiv

  76. [79]

    Real3d-portrait: One-shot realistic 3d talking portrait synthesis.ICLR, 2024

    Zhenhui Ye, Tianyun Zhong, Yi Ren, Jiaqi Yang, Weichuang Li, Jiangwei Huang, Ziyue Jiang, Jinzheng He, Rongjie Huang, Jinglin Liu, Chen Zhang, Xiang Yin, Zejun Ma, and Zhou Zhao. Real3d-portrait: One-shot realistic 3d talking portrait synthesis.ICLR, 2024. 2, 3, 4, 7, 8, 13, 14

    2024

  77. [80]

    Sadtalker: Learning realistic 3d motion coefficients for stylized audio- driven single image talking face animation

    Wenxuan Zhang, Xiaodong Cun, Xuan Wang, Yong Zhang, Xi Shen, Yu Guo, Ying Shan, and Fei Wang. Sadtalker: Learning realistic 3d motion coefficients for stylized audio- driven single image talking face animation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), 2023. 2, 3

    2023

  78. [81]

    Mimicmo- tion: High-quality human motion video generation with confidence-aware pose guidance

    Yuang Zhang, Jiaxi Gu, Li-Wen Wang, Han Wang, Junqi Cheng, Yuefeng Zhu, and Fangyuan Zou. Mimicmo- tion: High-quality human motion video generation with confidence-aware pose guidance. InInternational Confer- ence on Machine Learning, 2025. 3

    2025

  79. [82]

    Flow-guided one-shot talking face generation with a high-resolution audio-visual dataset

    Zhimeng Zhang, Lincheng Li, Yu Ding, and Changjie Fan. Flow-guided one-shot talking face generation with a high-resolution audio-visual dataset. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021. 6, 13

    2021

  80. [83]

    Ilsh: The imperial light- stage head dataset for human head view synthesis

    Jiali Zheng, Youngkyoon Jang, Athanasios Papaioan- nou, Christos Kampouris, Rolandos Alexandros Potamias, Foivos Paraperas Papantoniou, Efstathios Galanakis, Ale ˇs Leonardis, and Stefanos Zafeiriou. Ilsh: The imperial light- stage head dataset for human head view synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages...

    2023

Showing first 80 references.