Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

SyncTalk++: High-Fidelity and Efficient Synchronized Talking Heads Synthesis Using Gaussian Splatting

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SyncTalk++ delivers synchronized talking heads at 101 FPS using Gaussian splatting.

desk verdict Solid incremental extension of SyncTalk, but the abstract's superiority claim is contradicted by the paper's own Table I. read the letter →

arxiv 2506.14742 v1 pith:HYN5362Z submitted 2025-06-17 cs.CV

Ziqiao Peng , Wentao Hu , Junyuan Ma , Xiangyu Zhu , Xiaomei Zhang , Hao Zhao , Hui Tian , Jun He
show 2 more authors
Hongyan Liu Zhaoxin Fan
This is my paper
classification cs.CV
keywords talkingheadsynthesisaudio-drivenanimation3DGaussiansplattinglipsynchronizationfacialblendshapesposestabilizationout-of-distributionaudioreal-timerendering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the main obstacle to realistic speech-driven talking heads is synchronization across identity, lips, expressions, and head pose, and that a Gaussian-splatting pipeline (a point-based renderer that projects 3D Gaussian blobs to pixels) built around three sync modules can overcome it. It claims that lip movements stay aligned with speech because the audio features come from an encoder trained specifically for audio–lip synchronization rather than speech recognition, and that expressions stay natural because they are controlled by 3D facial blendshape coefficients. It further claims that head jitter is reduced by a stabilizer that weights stable facial keypoints, and that out-of-distribution audio is handled by an expression generator and a torso restorer. If correct, the system produces high-fidelity, real-time (101 FPS) talking heads from a few minutes of training video, ready for live streaming, virtual assistants, and film production.

What carries the argument

The load-bearing mechanism is the three-way split of the driving signal inside the Dynamic Portrait Renderer: a deformation MLP $F_{\mathrm{deform}}(f_\mu, f_l, f_e, R, T)$ predicts offsets for every Gaussian attribute (position, rotation, scale, opacity, spherical harmonics) from concatenated triplane geometry features $f_\mu$, lip features $f_l$, expression features $f_e$, and head pose $(R,T)$. Supporting it, the Face-Sync Controller supplies $f_l$ from an audio–visual encoder trained with a lip-sync discriminator and $f_e$ from 52 blendshape coefficients, while the Head-Sync Stabilizer supplies $(R,T)$ through optical-flow tracking with semantic weighting and bundle adjustment. This separation is what lets each synchronization factor be optimized independently before being recombined in the Gaussian deformation step.

What would settle it

Take a subject not present in any training video, drive the trained renderer with held-out in-the-wild audio, and compare LSE-C and LMD against the same renderer fed with features from an ASR model such as HuBERT; if the sync encoder's advantage disappears on unseen identities or audio, the 2D-to-3D transfer claim is falsified.

Watch

Extended reading notes

Core claim

The central claim is that synchronization, not rendering capacity, is the binding constraint in talking-head synthesis, and that separately controlling lips, expressions, pose, and identity within a 3D Gaussian Splatting renderer yields state-of-the-art fidelity and real-time speed. The Face-Sync Controller extracts audio features with a network trained to reconstruct 2D lip motion and combines them with 52 semantically meaningful blendshape coefficients, so lip movements and facial expressions are driven by different, disentangled signals. The Head-Sync Stabilizer refines initial pose estimates with optical-flow tracking, semantic weighting of unstable eye and eyebrow keypoints, and bundle adjustment to keep the head stable and attached to the torso. A Dynamic Portrait Renderer builds canonical 3D Gaussians on triplane features and deforms them with an MLP conditioned on lip features, expression features, and head pose; a Portrait-Sync Generator re-fuses the rendered face with the original high-resolution frame to preserve hair and texture. For out-of-distribution audio, a VQ-VAE (a vector-quantized autoencoder with a discrete codebook) over EmoTalk blendshape coefficients adapts expressions to the target identity, and a U-Net torso inpainter closes jaw-mismatch gaps.

Load-bearing premise

The retained audio features come from a network trained to reconstruct 2D face frames, and the whole lip-sync improvement depends on those features staying informative when they are fed into the MLP that deforms 3D Gaussians for identities and audio the network was not trained on.

Editorial extensions

If this is right

  • At 101 FPS on a single RTX 4090 and 1.5 hours of training per subject, the method is fast enough for live-streaming and video-conferencing use rather than offline rendering.
  • Because lip features come from an audio–visual sync encoder rather than an ASR model, lip movements should stay aligned for in-the-wild and TTS audio without retraining the renderer.
  • The Expression Generator and Torso Restorer let the same per-subject renderer accept out-of-distribution audio while keeping expressions speech-matched and head–torso seams closed.
  • The Portrait-Sync Generator restores hair and high-frequency detail by compositing the rendered face with the original frame, so output can remain high-resolution while the 3D part runs at lower cost.
  • User-study ratings across lip-sync, expression-sync, pose-sync, image quality, and realness all favor SyncTalk++ over the compared methods, supporting the claim that the synchronization-first design translates to perceived realism.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the 2D-trained audio–visual encoder transfers reliably to 3D deformation, the same renderer could be driven by any sufficiently sync-aligned audio representation, making the lip-sync module swappable without retraining the Gaussian fields.
  • The explicit separation of lip and expression features suggests an editing interface: a user could retarget expressions by editing blendshape coefficients while leaving lip sync untouched, which the paper does not demonstrate but its architecture permits.
  • The VQ-VAE codebook approach to OOD blendshapes implies that generalization is bounded by codebook coverage; testing on emotional or non-English audio far from the training distribution would show whether the discrete bottleneck helps or over-quantizes expressive speech.
  • Because the torso restorer is trained on random mask expansions, it may also serve as a general head–torso compositing repair module for other avatar renderers, not only Gaussian-splatting pipelines.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents SyncTalk++, a per-subject speech-driven talking-head synthesis system built on 3D Gaussian Splatting. It extends the authors' prior SyncTalk work by replacing NeRF with 3DGS, introducing a Face-Sync Controller that uses an audio-visual encoder and 3D blendshape coefficients, a Head-Sync Stabilizer with semantic weighting, a Dynamic Portrait Renderer with a triplane Gaussian representation, and two OOD-robustness modules: an OOD Audio Expression Generator based on a VQ-VAE and EmoTalk, and an OOD Audio Torso Restorer based on inpainting. The paper claims state-of-the-art synchronization, realism, and efficiency, with up to 101 FPS rendering, and supports these claims with reconstruction metrics, lip-sync metrics, a user study, and ablations.

Significance. If the claims were fully supported, this would be a practically important contribution: a per-subject talking-head system that renders at 101 FPS while improving lip synchronization, expression control, pose stability, and visual quality over a wide set of baselines. The manuscript has clear strengths: it compares against many 2D and 3D baselines, includes a user study with 42 participants and a reported Cronbach's alpha, provides ablations for the principal components, and gives a detailed architectural overview. However, the central claim of superiority in synchronization and realism is directly weakened by the paper's own numbers in Table I, which show SyncTalk++ falling behind its predecessor SyncTalk on LSE-C, NIQE, and BRISQUE. Because these metrics are the ones the paper itself uses for synchronization and no-reference quality, the headline claim is not currently established. The OOD robustness contribution is also evaluated only on two audio samples and without quantitative validation of the expression generator. These issues are fixable but require additional experiments and careful claim revision.

major comments (4)
  1. [Section IV.B, Table I; Abstract; Contributions] The central claim that SyncTalk++ 'outperforms state-of-the-art methods in synchronization and realism' is contradicted by the paper's own baseline comparison in Table I. SyncTalk++ has LSE-C 7.8298 versus SyncTalk's 8.1331 (higher is better, and Section IV.B defines LSE-C as the metric for synchronization between lip movements and audio), and it is worse than SyncTalk on NIQE (13.7215 vs. 13.1333) and BRISQUE (39.3769 vs. 33.2954), both lower-is-better quality metrics. The sentence in Section IV.B that 'our image quality is superior to other methods in all aspects' is therefore not supported by the table. The authors should either temper the abstract and contribution claims, or provide a detailed analysis of why these three metrics regress relative to the direct predecessor, and ideally report significance tests for the differences.
  2. [Tables I, III, VI, and VII] All quantitative tables report point estimates only. There are no standard deviations, no confidence intervals, and no significance tests. This matters particularly for the synchronization metrics: the LSE-C gap between SyncTalk++ and SyncTalk in Table I is 0.30 on a metric that is known to be noisy, and Table III uses only two OOD audio samples, so the reported superiority in lip synchronization may not be stable. The ablation in Table VI is reported for a single subject ('May'), and Table VII likewise appears to be a single-condition comparison without variance. Please report per-subject or per-audio-clip results, error bars, and appropriate paired significance tests for the main claims.
  3. [Section III.E, Section IV.B, Table III] OOD audio robustness is one of the two listed contributions, but the evidence is thin. Table III uses only two audio samples, Figure 11 is qualitative, and there is no quantitative evaluation of the OOD Audio Expression Generator itself: no comparison against directly using EmoTalk blendshape coefficients without the VQ-VAE, no expression-accuracy or emotion-consistency metric, and no ablation of the codebook size or the quantized latent space. The Torso Restorer has only Table VII without comparison to a simpler blending or inpainting baseline. The claim that the system 'enhances robustness to out-of-distribution audio' currently goes beyond the reported experiments. A broader OOD evaluation (more speakers, TTS systems, languages, and per-audio variance) is needed to support this contribution.
  4. [Section III.B and Eq. (19)] The lip-sync pipeline relies on a 2D-to-3D transfer that is not directly validated. The Audio-Visual Encoder is trained to reconstruct 2D face frames, and then only Conv(A) is retained and fed into the 3D deformation MLP in Eq. (19). The paper does not test whether the improvement over, say, HuBERT comes specifically from the audio-visual sync pretraining or from the discarded visual encoder/decoder branch. The ablation in Table VI replaces the encoder with HuBERT but does not ablate the pretraining objective or the visual branch. A concrete test would be comparing Conv(A) with a feature extractor trained with the same architecture but without the sync loss, or evaluating lip-sync on held-out subjects/audio with the current encoder. Without this, the mechanism behind the reported lip-sync gain remains a correctness risk.
minor comments (6)
  1. [Section III.D, Eqs. (22) and (24)] The loss weights lambda_L1, lambda_lpips, and lambda_perceptual are introduced but their values are never given; please report the hyperparameters or state that they follow SyncTalk.
  2. [Section III.E, Eq. (27)] The VQ-VAE loss includes a commitment loss weight beta, but the codebook size N, embedding dimension C, and beta are not specified, and no sensitivity analysis is provided.
  3. [Section I, Reference [33]] The paper repeatedly frames SyncTalk++ as an extension of the authors' own SyncTalk, but the quantitative comparison in Table I shows the new method losing on LSE-C, NIQE, and BRISQUE; this should be discussed explicitly rather than only claimed as an improvement.
  4. [Section IV.B, Table IV] The text says that 'SH, alpha' initialization achieves the best overall performance, but the table also shows that initializing all attributes (s, r, SH, alpha) has the best PSNR and LMD; please clarify which criterion 'best overall' refers to.
  5. [Section V] The phrase 'Restrict the application of deepfake technology' is a suggestion for society, not a technical contribution; consider shortening the ethics section or moving it to a supplementary discussion.
  6. [Abstract] The project URL 'https://ziqiaopeng.github.io/synctalk++' contains two plus signs; please verify that this URL is correct and accessible.

Circularity Check

1 steps flagged · score 4.0 of 10

Lip-sync evaluation is not fully independent: the audio encoder is trained with the same SyncNet that later computes the reported LSE-C; otherwise the derivation chain is self-contained.

  1. fitted input called prediction [Section III.B (Audio-Visual Encoder, Eq. (2)) and Section IV.B (Synchronization Assessment, Tables I/III)]
    "Under the supervision of the lip synchronization discriminator, we pre-train a highly synchronized audio-visual feature extractor related to lip movements... Simultaneously, we sample synchronized and non-synchronized segments using lip movement discriminators and employ the same sync loss as Eq. 2. ... We discard the facial encoder and decoder parts of the network, retaining only the audio convolution component Conv(A). ... we ... introduce Lip Sync Error Confidence (LSE-C), consistent with Wav2Lip [43], to evaluate the synchronization between lip movements and audio."

    The lip-sync discriminator used for supervision is [73] (Chung and Zisserman, 'Out of time'), which is the SyncNet that Wav2Lip [43] uses to compute LSE-C/LSE-D. The paper therefore optimizes Conv(A) to maximize the cosine-similarity confidence of the exact network that later produces the reported LSE-C scores in Table III. The lip-sync 'prediction' is not measured by an independent observer; it is scored by the same function the audio feature extractor was trained to satisfy. Because the deformation MLP and 3DGS rasterizer stand between f_l and the final video, the reduction is not complete, and Table I actually shows SyncTalk++ below SyncTalk on LSE-C; but the headline synchronization metric is partially forced by construction.

full rationale

The paper's central rendering pipeline (triplane canonical Gaussians, deformation MLP, losses in Eqs. 21-24) is self-contained and benchmarked against external baselines; no fitted constant is renamed as a prediction. The user study, PSNR/LPIPS/FID, and most no-reference metrics are external to the method's training objectives. The main circular step is the lip-sync metric: Section III.B trains the audio-visual encoder with a lip-sync discriminator [73], and Section IV.B reports LSE-C 'consistent with Wav2Lip [43]', whose lip-sync expert is that same SyncNet. Thus Table III's lip-sync advantage is partly a measure of how well the encoder satisfies its own training supervisor rather than an independent test of synchronization. This is a partial reduction, not a total one, because the deformation network and rasterizer intervene and because Table I shows SyncTalk++ losing to SyncTalk on LSE-C. The OOD Expression Generator relies on the authors' own EmoTalk [74] and the comparison omits the authors' SyncTalk in Table III, but EmoTalk is externally published and the OOD component is not the paper's only evidence; these are self-citation and evidence-selection concerns, not circular derivations. The abstract's blanket claim of superiority is also in tension with Table I (worse NIQE/BRISQUE than SyncTalk), which is a consistency issue outside the circularity pass.

Assumptions & free parameters 7 free parameters · 6 assumptions · 2 invented entities

The central claim rests on learned components whose hyperparameters are mostly undisclosed and on several domain assumptions: transfer of a 2D lip-sync encoder to 3D deformation, separability of seven blendshape coefficients from lip motion, reliability of EmoTalk outputs for OOD audio, and the validity of held-out frames of the same video as a test set. The listed free parameters are all hand-chosen or undisclosed and directly affect the reported performance.

free parameters (7)
  • Seven selected blendshape coefficients for eyebrows and eyes = 7 of 52 blendshape coefficients
    Chosen by hand in Section III.B, Eq. 5, to isolate expression control. If the set is not separable from lip and jaw motion, the expression-lip disentanglement degrades.
  • Optical flow keypoint threshold theta = not reported
    Equation 10 selects keypoints whose Laplacian-filtered flow exceeds theta. The threshold is not given and directly affects which points drive head pose estimation.
  • Random mask expansion range delta_ran = 10-30 pixels training, 15 pixels inference
    Section III.F uses this hand-set range to create simulated head-torso gaps for the inpainting network, controlling the defects the network learns to repair.
  • VQ-VAE codebook size N and embedding dimension C = not reported
    Equations 25-26 depend on codebook capacity. No values are given, so the OOD expression generator cannot be re-implemented as specified.
  • Two-stage loss weights lambda_L1, lambda_lpips, lambda_perceptual = not reported
    Equations 22 and 24 use these weights without numeric values, leaving the training balance between reconstruction, perceptual, and LPIPS losses unspecified.
  • VQ-VAE commitment loss weight beta = not reported
    Equation 27 includes beta for the commitment loss, but no value is provided.
  • Nose-tip mask boundary for facial attention masks = horizontal coordinate of nose tip landmark
    Section III.B divides the face into lower (lips) and upper (expressions) regions using this landmark. The boundary is a heuristic that affects feature disentanglement.
assumptions (6)
  • domain assumption The audio-visual encoder trained on LRS2 with a 2D lip-sync discriminator produces features that transfer to 3D Gaussian deformation (Section III.B).
    The paper keeps only Conv(A) and discards the visual encoder and decoder; no external benchmark verifies that these 2D-trained audio features remain informative for 3D lip deformation across identities and audio domains.
  • domain assumption The seven selected blendshape coefficients are semantically meaningful and independent of lip movements (Section III.B, Eq. 5).
    The independence of eyebrow and eye coefficients from lip and jaw motion is asserted rather than measured. If false, the masked attention disentanglement inherits the coupling error.
  • ad hoc to paper EmoTalk, the authors' prior ICCV work, generates blendshape coefficients that are speech-matched for OOD audio (Section III.E).
    The OOD expression generator's inputs come from EmoTalk, a self-cited component. The paper does not independently validate EmoTalk's coefficients on the target identities before using them to train the renderer.
  • domain assumption BFM identity parameters can be held fixed while optimizing focal length, rotation, translation, and expression per frame (Section III.C).
    The Head Motion Tracker assumes a stable identity shape and accurate landmarks. This standard BFM assumption is not verified against ground-truth poses for the test subjects.
  • domain assumption Held-out frames from the same well-edited training video are a valid proxy for generalization (Section IV.A).
    The reconstruction and synchronization tests use a temporal crop of the subject's own video. This does not test cross-subject or unconstrained video generalization.
  • standard math The 3D Gaussian Splatting rendering equations (Eq. 13-15) correctly model the face and camera (Section III.D).
    The rendering pipeline is adopted from Kerbl et al. and other prior work without re-derivation; numerical stability and approximation error are assumed acceptable.
invented entities (2)
  • Discrete codebook in OOD Audio Expression Generator
    purpose: To map out-of-distribution blendshape coefficients produced by EmoTalk into a distribution the target 3DGS renderer can decode without artifacts.
    The only support is the in-paper OOD experiments and ablations; there is no external dataset or independent evaluation establishing the codebook's generalization across identities.
  • Semantic weighting mask for eye and eyebrow keypoints
    purpose: To reduce head pose jitter by down-weighting keypoints in high-variance facial regions during tracking.
    The mask is a hand-designed heuristic. The paper provides an in-paper ablation but no external validation that down-weighted keypoints are reliably unstable across subjects and videos.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SyncTalk++: High-Fidelity and Efficient Synchronized Talking Heads Synthesis Using Gaussian Splatting." pith.science (2026). https://pith.science/paper/HYN5362Z

@misc{pith2026250614742,
  author       = {Pith},
  title        = {Pith review of: SyncTalk++: High-Fidelity and Efficient Synchronized Talking Heads Synthesis Using Gaussian Splatting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HYN5362Z}},
  note         = {Machine review of arXiv:2506.14742}
}
read the original abstract

Achieving high synchronization in the synthesis of realistic, speech-driven talking head videos presents a significant challenge. A lifelike talking head requires synchronized coordination of subject identity, lip movements, facial expressions, and head poses. The absence of these synchronizations is a fundamental flaw, leading to unrealistic results. To address the critical issue of synchronization, identified as the ''devil'' in creating realistic talking heads, we introduce SyncTalk++, which features a Dynamic Portrait Renderer with Gaussian Splatting to ensure consistent subject identity preservation and a Face-Sync Controller that aligns lip movements with speech while innovatively using a 3D facial blendshape model to reconstruct accurate facial expressions. To ensure natural head movements, we propose a Head-Sync Stabilizer, which optimizes head poses for greater stability. Additionally, SyncTalk++ enhances robustness to out-of-distribution (OOD) audio by incorporating an Expression Generator and a Torso Restorer, which generate speech-matched facial expressions and seamless torso regions. Our approach maintains consistency and continuity in visual details across frames and significantly improves rendering speed and quality, achieving up to 101 frames per second. Extensive experiments and user studies demonstrate that SyncTalk++ outperforms state-of-the-art methods in synchronization and realism. We recommend watching the supplementary video: https://ziqiaopeng.github.io/synctalk++.

Figures

Figures reproduced from arXiv: 2506.14742 by the authors.

Figure 1
Figure 1. The proposed SyncTalk++ uses 3D Gaussian Splatting for rendering. It can generate synchronized lip movements, facial expressions, and more stable [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of SyncTalk++. Given a cropped reference video of a talking head and the corresponding speech, SyncTalk++ can extract the Lip Feature fl , Expression Feature fe, and Head Pose (R, T) through two synchronization modules (a) and (b). Then, Gaussian Splatting is used to model and deform the head, producing a talking head video. The OOD Audio Expression Generator and Torso Restorer can generate speech-matched f… view at source ↗
Figure 3
Figure 3. Visualization of reconstruction quality. The Audio-Visual Encoder effectively captures and reconstructs lip movements. the content of facial movements well without causing facial structural distortion. During the training process, we first use the facial blendshape capture module, which is composed of ResNet [75], to capture facial expressions as E(B), where E represents the mapping from the blendshape coefficients … view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Facial Animation Capturer. We use 3D facial blendshape coefficients to capture the expressions of characters. areas for lips and expressions. Specifically, the new attention mechanisms are defined as follows: \begin {split} V_{\text {lip}} &= V \odot M_{\text {lip}}, \…
Figure 5
Figure 5. Figure 5: Overview of Gaussian Rendering. Canonical Gaussian fields utilize a triplane representation to encode 3D head features, which are processed by the MLP to yield canonical parameters. These parameters are then integrated with lip feature, expression feature, and head pos…
Figure 6
Figure 6. Figure 6: Visualization of the triplane feature grids. The reference images (left) are projected onto three orthogonal planes: (x, y), (y, z), and (x, z). average shape of a talking head and design a deformation module that modifies these Gaussians based on audio input, as shown…
Figure 7
Figure 7. Figure 7: Learning framework of blendshape coefficient space. The VQ￾VAE model handles out-of-distribution (OOD) blendshape coefficients by embedding them into a learned codebook, ensuring accurate reconstruction and addressing variations in facial expressions. and high-quality …
Figure 8
Figure 8. Figure 8: Structure of the Torso-Inpainting Restorer. We manually construct impaired inputs in training to build the network’s complementation ability. During rendering, a fixed 15-pixel expansion is applied to the facial mask to obtain a robust background region (1 − M − δ)Fsou…
Figure 9
Figure 9. Figure 9: Qualitative comparison of facial synthesis by different methods. Our method has the best visual effect on lip movements and facial expressions without the problem of separation of head and torso. Please zoom in for better visualization. movements still show discrepanci…
Figure 12
Figure 12. Figure 12: Comparison of different trackers. SyncTalk and TalkingGaussian trackers will cause obvious facial jitter and artifacts for long-haired characters, but SyncTalk++ will improve significantly. D. Ablation Study We conduct an ablation study to systematically evaluate the …
Figure 10
Figure 10. Figure 10: Qualitative comparison of facial synthesis driven by in-the-wild audios. Our method demonstrates the most accurate lip movement while maintaining the subject’s identity well. Neutral expression Frowning and squinting Raising eyebrows Raising eyebrows and closing eyes …
Figure 11
Figure 11. Figure 11: Expression generation using OOD Audio Expression Generator. For different expression coefficients, our method can achieve highly accurate eyebrow and eye generation. highest scores across all five metrics. Specifically, SyncTalk++ attains a Lip-sync Accuracy score of …
Figure 13
Figure 13. Figure 13: Ablation study of OOD Audio Torso Restorer Without using the Torso Restorer, there will be obvious pixel missing problems, and our method can repair them well. robust algorithms. Additionally, we attempt to distinguish the authenticity of videos based on rendering def…

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Automated Synthesis of Facial Mechanisms for Conversational Animatronic Robots

    cs.RO 2026-07 conditional novelty 6.0 of 10

    A parametric linkage face template plus hierarchical collision-driven optimization synthesizes manufacturable facial mechanisms from 2D portraits and runs them with dual-identity conversational motion.

  2. GGTalker: Talking Head Systhesis with Generalizable Gaussian Priors and Identity-Specific Adaptation

    cs.CV 2025-06 conditional novelty 5.0 of 10

    GGTalker combines large-scale audio-to-expression and expression-to-texture priors with rapid per-identity fine-tuning to create high-quality 3D talking heads from a short video.

Reference graph

Works this paper leans on

94 extracted references · 38 canonical work pages · cited by 2 Pith papers

  1. [1]

    Neural voice puppetry: Audio-driven facial reenactment,

    J. Thies, M. Elgharib, A. Tewari, C. Theobalt, and M. Nießner, “Neural voice puppetry: Audio-driven facial reenactment,” inComputer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVI 16. Springer, 2020, pp. 716–731

    2020

  2. [2]

    Megadance: Mixture-of-experts architecture for genre-aware 3d dance generation,

    K. Yang, X. Tang, Z. Peng, Y . Hu, J. He, and H. Liu, “Megadance: Mixture-of-experts architecture for genre-aware 3d dance generation,” arXiv preprint arXiv:2505.17543, 2025. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 16

    arXiv 2025

  3. [3]

    Selftalk: A self-supervised commutative training diagram to compre- hend 3d talking faces,

    Z. Peng, Y . Luo, Y . Shi, H. Xu, X. Zhu, H. Liu, J. He, and Z. Fan, “Selftalk: A self-supervised commutative training diagram to compre- hend 3d talking faces,” inProceedings of the 31st ACM International Conference on Multimedia, 2023, pp. 5292–5301

    2023

  4. [4]

    Meta-Learning Empowered Meta-Face: Personalized Speaking Style Adaptation for Audio-Driven 3D Talking Face Animation

    X. Zhou, F. Li, Z. Peng, K. Wu, J. He, B. Qin, Z. Fan, and H. Liu, “Meta-learning empowered meta-face: Personalized speaking style adap- tation for audio-driven 3d talking face animation,”arXiv preprint arXiv:2408.09357, 2024

    work page Pith review arXiv 2024

  5. [5]

    Deep video portraits,

    H. Kim, P. Garrido, A. Tewari, W. Xu, J. Thies, M. Niessner, P. P ´erez, C. Richardt, M. Zollh¨ofer, and C. Theobalt, “Deep video portraits,”ACM transactions on graphics (TOG), vol. 37, no. 4, pp. 1–14, 2018

    2018

  6. [6]

    Dualtalk: Dual-speaker interaction for 3d talking head conversations,

    Z. Peng, Y . Fan, H. Wu, X. Wang, H. Liu, J. He, and Z. Fan, “Dualtalk: Dual-speaker interaction for 3d talking head conversations,” inProceed- ings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 21 055–21 064

    2025

  7. [7]

    VGG-Tex: A Vivid Geometry-Guided Facial Texture Estimation Model for High Fidelity Monocular 3D Face Reconstruction

    H. Wu, Z. Peng, X. Zhou, Y . Cheng, J. He, H. Liu, and Z. Fan, “Vgg-tex: A vivid geometry-guided facial texture estimation model for high fidelity monocular 3d face reconstruction,”arXiv preprint arXiv:2409.09740, 2024

    work page Pith review arXiv 2024

  8. [8]

    Generative adversarial networks,

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y . Bengio, “Generative adversarial networks,” Communications of the ACM, vol. 63, no. 11, pp. 139–144, 2020

    2020

Show all 94 references
  1. [9]

    Dinet: Deformation inpainting network for realistic face visually dubbing on high resolution video,

    Z. Zhang, Z. Hu, W. Deng, C. Fan, T. Lv, and Y . Ding, “Dinet: Deformation inpainting network for realistic face visually dubbing on high resolution video,”arXiv preprint arXiv:2303.03988, 2023

    2023 arXiv

  2. [10]

    Seeing what you said: Talking face generation guided by a lip reading expert,

    J. Wang, X. Qian, M. Zhang, R. T. Tan, and H. Li, “Seeing what you said: Talking face generation guided by a lip reading expert,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 14 653–14 662

    2023

  3. [11]

    Stylesync: High-fidelity generalized and personalized lip sync in style-based generator,

    J. Guan, Z. Zhang, H. Zhou, T. Hu, K. Wang, D. He, H. Feng, J. Liu, E. Ding, Z. Liuet al., “Stylesync: High-fidelity generalized and personalized lip sync in style-based generator,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. ...

    2023

  4. [12]

    Identity- preserving talking face generation with landmark and appearance priors,

    W. Zhong, C. Fang, Y . Cai, P. Wei, G. Zhao, L. Lin, and G. Li, “Identity- preserving talking face generation with landmark and appearance priors,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 9729–9738

    2023

  5. [13]

    Emmn: Emotional motion memory network for audio-driven emotional talking face generation,

    S. Tan, B. Ji, and Y . Pan, “Emmn: Emotional motion memory network for audio-driven emotional talking face generation,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 22 146–22 156

    2023

  6. [14]

    Liveportrait: Efficient portrait animation with stitching and retargeting control,

    J. Guo, D. Zhang, X. Liu, Z. Zhong, Y . Zhang, P. Wan, and D. Zhang, “Liveportrait: Efficient portrait animation with stitching and retargeting control,”arXiv preprint arXiv:2407.03168, 2024

    2024 arXiv

  7. [15]

    Flowvqtalker: High-quality emotional talking face generation through normalizing flow and quantization,

    S. Tan, B. Ji, and Y . Pan, “Flowvqtalker: High-quality emotional talking face generation through normalizing flow and quantization,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 26 317–26 327

    2024

  8. [16]

    Ganhead: Towards generative animatable neural head avatars,

    S. Wu, Y . Yan, Y . Li, Y . Cheng, W. Zhu, K. Gao, X. Li, and G. Zhai, “Ganhead: Towards generative animatable neural head avatars,” inPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 437–447

    2023

  9. [17]

    Emo: Emote portrait alive- generating expressive portrait videos with audio2video diffusion model under weak conditions,

    L. Tian, Q. Wang, B. Zhang, and L. Bo, “Emo: Emote portrait alive- generating expressive portrait videos with audio2video diffusion model under weak conditions,”arXiv preprint arXiv:2402.17485, 2024

    2024 arXiv

  10. [18]

    Dreamtalk: When expressive talking head generation meets diffusion probabilistic models,

    Y . Ma, S. Zhang, J. Wang, X. Wang, Y . Zhang, and Z. Deng, “Dreamtalk: When expressive talking head generation meets diffusion probabilistic models,”arXiv preprint arXiv:2312.09767, 2023

    2023 arXiv

  11. [19]

    Difftalk: Crafting diffusion models for generalized audio-driven portraits anima- tion,

    S. Shen, W. Zhao, Z. Meng, W. Li, Z. Zhu, J. Zhou, and J. Lu, “Difftalk: Crafting diffusion models for generalized audio-driven portraits anima- tion,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 1982–1991

    2023

  12. [20]

    Nerf: Representing scenes as neural radiance fields for view synthesis,

    B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,”Communications of the ACM, vol. 65, no. 1, pp. 99–106, 2021

    2021

  13. [21]

    Ad- nerf: Audio driven neural radiance fields for talking head synthesis,

    Y . Guo, K. Chen, S. Liang, Y .-J. Liu, H. Bao, and J. Zhang, “Ad- nerf: Audio driven neural radiance fields for talking head synthesis,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 5784–5794

    2021

  14. [22]

    Dfa-nerf: Person- alized talking head generation via disentangled face attributes neural rendering,

    S. Yao, R. Zhong, Y . Yan, G. Zhai, and X. Yang, “Dfa-nerf: Person- alized talking head generation via disentangled face attributes neural rendering,”arXiv preprint arXiv:2201.00791, 2022

    2022 arXiv

  15. [23]

    Learning dynamic facial radiance fields for few-shot talking head synthesis,

    S. Shen, W. Li, Z. Zhu, Y . Duan, J. Zhou, and J. Lu, “Learning dynamic facial radiance fields for few-shot talking head synthesis,” inEuropean Conference on Computer Vision. Springer, 2022, pp. 666–682

    2022

  16. [24]

    Geneface: Generalized and high-fidelity audio-driven 3d talking face synthesis,

    Z. Ye, Z. Jiang, Y . Ren, J. Liu, J. He, and Z. Zhao, “Geneface: Generalized and high-fidelity audio-driven 3d talking face synthesis,” arXiv preprint arXiv:2301.13430, 2023

    2023 arXiv

  17. [25]

    Efficient region-aware neural radiance fields for high-fidelity talking portrait synthesis,

    J. Li, J. Zhang, X. Bai, J. Zhou, and L. Gu, “Efficient region-aware neural radiance fields for high-fidelity talking portrait synthesis,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 7568–7578

    2023

  18. [26]

    Learning dynamic tetrahedra for high-quality talking head synthesis,

    Z. Zhang, R. Zheng, B. Li, C. Han, T. Li, M. Wang, T. Guo, J. Chen, Z. Liu, and M. Yang, “Learning dynamic tetrahedra for high-quality talking head synthesis,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 5209–5219

    2024

  19. [27]

    Gpavatar: Generalizable and precise head avatar from image (s),

    X. Chu, Y . Li, A. Zeng, T. Yang, L. Lin, Y . Liu, and T. Harada, “Gpavatar: Generalizable and precise head avatar from image (s),”arXiv preprint arXiv:2401.10215, 2024

    2024 arXiv

  20. [28]

    Er-nerf++: Efficient region-aware neural radiance fields for high-fidelity talking portrait synthesis,

    J. Li, J. Zhang, X. Bai, J. Zheng, J. Zhou, and L. Gu, “Er-nerf++: Efficient region-aware neural radiance fields for high-fidelity talking portrait synthesis,”Information Fusion, vol. 110, p. 102456, 2024

    2024

  21. [29]

    3d gaussian splatting for real-time radiance field rendering

    B. Kerbl, G. Kopanas, T. Leimk ¨uhler, and G. Drettakis, “3d gaussian splatting for real-time radiance field rendering.”ACM Trans. Graph., vol. 42, no. 4, pp. 139–1, 2023

    2023

  22. [30]

    Gaussiantalker: Speaker-specific talking head synthesis via 3d gaussian splatting,

    H. Yu, Z. Qu, Q. Yu, J. Chen, Z. Jiang, Z. Chen, S. Zhang, J. Xu, F. Wu, C. Lvet al., “Gaussiantalker: Speaker-specific talking head synthesis via 3d gaussian splatting,”arXiv preprint arXiv:2404.14037, 2024

    2024 arXiv

  23. [31]

    Talkinggaussian: Structure-persistent 3d talking head synthesis via gaussian splatting,

    J. Li, J. Zhang, X. Bai, J. Zheng, X. Ning, J. Zhou, and L. Gu, “Talkinggaussian: Structure-persistent 3d talking head synthesis via gaussian splatting,”arXiv preprint arXiv:2404.15264, 2024

    2024 arXiv

  24. [32]

    Deep speech 2: End-to-end speech recognition in english and mandarin,

    D. Amodei, S. Ananthanarayanan, R. Anubhai, J. Bai, E. Battenberg, C. Case, J. Casper, B. Catanzaro, Q. Cheng, G. Chenet al., “Deep speech 2: End-to-end speech recognition in english and mandarin,” in International conference on machine learning. PMLR, 2016, pp. 173– 182

    2016

  25. [33]

    Synctalk: The devil is in the synchronization for talking head synthesis,

    Z. Peng, W. Hu, Y . Shi, X. Zhu, X. Zhang, H. Zhao, J. He, H. Liu, and Z. Fan, “Synctalk: The devil is in the synchronization for talking head synthesis,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 666–676

    2024

  26. [34]

    Lip movements generation at a glance,

    L. Chen, Z. Li, R. K. Maddox, Z. Duan, and C. Xu, “Lip movements generation at a glance,” inProceedings of the European conference on computer vision (ECCV), 2018, pp. 520–535

    2018

  27. [35]

    Towards automatic face-to-face translation,

    P. KR, R. Mukhopadhyay, J. Philip, A. Jha, V . Namboodiri, and C. Jawahar, “Towards automatic face-to-face translation,” inProceedings of the 27th ACM international conference on multimedia, 2019, pp. 1428–1436

    2019

  28. [36]

    Hierarchical cross-modal talking face generation with dynamic pixel-wise loss,

    L. Chen, R. K. Maddox, Z. Duan, and C. Xu, “Hierarchical cross-modal talking face generation with dynamic pixel-wise loss,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 7832–7841

    2019

  29. [37]

    Talking face generation by adversarially disentangled audio-visual representation,

    H. Zhou, Y . Liu, Z. Liu, P. Luo, and X. Wang, “Talking face generation by adversarially disentangled audio-visual representation,” inProceed- ings of the AAAI conference on artificial intelligence, vol. 33, no. 01, 2019, pp. 9299–9306

    2019

  30. [38]

    Speech-driven facial animation using cascaded gans for learning of motion and texture,

    D. Das, S. Biswas, S. Sinha, and B. Bhowmick, “Speech-driven facial animation using cascaded gans for learning of motion and texture,” inComputer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXX 16. Springer, 2020, pp. 408–424

    2020

  31. [39]

    Realistic speech-driven facial animation with gans,

    K. V ougioukas, S. Petridis, and M. Pantic, “Realistic speech-driven facial animation with gans,”International Journal of Computer Vision, vol. 128, pp. 1398–1413, 2020

    2020

  32. [40]

    Learned spatial representations for few-shot talking-head synthesis,

    M. Meshry, S. Suri, L. S. Davis, and A. Shrivastava, “Learned spatial representations for few-shot talking-head synthesis,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 13 829–13 838

    2021

  33. [41]

    Pose- controllable talking face generation by implicitly modularized audio- visual representation,

    H. Zhou, Y . Sun, W. Wu, C. C. Loy, X. Wang, and Z. Liu, “Pose- controllable talking face generation by implicitly modularized audio- visual representation,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 4176–4186

    2021

  34. [42]

    Everybody’s talkin’: Let me talk as you want,

    L. Song, W. Wu, C. Qian, R. He, and C. C. Loy, “Everybody’s talkin’: Let me talk as you want,”IEEE Transactions on Information Forensics and Security, vol. 17, pp. 585–598, 2022

    2022

  35. [43]

    A lip sync expert is all you need for speech to lip generation in the wild,

    K. Prajwal, R. Mukhopadhyay, V . P. Namboodiri, and C. Jawahar, “A lip sync expert is all you need for speech to lip generation in the wild,” in Proceedings of the 28th ACM international conference on multimedia, 2020, pp. 484–492

    2020

  36. [44]

    Makelttalk: speaker-aware talking-head animation,

    Y . Zhou, X. Han, E. Shechtman, J. Echevarria, E. Kalogerakis, and D. Li, “Makelttalk: speaker-aware talking-head animation,”ACM Transactions On Graphics (TOG), vol. 39, no. 6, pp. 1–15, 2020. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 17

    2020

  37. [45]

    Live speech portraits: real-time photore- alistic talking-head animation,

    Y . Lu, J. Chai, and X. Cao, “Live speech portraits: real-time photore- alistic talking-head animation,”ACM Transactions on Graphics (TOG), vol. 40, no. 6, pp. 1–17, 2021

    2021

  38. [46]

    Audio2head: Audio- driven one-shot talking-head generation with natural head motion,

    S. Wang, L. Li, Y . Ding, C. Fan, and X. Yu, “Audio2head: Audio- driven one-shot talking-head generation with natural head motion,”arXiv preprint arXiv:2107.09293, 2021

    2021 arXiv

  39. [47]

    Sadtalker: Learning realistic 3d motion coefficients for styl- ized audio-driven single image talking face animation,

    W. Zhang, X. Cun, X. Wang, Y . Zhang, X. Shen, Y . Guo, Y . Shan, and F. Wang, “Sadtalker: Learning realistic 3d motion coefficients for styl- ized audio-driven single image talking face animation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogn...

    2023

  40. [48]

    Hallo: Hierarchical audio-driven visual synthesis for portrait image animation,

    M. Xu, H. Li, Q. Su, H. Shang, L. Zhang, C. Liu, J. Wang, L. Van Gool, Y . Yao, and S. Zhu, “Hallo: Hierarchical audio-driven visual synthesis for portrait image animation,”arXiv preprint arXiv:2406.08801, 2024

    2024 arXiv

  41. [49]

    Echomimic: Lifelike audio- driven portrait animations through editable landmark conditions,

    Z. Chen, J. Cao, Z. Chen, Y . Li, and C. Ma, “Echomimic: Lifelike audio- driven portrait animations through editable landmark conditions,”arXiv preprint arXiv:2407.08136, 2024

    2024 arXiv

  42. [50]

    Omnisync: Towards universal lip synchronization via diffusion transformers,

    Z. Peng, J. Liu, H. Zhang, X. Liu, S. Tang, P. Wan, D. Zhang, H. Liu, and J. He, “Omnisync: Towards universal lip synchronization via diffusion transformers,”arXiv preprint arXiv:2505.21448, 2025

    2025

  43. [51]

    High- resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10 684–10 695

    2022

  44. [52]

    Nerf in the wild: Neural radiance fields for unconstrained photo collections,

    R. Martin-Brualla, N. Radwan, M. S. Sajjadi, J. T. Barron, A. Doso- vitskiy, and D. Duckworth, “Nerf in the wild: Neural radiance fields for unconstrained photo collections,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 7210–7219

    2021

  45. [53]

    Nerf: Neural radiance field in 3d vision, a comprehensive review,

    K. Gao, Y . Gao, H. He, D. Lu, L. Xu, and J. Li, “Nerf: Neural radiance field in 3d vision, a comprehensive review,”arXiv preprint arXiv:2210.00379, 2022

    2022 arXiv

  46. [54]

    Semantic-aware implicit neural audio-driven video portrait generation,

    X. Liu, Y . Xu, Q. Wu, H. Zhou, W. Wu, and B. Zhou, “Semantic-aware implicit neural audio-driven video portrait generation,” inEuropean Conference on Computer Vision. Springer, 2022, pp. 106–125

    2022

  47. [55]

    Real-time neural radiance talking portrait synthesis via audio-spatial decomposition,

    J. Tang, K. Wang, H. Zhou, X. Chen, D. He, T. Hu, J. Liu, G. Zeng, and J. Wang, “Real-time neural radiance talking portrait synthesis via audio-spatial decomposition,”arXiv preprint arXiv:2211.12368, 2022

    2022 arXiv

  48. [56]

    Instant neural graphics primitives with a multiresolution hash encoding,

    T. M ¨uller, A. Evans, C. Schied, and A. Keller, “Instant neural graphics primitives with a multiresolution hash encoding,”ACM transactions on graphics (TOG), vol. 41, no. 4, pp. 1–15, 2022

    2022

  49. [57]

    Portrait4d: Learning one-shot 4d head avatar synthesis using synthetic data,

    Y . Deng, D. Wang, X. Ren, X. Chen, and B. Wang, “Portrait4d: Learning one-shot 4d head avatar synthesis using synthetic data,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 7119–7130

    2024

  50. [58]

    Dynamic neural radiance fields for monocular 4d facial avatar reconstruction,

    G. Gafni, J. Thies, M. Zollhofer, and M. Nießner, “Dynamic neural radiance fields for monocular 4d facial avatar reconstruction,” inPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 8649–8658

    2021

  51. [59]

    Im avatar: Implicit morphable head avatars from videos,

    Y . Zheng, V . F. Abrevaya, M. C. B ¨uhler, X. Chen, M. J. Black, and O. Hilliges, “Im avatar: Implicit morphable head avatars from videos,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 13 545–13 555

    2022

  52. [60]

    Instant volumetric head avatars,

    W. Zielonka, T. Bolkart, and J. Thies, “Instant volumetric head avatars,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 4574–4584

    2023

  53. [61]

    Pointavatar: Deformable point-based head avatars from videos,

    Y . Zheng, W. Yifan, G. Wetzstein, M. J. Black, and O. Hilliges, “Pointavatar: Deformable point-based head avatars from videos,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 21 057–21 067

    2023

  54. [62]

    Scaffold-gs: Structured 3d gaussians for view-adaptive rendering,

    T. Lu, M. Yu, L. Xu, Y . Xiangli, L. Wang, D. Lin, and B. Dai, “Scaffold-gs: Structured 3d gaussians for view-adaptive rendering,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 20 654–20 664

    2024

  55. [63]

    Mip-splatting: Alias-free 3d gaussian splatting,

    Z. Yu, A. Chen, B. Huang, T. Sattler, and A. Geiger, “Mip-splatting: Alias-free 3d gaussian splatting,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 19 447–19 456

    2024

  56. [64]

    3dgs- avatar: Animatable avatars via deformable 3d gaussian splatting,

    Z. Qian, S. Wang, M. Mihajlovic, A. Geiger, and S. Tang, “3dgs- avatar: Animatable avatars via deformable 3d gaussian splatting,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 5020–5030

    2024

  57. [65]

    Gauhuman: Articulated gaussian splatting from monocular human videos,

    S. Hu, T. Hu, and Z. Liu, “Gauhuman: Articulated gaussian splatting from monocular human videos,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 20 418–20 431

    2024

  58. [66]

    Psavatar: A point- based morphable shape model for real-time head avatar creation with 3d gaussian splatting,

    Z. Zhao, Z. Bao, Q. Li, G. Qiu, and K. Liu, “Psavatar: A point- based morphable shape model for real-time head avatar creation with 3d gaussian splatting,”arXiv preprint arXiv:2401.12900, 2024

    2024 arXiv

  59. [67]

    Gaussianavatars: Photorealistic head avatars with rigged 3d gaussians,

    S. Qian, T. Kirschstein, L. Schoneveld, D. Davoli, S. Giebenhain, and M. Nießner, “Gaussianavatars: Photorealistic head avatars with rigged 3d gaussians,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 20 299–20 309

    2024

  60. [68]

    Gaussian head avatar: Ultra high-fidelity head avatar via dynamic gaussians,

    Y . Xu, B. Chen, Z. Li, H. Zhang, L. Wang, Z. Zheng, and Y . Liu, “Gaussian head avatar: Ultra high-fidelity head avatar via dynamic gaussians,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 1931–1941

    2024

  61. [69]

    Gaussiantalker: Real-time high-fidelity talking head synthesis with audio-driven 3d gaussian splatting,

    K. Cho, J. Lee, H. Yoon, Y . Hong, J. Ko, S. Ahn, and S. Kim, “Gaussiantalker: Real-time high-fidelity talking head synthesis with audio-driven 3d gaussian splatting,”arXiv preprint arXiv:2404.16012, 2024

    2024 arXiv

  62. [70]

    wav2vec 2.0: A framework for self-supervised learning of speech representations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” Advances in neural information processing systems, vol. 33, pp. 12 449– 12 460, 2020

    2020

  63. [71]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 3451–3460, 2021

    2021

  64. [72]

    Deep audio-visual speech recognition,

    T. Afouras, J. S. Chung, A. Senior, O. Vinyals, and A. Zisserman, “Deep audio-visual speech recognition,”IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 12, pp. 8717–8727, 2018

    2018

  65. [73]

    Out of time: automated lip sync in the wild,

    J. S. Chung and A. Zisserman, “Out of time: automated lip sync in the wild,” inComputer Vision–ACCV 2016 Workshops: ACCV 2016 In- ternational Workshops, Taipei, Taiwan, November 20-24, 2016, Revised Selected Papers, Part II 13. Springer, 2017, pp. 251–263

    2016

  66. [74]

    Emotalk: Speech-driven emotional disentanglement for 3d face animation,

    Z. Peng, H. Wu, Z. Song, H. Xu, X. Zhu, J. He, H. Liu, and Z. Fan, “Emotalk: Speech-driven emotional disentanglement for 3d face animation,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 20 687–20 697

    2023

  67. [75]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778

    2016

  68. [76]

    face-alignment: 2d and 3d face alignment library build using pytorch,

    A. Bulat, “face-alignment: 2d and 3d face alignment library build using pytorch,” https://github.com/1adrianb/face-alignment, 2017

    2017

  69. [77]

    A 3d face model for pose and illumination invariant face recognition,

    P. Paysan, R. Knothe, B. Amberg, S. Romdhani, and T. Vetter, “A 3d face model for pose and illumination invariant face recognition,” in2009 sixth IEEE international conference on advanced video and signal based surveillance. Ieee, 2009, pp. 296–301

    2009

  70. [78]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  71. [79]

    Ewa volume splatting,

    M. Zwicker, H. Pfister, J. Van Baar, and M. Gross, “Ewa volume splatting,” inProceedings Visualization, 2001. VIS’01.IEEE, 2001, pp. 29–538

    2001

  72. [80]

    Efficient geometry-aware 3d generative adversarial networks,

    E. R. Chan, C. Z. Lin, M. A. Chan, K. Nagano, B. Pan, S. De Mello, O. Gallo, L. J. Guibas, J. Tremblay, S. Khamiset al., “Efficient geometry-aware 3d generative adversarial networks,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp...

    2022

  73. [81]

    K-planes: Explicit radiance fields in space, time, and appearance,

    S. Fridovich-Keil, G. Meanti, F. R. Warburg, B. Recht, and A. Kanazawa, “K-planes: Explicit radiance fields in space, time, and appearance,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 12 479–12 488

    2023

  74. [82]

    Tri- miprf: Tri-mip representation for efficient anti-aliasing neural radiance fields,

    W. Hu, Y . Wang, L. Ma, B. Yang, L. Gao, X. Liu, and Y . Ma, “Tri- miprf: Tri-mip representation for efficient anti-aliasing neural radiance fields,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 19 774–19 783

    2023

  75. [83]

    Neural discrete representation learning,

    A. Van Den Oord, O. Vinyalset al., “Neural discrete representation learning,”Advances in neural information processing systems, vol. 30, 2017

    2017

  76. [84]

    Videoretalking: Audio-based lip synchronization for talking head video editing in the wild,

    K. Cheng, X. Cun, Y . Zhang, M. Xia, F. Yin, M. Zhu, X. Wang, J. Wang, and N. Wang, “Videoretalking: Audio-based lip synchronization for talking head video editing in the wild,” inSIGGRAPH Asia 2022 Conference Papers, 2022, pp. 1–9

    2022

  77. [85]

    The unreasonable effectiveness of deep features as a perceptual metric,

    R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 586–595

    2018

  78. [86]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium,

    M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “Gans trained by a two time-scale update rule converge to a local nash equilibrium,”Advances in neural information processing systems, vol. 30, 2017. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 18

    2017

  79. [87]

    Ranksrgan: Generative adver- sarial networks with ranker for image super-resolution,

    W. Zhang, Y . Liu, C. Dong, and Y . Qiao, “Ranksrgan: Generative adver- sarial networks with ranker for image super-resolution,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 3096–3105

    2019

  80. [88]

    Making a “completely blind

    A. Mittal, R. Soundararajan, and A. C. Bovik, “Making a “completely blind” image quality analyzer,”IEEE Signal processing letters, vol. 20, no. 3, pp. 209–212, 2012

    2012

  81. [89]

    No-reference image quality assessment in the spatial domain,

    A. Mittal, A. K. Moorthy, and A. C. Bovik, “No-reference image quality assessment in the spatial domain,”IEEE Transactions on image processing, vol. 21, no. 12, pp. 4695–4708, 2012

    2012

  82. [90]

    Blindly assess image quality in the wild guided by a self-adaptive hyper network,

    S. Su, Q. Yan, Y . Zhu, C. Zhang, X. Ge, J. Sun, and Y . Zhang, “Blindly assess image quality in the wild guided by a self-adaptive hyper network,” inIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020

    2020

  83. [91]

    Cross-dataset learning and person-specific normalisation for automatic action unit detection,

    T. Baltru ˇsaitis, M. Mahmoud, and P. Robinson, “Cross-dataset learning and person-specific normalisation for automatic action unit detection,” in 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), vol. 6. IEEE, 2015, pp. 1–6

    2015

  84. [92]

    Protecting celebrities from deepfake with identity consistency transformer,

    X. Dong, J. Bao, D. Chen, T. Zhang, W. Zhang, N. Yu, D. Chen, F. Wen, and B. Guo, “Protecting celebrities from deepfake with identity consistency transformer,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 9468–9478

    2022

  85. [93]

    Deepfake detection by analyz- ing convolutional traces,

    L. Guarnera, O. Giudice, and S. Battiato, “Deepfake detection by analyz- ing convolutional traces,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, 2020, pp. 666– 667

    2020

  86. [94]

    Deepfakes and beyond: A survey of face manipulation and fake detection,

    R. Tolosana, R. Vera-Rodriguez, J. Fierrez, A. Morales, and J. Ortega- Garcia, “Deepfakes and beyond: A survey of face manipulation and fake detection,”Information Fusion, vol. 64, pp. 131–148, 2020

    2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.