Pith. sign in

REVIEW 6 cited by

GeneFace: Generalized and High-Fidelity Audio-Driven 3D Talking Face Synthesis

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2301.13430 v1 pith:5SF6W5HC submitted 2023-01-31 cs.CV

Zhenhui Ye , Ziyue Jiang , Yi Ren , Jinglin Liu , JinZheng He , Zhou Zhao This is my paper
classification cs.CV
keywords audiofacegeneralizedhigh-fidelitynerf-basedtalkinggenefacegeneration
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Generating photo-realistic video portrait with arbitrary speech audio is a crucial problem in film-making and virtual reality. Recently, several works explore the usage of neural radiance field in this task to improve 3D realness and image fidelity. However, the generalizability of previous NeRF-based methods to out-of-domain audio is limited by the small scale of training data. In this work, we propose GeneFace, a generalized and high-fidelity NeRF-based talking face generation method, which can generate natural results corresponding to various out-of-domain audio. Specifically, we learn a variaitional motion generator on a large lip-reading corpus, and introduce a domain adaptative post-net to calibrate the result. Moreover, we learn a NeRF-based renderer conditioned on the predicted facial motion. A head-aware torso-NeRF is proposed to eliminate the head-torso separation problem. Extensive experiments show that our method achieves more generalized and high-fidelity talking face generation compared to previous methods.

Discussion (0). Sign in to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Mask-Free Audio-driven Talking Face Generation for Enhanced Visual Quality and Identity Preservation

    cs.CV 2025-07 conditional novelty 6.0 of 10

    MF-Talk, a mask-free and identity-reference-free three-stage pipeline, improves visual quality and identity preservation in talking-face generation while remaining competitive on lip-sync.

  2. MoDA: Multi-modal Diffusion Architecture for Talking Head Generation

    cs.GR 2025-07 conditional novelty 5.0 of 10

    MoDA uses flow matching in a compact face-motion space with a progressively fused multi-modal transformer to generate expressive, lip-synced talking-head videos from a single image and audio.

  3. Few-Shot Identity Adaptation for 3D Talking Heads via Global Gaussian Field

    cs.CV 2025-06 conditional novelty 5.0 of 10

    A shared global Gaussian field plus identity embeddings lets a 3D talking head model adapt to new speakers with a few seconds of footage while improving quality over prior per-identity models.

  4. GGTalker: Talking Head Systhesis with Generalizable Gaussian Priors and Identity-Specific Adaptation

    cs.CV 2025-06 conditional novelty 5.0 of 10

    GGTalker combines large-scale audio-to-expression and expression-to-texture priors with rapid per-identity fine-tuning to create high-quality 3D talking heads from a short video.

  5. SyncTalk++: High-Fidelity and Efficient Synchronized Talking Heads Synthesis Using Gaussian Splatting

    cs.CV 2025-06 conditional novelty 5.0 of 10

    SyncTalk++ synthesizes speech-driven talking-head videos via 3D Gaussian Splatting and reports state-of-the-art synchronization and quality at up to 101 FPS.

  6. Audio2Face-3D: Audio-driven Realistic Facial Animation For Digital Avatars

    cs.GR 2025-08 conditional novelty 4.0 of 10

    Audio2Face-3D: a real-time, emotion-controllable, open-sourced audio-to-facial-animation system from NVIDIA for digital avatars.

Pith tools