GenSync: A Generalized Talking Head Framework for Audio-driven Multi-Subject Lip-Sync using 3D Gaussian Splatting

Anushka Agarwal; Muhammad Yusuf Hassan; Talha Chafekar

REVIEW 4 major objections 5 minor 1 cited by

GenSync: A Generalized Talking Head Framework for Audio-driven Multi-Subject Lip-Sync using 3D Gaussian Splatting

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GenSync shows a single 3D Gaussian model can lip-sync multiple speakers from audio, cutting training time 6.8x.

desk verdict A fixed-set multi-identity 3DGS talking head model with a modest but real speedup; the 'generalization' claim is untested and the paper overstates it. read the letter →

arxiv 2505.01928 v1 pith:RYXD7RMI submitted 2025-05-03 cs.CV

Anushka Agarwal , Muhammad Yusuf Hassan , Talha Chafekar This is my paper · ORCID

classification cs.CV

keywords audio-drivenlip-synctalkingheadsynthesis3DGaussianSplattingmulti-identitydisentangledrepresentationdeformationnetworkcross-attention

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a single 3D Gaussian Splatting model can learn to lip-sync videos for many different speakers from audio, without retraining per identity. It does this by separating speaker identity from the audio-driven mouth motion, so one shared deformation network can serve all speakers. If true, the training cost for a multi-speaker system drops dramatically, roughly 6.8x, while keeping lip-sync accuracy and visual quality close to per-identity baselines. A reader should care because this points the way toward scalable, practical talking-head avatars and dubbing systems.

What carries the argument

The central mechanism is the Identity-Aware Disentanglement Module, whose factorization is $M(a,i) = C[(U_1 a) \odot (U_2 i)] + W_2 a + W_3 i$. The Hadamard product $\odot$ lets articulation dynamics and speaker identity interact non-linearly in a shared latent space, while the additive terms preserve each modality's independent contribution. The output is fed as key and value into a cross-attention layer whose query is canonical Gaussian features, producing a fused spatial-audio embedding that a deformation MLP turns into per-Gaussian offsets.

What would settle it

Train GenSync on ten identities, hold out an eleventh identity, then render videos for that identity from its audio; if the SyncNet score drops well below the per-identity GaussianTalker baseline or the identity embedding fails to produce recognizable mouth shapes, the multi-speaker generalization claim is refuted.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that identity and speech articulation can be factorized in a Gaussian-splatting talking head, with a multiplicative interaction between the audio embedding and a learned identity vector. The resulting module, combined with a spatial-audio cross-attention network, lets a single deformation MLP predict per-Gaussian offsets for multiple identities. Trained jointly on 10 identities, the model reaches LPIPS of 0.078 and FID of 21.59, essentially matching GaussianTalker's 0.073 and 20.51, while improving the SyncNet lip-sync score from 12.26 to 11.98 and cutting training time from about 62 hours to 9 hours.

Load-bearing premise

The shared deformation network, conditioned on a learned identity vector and audio, can capture the mouth shapes and articulation of any speaker from roughly ten minutes of training footage per identity, a claim tested on only ten identities and never on a held-out identity.

Editorial extensions

If this is right

A single GenSync model replaces one-per-identity models for a fixed speaker set, cutting total training time from roughly 62 hours to 9 hours for 10 identities.
Adding a new speaker requires only the canonical-stage training for that identity plus joint fine-tuning, instead of a full per-identity deformation model.
The identity embedding captures speaking style: swapping embeddings transfers articulation characteristics across speakers without retraining.
The model tolerates novel audio distributions, generating plausible lip motion even when the driving voice is from a speaker of a different gender.
Competitive LPIPS and FID scores with a better SyncNet score suggest the shared model does not sacrifice lip-sync fidelity for multi-identity convenience.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: If the identity vector generalizes to unseen faces, the same architecture could support few-shot enrollment for new speakers from minutes of footage, but the paper does not test held-out identities, so this remains open.
Editorial inference: The disentanglement module is agnostic to the audio encoder, so pairing it with a stronger audio feature extractor could extend GenSync to multilingual or singing input.
Editorial inference: The reported 9-hour training figure assumes the two-stage schedule; a single-stage training approach, which the paper lists as future work, might cut cost further or change the quality trade-off.
Editorial inference: The identity-swap experiment suggests the embedding encodes articulatory style, so a natural next test is whether interpolating between identity embeddings produces intermediate speaking styles.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The one real thing this paper does is show that GaussianTalker's deformation pipeline can be shared across ten identities with a simple identity-audio multiplicative fusion, and that the numbers stay close to the per-identity baseline while training much faster. That is a concrete, useful result for people building multi-speaker avatar pipelines. The combination of MI-NeRF's factorized identity-audio idea with GaussianTalker's 3DGS deformation network is new as far as the cited literature goes, and the identity-swapping experiment (feeding A's embedding into B) is a clever probe that suggests the identity vector actually carries speaking style. Credit where due: the method section is clear, the baseline comparison is direct, and the qualitative figures support the claim that quality is comparable for the fixed set of identities tested.

The soft spots are exactly where the stress-test points. The abstract says a unified network synthesizes videos for multiple speakers and eliminates per-identity training, but Section 3.1 still trains a per-identity canonical stage (8k iterations per identity). That is an overstatement, though not a fatal one: the canonical stage is a small fraction of total cost, and the joint deformation network is genuinely shared. The bigger problem is the 'generalized' framing. The identity vector is a learned per-identity embedding; there is no procedure for inferring it for an unseen identity, and Section 3.3 only drives novel audio through the training identities. So the paper demonstrates a multi-identity model for a fixed set, not a generalized talking head. That is still a legitimate contribution, but the claim should be scaled back.

Table 1 has no error bars or repeated runs, the dataset is described only as 'public YouTube videos' without identity counts or splits, and no code or data are provided. These are standard workshop-level shortcomings, but they mean the 6.8x speedup and the metric parity are single-run anecdotes. The SyncNet score for GenSync is actually slightly better than GaussianTalker's, which is plausible given the shared training, but it needs more runs to be credible.

Who is this for? Researchers working on multi-identity talking head synthesis who want a simple baseline for 3DGS-based approaches. As a CVPR workshop paper, it is serviceable; as a main-conference claim, it would need held-out identity tests, error bars, and a clearer statement of what 'generalization' means.

My take: it deserves peer review rather than desk rejection, because the fixed-set multi-identity result is a real building block and the comparison to GaussianTalker is informative. But it should not be accepted as is. The authors should either add a held-out identity experiment (optimize the identity vector on a short video, then test the shared deformation network) or explicitly reframe the claim as fixed-set multi-identity with faster joint training. The math and the basic pipeline hold up; the writing and evaluation overreach.

Referee Report

4 major / 5 minor

Summary. The paper presents GenSync, a multi-identity audio-driven lip-sync framework built on 3D Gaussian Splatting. The method uses a disentanglement module that combines audio embeddings with a learned identity vector via a multiplicative transform, followed by a fused spatial-audio attention network that predicts per-Gaussian deformations. The authors train jointly on 10 identities from YouTube videos, after a per-identity canonical stage, and report LPIPS, FID, and SyncNet scores comparable to GaussianTalker while claiming 6.8x faster training. They also show qualitative results for switching identity embeddings and for driving with novel audio distributions.

Significance. If the generalization claims are substantiated, GenSync would address a real scalability limitation of per-identity 3DGS talking-head models and would be a practical contribution to audio-driven avatar synthesis. The factorized identity-audio formulation is a sensible design direction, and the training-time reduction is a meaningful practical goal. However, the evidence provided does not yet support the central 'generalized multi-speaker' claim: the method is evaluated only on the 10 identities used in joint training, the identity-vector inference for unseen speakers is not specified, and the quantitative comparison lacks error bars or repeated runs. The remaining per-identity canonical stage also contradicts the abstract's 'no per-identity training' wording.

major comments (4)

[Abstract and Section 3.1] The abstract states that GenSync 'eliminates the need for per-identity training,' but Section 3.1 reports that the canonical stage trains for 8k iterations per identity before joint training. This is a direct contradiction. Please either revise the abstract to reflect a two-stage procedure with a per-identity canonical fitting phase or provide evidence that the canonical stage is not identity-specific training in the sense meant by the claim.
[Section 2.1, Eq. (1)] The identity vector i is a learned parameter per training identity, but the paper never specifies how to obtain i for a new speaker. All experiments use the same 10 identities from joint training (Section 3.3 still uses training identities with A's audio). Consequently, the central claim of a 'generalized multi-speaker' model is untested. Add a held-out identity experiment where, for example, only i is optimized while the shared network is frozen, or explicitly restrict the claims to fixed-set multi-identity synthesis.
[Table 1 and Section 3.1] The quantitative comparison reports single point values without error bars, repeated runs, or statistical tests. Given the small gaps (LPIPS 0.073 vs 0.078; Sync 12.26 vs 11.98), the reader cannot judge whether GenSync is actually comparable to GaussianTalker. Report mean and standard deviation over at least three independent runs, or provide per-identity breakdowns, to support the comparability claim.
[Section 3, dataset description] The dataset is described only as 'publicly available videos sourced from YouTube' trimmed to under one minute at 512x512, with a 9:1 train/test split. This is insufficient for reproducibility and for interpreting the comparison. Specify the number of identities, the number of frames per identity, audio feature extraction, and whether both methods are evaluated on identical test frames. Also clarify whether the reported 9-hour training time includes the per-identity canonical stages for all 10 identities.

minor comments (5)

[References] References [3] and [4] both cite the same MI-NeRF paper; please remove the duplicate.
[Section 4, Conclusion] 'Up to 6.8× training speed' should be phrased as '6.8× faster training' or 'training speedup' for clarity.
[Figure 1 caption] The figure does not define the arrow styles or the meaning of the different colored blocks; add a legend or describe them in the caption.
[Section 3.2] The identity-switching experiment is purely qualitative; a quantitative measure such as mouth-aperture over time would strengthen the claim that the identity embedding captures speaking style.
[Section 2.1 and Table 1] The term 'disentanglement' is asserted but not measured; an ablation separating the multiplicative and additive terms, or a visualization of the latent space, would support the disentanglement claim. Also, the text says synchronization scores are 'slightly higher' while the table marks Sync with a down arrow, so the direction of improvement should be stated explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: GenSync's claims are empirical comparisons against external benchmarks, not derivations that reduce to their inputs.

full rationale

The paper's central claims are empirical: a shared deformation network trained jointly on 10 identities, evaluated with LPIPS, FID, and SyncNet against GaussianTalker, plus a reported 6.8x training-time reduction. No equation in the paper defines a target metric in terms of a fitted parameter that is then called a prediction. The Disentanglement Module (Eqn. 1) is a network architecture, not a fitted proxy for the evaluation metrics. The identity-embedding switching experiment (Sec. 3.2) is qualitative and does not by construction force any reported number. The '6.8x faster' claim is a self-reported timing comparison, not a tautology, and the comparison is external to the paper's own fitted values. Citations to MI-NeRF and GaussianTalker are prior external work, not self-citations, and they are used as baselines or architectural inspiration rather than as load-bearing justification for the paper's own results. The notable weakness is that the 'generalized multi-speaker' claim is not tested on a held-out identity, and the per-identity canonical stage (8k iterations per identity) partially undercuts the abstract's 'eliminates the need for per-identity training.' However, these are correctness and completeness concerns, not circular reasoning: the reported metrics could in principle be false independently of how the model is defined. No circular step can be exhibited from the paper's text, so the circularity score is 0.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on learned identity embeddings, a fixed training schedule, and a shared canonical space assumption, plus unstated audio feature extraction. No new physical or conceptual entities are introduced beyond standard neural network modules.

free parameters (2)

per-identity identity embedding i = learned 10 vectors, values not reported
Section 2.1 learns an identity vector i per speaker; the model's behavior depends on these learned embeddings.
training schedule (8k canonical iterations per identity, 50k joint iterations) = 8k per identity, 50k joint
These numbers were chosen by the authors and directly determine the reported training time and quality.

assumptions (3)

domain assumption A single shared canonical face representation can be deformed to represent multiple identities.
Section 2.2 uses canonical features f(mu_c) shared across identities; no evidence is given that one canonical space captures diverse face shapes.
domain assumption Audio embeddings from an unspecified encoder contain enough articulatory information to drive lip motion.
Section 2.2 conditions deformation on audio embeddings a, but the audio encoder is not described or validated.
domain assumption OpenFace and Basel Face Model provide accurate eye and facial features for the cropped YouTube videos.
Section 3 preprocessing relies on these tools; errors in feature extraction would propagate to the attention inputs and deformations.

how reviews work

0 comments

Cite this review

Pith. "Pith review of GenSync: A Generalized Talking Head Framework for Audio-driven Multi-Subject Lip-Sync using 3D Gaussian Splatting." pith.science (2026). https://pith.science/paper/RYXD7RMI

@misc{pith2026250501928,
  author       = {Pith},
  title        = {Pith review of: GenSync: A Generalized Talking Head Framework for Audio-driven Multi-Subject Lip-Sync using 3D Gaussian Splatting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RYXD7RMI}},
  note         = {Machine review of arXiv:2505.01928}
}

read the original abstract

We introduce GenSync, a novel framework for multi-identity lip-synced video synthesis using 3D Gaussian Splatting. Unlike most existing 3D methods that require training a new model for each identity , GenSync learns a unified network that synthesizes lip-synced videos for multiple speakers. By incorporating a Disentanglement Module, our approach separates identity-specific features from audio representations, enabling efficient multi-identity video synthesis. This design reduces computational overhead and achieves 6.8x faster training compared to state-of-the-art models, while maintaining high lip-sync accuracy and visual quality.

Figures

Figures reproduced from arXiv: 2505.01928 by the authors.

**Figure 1.** Overview of GenSync’s pipeline. The Deformation Module decouples representations from the audio features (a) and the identity vector (i) using a multiplicative transform (Eqn. 1). The Fused Spatial-Audio Attention Module computes cross attention among the canonical features f(µc) and the concatenation of the eye features (e), viewpoint (v), and the output of the Disentanglement Module. This is used to compute the de… view at source ↗

**Figure 2.** Comparative results between GenSync (ours) and GaussianTalker (baseline) for frame-wise images from a rendered video. The red-highlighted text indicates the current syllable being spoken. GenSync utilizes a single shared model across all identities, whereas the baseline requires separately trained models. Despite this, GenSync achieves performance comparable to GaussianTalker. and SyncNet (Sync) [16] scores. LPIPS m… view at source ↗

**Figure 4.** GenSync’s output for Identity B, C and D with Iden [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗

Figures from the paper (1 more)

**Figure 3.** Figure 3: GenSync’s output for Identity A and Identity B. We use [PITH_FULL_IMAGE:figures/full_fig_p004_3.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

FaceEditTalker: Controllable Talking Head Generation with Facial Attribute Editing
cs.CV 2025-05 conditional novelty 5.0 of 10

A single framework can edit predefined facial attributes in audio-synchronized talking head videos while preserving identity and lip-sync quality.

Reference graph

Works this paper leans on

20 extracted references · 10 canonical work pages · cited by 1 Pith paper

[1]

Openface: an open source facial behavior anal- ysis toolkit

Tadas Baltru ˇsaitis, Peter Robinson, and Louis-Philippe Morency. Openface: an open source facial behavior anal- ysis toolkit. In 2016 IEEE winter conference on applications of computer vision (WACV), pages 1–10. IEEE, 2016. 2

work page 2016
[2]

Dreamavatar: Text-and-shape guided 3d hu- man avatar generation via diffusion models

Yukang Cao, Yan-Pei Cao, Kai Han, Ying Shan, and Kwan- Yee K Wong. Dreamavatar: Text-and-shape guided 3d hu- man avatar generation via diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 958–968, 2024. 1

2024
[4]

Mi-nerf: Learning a single face nerf from multiple identities

Aggelina Chatziagapi, Grigorios G Chrysos, and Dimitris Samaras. Mi-nerf: Learning a single face nerf from multiple identities. arXiv preprint arXiv:2403.19920, 2024. 1

arXiv 2024
[5]

Gaus- siantalker: Real-time high-fidelity talking head synthesis with audio-driven 3d gaussian splatting

Kyusun Cho, Joungbin Lee, Heeji Yoon, Yeobin Hong, Jaehoon Ko, Sangjun Ahn, and Seungryong Kim. Gaus- siantalker: Real-time high-fidelity talking head synthesis with audio-driven 3d gaussian splatting. arXiv preprint arXiv:2404.16012, 2024. 1, 2, 3

arXiv 2024
[6]

Morphable face models - an open frame- work

Thomas Gerig, Andreas Morel-Forster, Clemens Blumer, Bernhard Egger, Marcel Luthi, Sandro Schoenborn, and Thomas Vetter. Morphable face models - an open frame- work. In 2018 13th IEEE International Conference on Auto- matic Face & Gesture Recognition (FG 2018), pages 75–82,

work page 2018
[7]

Talk-act: Enhance textural- awareness for 2d speaking avatar reenactment with diffusion model

Jiazhi Guan, Quanwei Yang, Kaisiyuan Wang, Hang Zhou, Shengyi He, Zhiliang Xu, Haocheng Feng, Errui Ding, Jing- dong Wang, Hongtao Xie, et al. Talk-act: Enhance textural- awareness for 2d speaking avatar reenactment with diffusion model. In SIGGRAPH Asia 2024 Conference Papers, pages 1–11, 2024. 1

work page 2024
[8]

Ad-nerf: Audio driven neural ra- diance fields for talking head synthesis

Yudong Guo, Keyu Chen, Sen Liang, Yong-Jin Liu, Hujun Bao, and Juyong Zhang. Ad-nerf: Audio driven neural ra- diance fields for talking head synthesis. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5784–5794, 2021. 1

work page 2021
[9]

Generative adversarial net- works (gans) for audio-visual speech recognition in artificial intelligence iot

Y He, KP Seng, and LM Ang. Generative adversarial net- works (gans) for audio-visual speech recognition in artificial intelligence iot. information 2023, 14, 575, 2023. 1

work page 2023

Show all 20 references

[10]

Gans trained by a two time-scale update rule converge to a local nash equilib- rium

Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 2

2017
[11]

A survey of audio synthesis and lip-syncing for synthetic video generation

Anup Kadam, Sagar Rane, Arpit Kumar Mishra, Shailesh Kumar Sahu, Shubham Singh, and Shivam Kumar Pathak. A survey of audio synthesis and lip-syncing for synthetic video generation. EAI Endorsed Transactions on Creative Technologies, 8(28):e2–e2, 2021. 1

2021
[12]

3d gaussian splatting for real-time radiance field rendering

Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1,
[13]

Diff2lip: Audio conditioned dif- fusion models for lip-synchronization

Soumik Mukhopadhyay, Saksham Suri, Ravi Teja Gadde, and Abhinav Shrivastava. Diff2lip: Audio conditioned dif- fusion models for lip-synchronization. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 5292–5302, 2024. 1

2024
[14]

Synctalk: The devil is in the synchronization for talking head synthesis

Ziqiao Peng, Wentao Hu, Yue Shi, Xiangyu Zhu, Xiaomei Zhang, Hao Zhao, Jun He, Hongyan Liu, and Zhaoxin Fan. Synctalk: The devil is in the synchronization for talking head synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 666–676,
[15]

3dgs-avatar: Animatable avatars via deformable 3d gaussian splatting

Zhiyin Qian, Shaofei Wang, Marko Mihajlovic, Andreas Geiger, and Siyu Tang. 3dgs-avatar: Animatable avatars via deformable 3d gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5020–5030, 2024. 1

2024
[16]

Syncnet: Using causal con- volutions and correlating objective for time delay estimation in audio signals

Akshay Raina and Vipul Arora. Syncnet: Using causal con- volutions and correlating objective for time delay estimation in audio signals. arXiv preprint arXiv:2203.14639, 2022. 3

2022 arXiv
[17]

First order motion model for image animation

Aliaksandr Siarohin, St ´ephane Lathuili`ere, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe. First order motion model for image animation. Advances in neural information processing systems, 32, 2019. 1

2019
[18]

Masked lip-sync prediction by audio-visual contextual exploitation in transformers

Yasheng Sun, Hang Zhou, Kaisiyuan Wang, Qianyi Wu, Zhibin Hong, Jingtuo Liu, Errui Ding, Jingdong Wang, Zi- wei Liu, and Koike Hideki. Masked lip-sync prediction by audio-visual contextual exploitation in transformers. In SIG- GRAPH Asia 2022 Conference Papers, pages 1–9, 2022. 1

2022
[19]

X2face: A network for controlling face generation using images, audio, and pose codes

Olivia Wiles, A Koepke, and Andrew Zisserman. X2face: A network for controlling face generation using images, audio, and pose codes. In Proceedings of the European conference on computer vision (ECCV), pages 670–686, 2018. 1

2018
[20]

X-portrait: Expressive portrait anima- tion with hierarchical motion attention

You Xie, Hongyi Xu, Guoxian Song, Chao Wang, Yichun Shi, and Linjie Luo. X-portrait: Expressive portrait anima- tion with hierarchical motion attention. In ACM SIGGRAPH 2024 Conference Papers, pages 1–11, 2024. 1

2024
[21]

The unreasonable effectiveness of deep features as a perceptual metric

Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018. 2

2018

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

Openface: an open source facial behavior anal- ysis toolkit

Tadas Baltru ˇsaitis, Peter Robinson, and Louis-Philippe Morency. Openface: an open source facial behavior anal- ysis toolkit. In 2016 IEEE winter conference on applications of computer vision (WACV), pages 1–10. IEEE, 2016. 2

work page 2016

[2] [2]

Dreamavatar: Text-and-shape guided 3d hu- man avatar generation via diffusion models

Yukang Cao, Yan-Pei Cao, Kai Han, Ying Shan, and Kwan- Yee K Wong. Dreamavatar: Text-and-shape guided 3d hu- man avatar generation via diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 958–968, 2024. 1

2024

[3] [4]

Mi-nerf: Learning a single face nerf from multiple identities

Aggelina Chatziagapi, Grigorios G Chrysos, and Dimitris Samaras. Mi-nerf: Learning a single face nerf from multiple identities. arXiv preprint arXiv:2403.19920, 2024. 1

arXiv 2024

[4] [5]

Gaus- siantalker: Real-time high-fidelity talking head synthesis with audio-driven 3d gaussian splatting

Kyusun Cho, Joungbin Lee, Heeji Yoon, Yeobin Hong, Jaehoon Ko, Sangjun Ahn, and Seungryong Kim. Gaus- siantalker: Real-time high-fidelity talking head synthesis with audio-driven 3d gaussian splatting. arXiv preprint arXiv:2404.16012, 2024. 1, 2, 3

arXiv 2024

[5] [6]

Morphable face models - an open frame- work

Thomas Gerig, Andreas Morel-Forster, Clemens Blumer, Bernhard Egger, Marcel Luthi, Sandro Schoenborn, and Thomas Vetter. Morphable face models - an open frame- work. In 2018 13th IEEE International Conference on Auto- matic Face & Gesture Recognition (FG 2018), pages 75–82,

work page 2018

[6] [7]

Talk-act: Enhance textural- awareness for 2d speaking avatar reenactment with diffusion model

Jiazhi Guan, Quanwei Yang, Kaisiyuan Wang, Hang Zhou, Shengyi He, Zhiliang Xu, Haocheng Feng, Errui Ding, Jing- dong Wang, Hongtao Xie, et al. Talk-act: Enhance textural- awareness for 2d speaking avatar reenactment with diffusion model. In SIGGRAPH Asia 2024 Conference Papers, pages 1–11, 2024. 1

work page 2024

[7] [8]

Ad-nerf: Audio driven neural ra- diance fields for talking head synthesis

Yudong Guo, Keyu Chen, Sen Liang, Yong-Jin Liu, Hujun Bao, and Juyong Zhang. Ad-nerf: Audio driven neural ra- diance fields for talking head synthesis. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5784–5794, 2021. 1

work page 2021

[8] [9]

Generative adversarial net- works (gans) for audio-visual speech recognition in artificial intelligence iot

Y He, KP Seng, and LM Ang. Generative adversarial net- works (gans) for audio-visual speech recognition in artificial intelligence iot. information 2023, 14, 575, 2023. 1

work page 2023