Pith. sign in

REVIEW 5 cited by

GaussianTalker: Real-Time High-Fidelity Talking Head Synthesis with Audio-Driven 3D Gaussian Splatting

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2404.16012 v2 pith:NACEVQ7I submitted 2024-04-24 cs.CV cs.MM

Kyusun Cho , Joungbin Lee , Heeji Yoon , Yeobin Hong , Jaehoon Ko , Sangjun Ahn , Seungryong Kim This is my paper
classification cs.CVcs.MM
keywords gaussiantalkergaussianaudiopreviousrenderingattributesfeaturefeatures
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We propose GaussianTalker, a novel framework for real-time generation of pose-controllable talking heads. It leverages the fast rendering capabilities of 3D Gaussian Splatting (3DGS) while addressing the challenges of directly controlling 3DGS with speech audio. GaussianTalker constructs a canonical 3DGS representation of the head and deforms it in sync with the audio. A key insight is to encode the 3D Gaussian attributes into a shared implicit feature representation, where it is merged with audio features to manipulate each Gaussian attribute. This design exploits the spatial-aware features and enforces interactions between neighboring points. The feature embeddings are then fed to a spatial-audio attention module, which predicts frame-wise offsets for the attributes of each Gaussian. It is more stable than previous concatenation or multiplication approaches for manipulating the numerous Gaussians and their intricate parameters. Experimental results showcase GaussianTalker's superiority in facial fidelity, lip synchronization accuracy, and rendering speed compared to previous methods. Specifically, GaussianTalker achieves a remarkable rendering speed up to 120 FPS, surpassing previous benchmarks. Our code is made available at https://github.com/KU-CVLAB/GaussianTalker/ .

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. RigorBench: Benchmarking Engineering Process Discipline in Autonomous AI Coding Agents

    cs.SE 2026-06 unverdicted novelty 8.0 of 10

    RigorBench is the first benchmark for process discipline in autonomous AI coding agents, reporting 41% higher process quality scores and 17% higher outcome correctness when agents follow structured engineering practices.

  2. RigorBench: Benchmarking Engineering Process Discipline in Autonomous AI Coding Agents

    cs.SE 2026-06 unverdicted novelty 7.0 of 10

    RigorBench evaluates AI coding agents on process discipline via five pillars and reports 41% higher process scores and 17% better outcome correctness with structured approaches on 30 tasks.

  3. GeoAvatar: Adaptive Geometrical Gaussian Splatting for 3D Head Avatar

    cs.GR 2025-07 conditional novelty 7.0 of 10

    GeoAvatar improves 3D head avatar quality by adaptively regulating Gaussian offsets per facial region, adding a detailed mouth structure with part-wise deformation, and releasing a new expressive monocular dataset, Dy...

  4. SDTalk: Structured Facial Priors and Dual-Branch Motion Fields for Generalizable Gaussian Talking Head Synthesis

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    SDTalk proposes a generalizable one-shot 3DGS talking head method that uses structured facial priors for complete reconstruction and dual-branch motion fields for dynamics, outperforming prior identity-specific approaches.

  5. Human Motion Video Generation: A Survey

    cs.CV 2025-09 conditional novelty 4.0 of 10

    A comprehensive survey with a five-phase pipeline model for human motion video generation, covering over 200 papers and adding a new benchmark comparison of nine pose-guided methods.

Pith tools