Pith. sign in

REVIEW 3 major objections 5 minor 3 cited by

GazeGaussian: High-Fidelity Gaze Redirection with 3D Gaussian Splatting

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GazeGaussian, a two-stream 3D Gaussian splatting model that explicitly rotates eye Gaussians according to the target gaze direction, achieves state-of-the-art gaze redirection accuracy and rendering speed (74 FPS), outperforming existing…

desk verdict A genuine and well-engineered 3DGS gaze redirection system whose novelty is real, but whose headline speed and SOTA accuracy claims need caveats about per-input finetuning and an ETH-XGaze-trained estimator appearing on both sides of the evaluation. read the letter →

arxiv 2411.12981 v2 pith:3TVXQBA3 submitted 2024-11-20 cs.CV

Xiaobao Wei , Peng Chen , Guangyu Li , Ming Lu , Hui Chen , Feng Tian This is my paper · ORCID
classification cs.CV
keywords gazeredirection3DGaussianSplattingestimationneuralrenderingheadavatarsynthesisface-eyedisentanglementdataaugmentationexpression-guided
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Gaze redirection changes the direction a person appears to look in a photo, and it is used to create training data for gaze estimators. The paper's central claim is that this can be done more accurately and far faster with 3D Gaussian splatting than with the neural radiance fields used by prior state-of-the-art methods. GazeGaussian represents the face and the eyes as two separate sets of 3D Gaussians, rotates the eye Gaussians explicitly according to the desired gaze direction, and uses an expression-guided renderer to keep each subject's identity intact. The authors report that this first 3DGS-based gaze redirection pipeline outperforms previous methods on redirection accuracy, image quality, identity preservation, and rendering speed across four datasets, reaching about 74 frames per second after a short per-image initialization. If correct, high-quality gaze redirection becomes a practical, real-time tool for improving gaze estimation on out-of-distribution data.

What carries the argument

The key mechanism is a two-stream 3D Gaussian Splatting (3DGS) model: a face-only stream deforms canonical Gaussians with MLPs conditioned on expression codes and head pose, while an eye stream introduces a Gaussian eye rotation representation that rotates eye Gaussians in canonical space according to the target gaze direction, with learned bias MLPs to absorb gaze-label noise. The transformed Gaussians from both streams are concatenated, rasterized into feature maps, and fed into an expression-guided neural renderer (EGNR) that injects subject-specific latent codes via cross-attention at the bottleneck. This explicit geometric control of the eye region is the feature that distinguishes the method from prior implicit feature-map-based approaches.

What would settle it

Evaluate redirected images from GazeGaussian and the baselines with a gaze estimator trained on a different dataset (e.g., MPIIFaceGaze or a synthetic rigged-eye dataset with known ground-truth gaze), and compare gaze angular errors; if GazeGaussian's advantage shrinks or disappears, the method is overfitting to the particular estimator used in training. Alternatively, compare the 3D rotation of the eye Gaussians against known ground-truth eye rotation from a calibrated multi-camera rig.

Watch

Extended reading notes

Core claim

The central discovery is that decoupling the head into a face-only Gaussian stream and an eye Gaussian stream, and explicitly rotating the eye Gaussians in 3D according to the target gaze vector, gives both more precise gaze redirection and more detailed face synthesis than implicit feature-map manipulation. The paper validates this on ETH-XGaze, ColumbiaGaze, MPIIFaceGaze, and GazeCapture, reporting lower gaze and head angular errors, higher SSIM, PSNR, LPIPS, FID, and identity-similarity scores than GazeNeRF, HeadNeRF, STED, and Gaussian Head Avatar, while rendering at 74 FPS. The two-stream design plus an expression-guided neural renderer is what makes both precise gaze control and cross-subject generalization possible.

Load-bearing premise

The evaluation assumes that the pre-trained gaze estimator used both as a training loss and as the accuracy metric (trained on ETH-XGaze) measures true gaze direction; if the model learns to exploit that estimator's blind spots rather than geometrically rotating the eyeball, the reported angular-error improvements would not reflect genuine redirection quality.

Editorial extensions

If this is right

  • Gaze redirection becomes fast enough (about 74 FPS after initialization) to serve as a real-time data augmentation engine for gaze estimators.
  • The face–eye disentanglement allows direct, controllable eye movement while preserving identity and expression, which is useful for avatar animation and content editing.
  • The expression-guided renderer generalizes to unseen subjects, addressing a known limitation of 3DGS head avatars that are typically person-specific.
  • The reported state-of-the-art results on four datasets suggest 3DGS is a practical alternative to NeRF for gaze-related synthesis and possibly other fine-grained face editing tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A portion of the reported accuracy gain may come from the model fitting the specific gaze estimator used both in the loss function and in the evaluation; testing with a held-out estimator trained on a different dataset would reveal whether the geometric eye rotation itself is more accurate.
  • Because the eye rotation is explicit and parameterized by gaze direction, the learned rotation parameters could be read off as a direct 3D gaze estimate, potentially making the method serve as a gaze estimator as well as a synthesis tool.
  • The per-input initialization takes about 30 seconds, so a fully amortized version that avoids per-image fine-tuning would be needed for truly interactive applications; otherwise the 74 FPS figure applies only after that per-input step.
  • The two-stream recipe could generalize to other region-specific face edits (e.g., blinking, mouth movements) by assigning each region its own Gaussian stream and control signal.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces GazeGaussian, a gaze redirection method built on 3D Gaussian Splatting (3DGS). It represents the face and eye regions with two separate streams of Gaussians, deforms the face branch with expression/head-pose conditioned MLPs, and models eye rotation via learned offsets and multiplications conditioned on gaze direction. An expression-guided neural renderer injects subject-specific latent codes to improve generalization. The method is evaluated on ETH-XGaze, ColumbiaGaze, MPIIFaceGaze, and GazeCapture, reporting improvements over baselines in gaze and head angular errors, image quality, identity preservation, and rendering speed (74 FPS). The paper claims to be the first 3DGS-based gaze redirection pipeline and to achieve state-of-the-art performance.

Significance. If the claims hold, GazeGaussian is a meaningful advance: it brings explicit, controllable 3DGS representations to gaze redirection, decouples eye and face deformations, and introduces a subject-conditional renderer that improves generalization across identities. The experimental scope is broad (four datasets), the ablations are systematic, and the code is promised. The main significance is tempered by a train/evaluation metric overlap: the gaze redirection loss and the reported gaze errors both rely on ETH-XGaze-trained estimator families, so the headline accuracy gains need independent confirmation before they can be relied upon.

major comments (3)
  1. [Sec. 4.4 (Eq. 12), Sec. 5.1, Supplementary Sec. 9] The gaze redirection loss in Eq. (12) minimizes the angular error between the rendered and ground-truth images as judged by a VGG-based gaze estimator fine-tuned on ETH-XGaze (Supplementary Sec. 9). All reported gaze and head errors in Tables 1 and 2 use a ResNet50 gaze estimator trained on ETH-XGaze. The model is therefore explicitly trained to minimize the same quantity family on which it is evaluated, and both estimators share the same training-domain bias. With the high capacity of 3DGS plus a neural renderer, the model may satisfy the estimator by producing estimator-friendly eye appearances rather than by truly rotating the eyeball to the target gaze. This affects the central SOTA accuracy claim. Please add an independent evaluation that breaks this loop: for example, report gaze errors using a held-out estimator trained on a different dataset, or use synthetic/geometric ground truth for eye orientation. Without such a check, the reported accuracy numbers are not sufficient to support the paper's main claim.
  2. [Abstract and Sec. 4.2.2 (Eqs. 4-5)] The abstract and contributions describe the method as enabling 'rigid eye rotation' based on the target gaze. However, Eq. (4) computes the eye Gaussian center as µe = Ee_μ(µe_0, τ) + Ge_μ(µe_0, φ) µe_0, which is an MLP-predicted offset plus a learned affine transformation, with no orthogonality or rotation constraint on Ge_μ. Similarly, Eq. (5) applies learned scales/rotations to other attributes without enforcing rigidity. The method is better described as a learned, gaze-conditioned affine deformation of eye Gaussians, which is still an explicit spatial control but not a rigid rotation. Please revise the terminology, or change the formulation to explicitly predict and apply a rotation (e.g., a quaternion) to support the 'rigid eye rotation' claim.
  3. [Abstract, Table 1, Supplementary Sec. 10] The abstract highlights 'rendering speed' and Table 1 reports 74 FPS for GazeGaussian. However, Supplementary Sec. 10 states that 'during inference, GazeGaussian fine-tunes on a single input image, taking approximately 30 seconds for fine-tuning and 0.2 seconds per image for generation.' The reported FPS therefore measures only the forward rendering pass, not the end-to-end inference cost required for a new input image. This materially affects the comparison with baselines, which do not all require per-input fine-tuning. Please report the full per-image inference time (fine-tuning plus rendering) and either remove or qualify the 'rendering speed' claim in the abstract and Table 1.
minor comments (5)
  1. [Sec. 5.3] The phrase 'To access the generalization capability' should be 'To assess the generalization capability'.
  2. [Table 3] The rows in Table 3 are shown only with checkmarks, making it hard to map each row to the variant described in Sec. 5.4. Please add explicit row labels (e.g., 'Vanilla', 'w/o Eye Rep.', 'w/o Expression-Guided', 'w/o Two-stream', 'Ours') to the table.
  3. [Sec. 5.1 and Sec. 4.4] Sec. 5.1 says the metrics use 'the same ResNet50-based estimator in GazeNeRF', while Sec. 4.4 and Supplementary Sec. 9 reveal the loss uses a different VGG-based estimator fine-tuned on ETH-XGaze. The main text should clearly distinguish the loss network and the evaluation network, and state their training domains, to avoid the impression that the metric is the exact loss network.
  4. [Sec. 4.2.2] Eqs. (4)-(5) introduce MLPs Ee_μ and Ge_μ but do not specify their inputs' dimensionality or architecture. A sentence noting that the gaze direction φ and latent code τ are concatenated and fed into these MLPs would improve reproducibility.
  5. [Sec. 5.4] The ablation text refers to 'the first row of the table and the first column of the visualizations' without naming the variant; making the row-to-variant mapping explicit in the table and figure captions would help readers follow the ablation discussion.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity found; the training and evaluation use different gaze estimators, and image-quality metrics are external.

full rationale

The paper is an empirical supervised-learning method, not a derivation, so classic derivation-equals-input circularity does not arise. The only plausible concern is that the gaze redirection loss (Eq. 12) uses a pre-trained VGG-based gaze estimator, while the evaluation metric (Sec. 5.1) uses a ResNet50-based estimator. These are distinct architectures, both trained on ETH-XGaze, so the model is not optimizing the exact evaluation quantity. This is a measurement-validation concern (estimator bias), not a circularity of the type where a prediction is equivalent to its input by construction. Image quality metrics (SSIM, PSNR, LPIPS, FID) and identity similarity are external to the training loss. There is no self-citation chain, no imported uniqueness theorem, and no renaming of a known result as a new contribution. The paper follows existing baselines (GazeNeRF, Gaussian Head Avatar) for standard preprocessing and initialization, which is normal practice and not circular. Therefore, no specific circular step can be exhibited, and the correct score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method is a supervised learning pipeline; its free parameters are hyperparameters chosen by hand. The central claim depends on the reliability of external tracking and gaze-estimation components, which are treated as trustworthy inputs.

free parameters (3)
  • Distance thresholds d1, d2 = d1=0.15, d2=0.25
    Empirically set to blend expression vs. head-pose influence on face Gaussians (Sec 4.2.1).
  • Loss weights lambda_SSIM, lambda_VGG, lambda_I, lambda_G = 0.1, 0.1, 1.0, 0.1
    Hand-selected in Sec 4.4 to balance image synthesis and gaze redirection losses.
  • Number of training epochs = 10 for SDF, 20 for full pipeline
    Training schedule from supplementary Sec 9; chosen for convergence, not derived.
assumptions (3)
  • domain assumption FLAME/3DMM tracking provides accurate identity and expression latent codes and camera poses for every training image across all four datasets.
    Sec 5.1 uses the tracking from [51]. If tracking is inaccurate, the face deformation field and expression-guided renderer receive faulty conditioning, harming cross-subject generalization.
  • domain assumption The pre-trained gaze estimators used in the redirection loss and evaluation respond to true geometric gaze direction rather than dataset-specific artifacts.
    Eq (12) and Sec 5.1 use a VGG estimator for training and a ResNet50 estimator for evaluation, both trained on ETH-XGaze. The entire redirection-accuracy evaluation rests on this premise.
  • domain assumption A neutral mesh extracted from an SDF field on the training set provides a valid coarse geometry for initializing Gaussians across different subjects.
    Sec 4.4 and Fig. 2. If the neutral mesh is biased toward training subjects, generalization to unseen identities would degrade.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GazeGaussian: High-Fidelity Gaze Redirection with 3D Gaussian Splatting." pith.science (2026). https://pith.science/paper/3TVXQBA3

@misc{pith2026241112981,
  author       = {Pith},
  title        = {Pith review of: GazeGaussian: High-Fidelity Gaze Redirection with 3D Gaussian Splatting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3TVXQBA3}},
  note         = {Machine review of arXiv:2411.12981}
}
read the original abstract

Gaze estimation encounters generalization challenges when dealing with out-of-distribution data. To address this problem, recent methods use neural radiance fields (NeRF) to generate augmented data. However, existing methods based on NeRF are computationally expensive and lack facial details. 3D Gaussian Splatting (3DGS) has become the prevailing representation of neural fields. While 3DGS has been extensively examined in head avatars, it faces challenges with accurate gaze control and generalization across different subjects. In this work, we propose GazeGaussian, the first high-fidelity gaze redirection method that uses a two-stream 3DGS model to represent the face and eye regions separately. Leveraging the unstructured nature of 3DGS, we develop a novel representation of the eye for rigid eye rotation based on the target gaze direction. To enable synthesis generalization across various subjects, we integrate an expression-guided module to inject subject-specific information into the neural renderer. Comprehensive experiments show that GazeGaussian outperforms existing methods in rendering speed, gaze redirection accuracy, and facial synthesis across multiple datasets. The code is available at: https://ucwxb.github.io/GazeGaussian.

Figures

Figures reproduced from arXiv: 2411.12981 by the authors.

Figure 1
Figure 1. Gaze redirection: Given an input image, GazeGaussian [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Pipeline of GazeGaussian. We initialize face-only and eye regions from a pre-trained neutral mesh. Using target expression codes, [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Within-dataset visualization: head images are generated from the ETH-XGaze test set using our GazeGaussian, GazeNeRF, and [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Cross-dataset visualization: head images are from the MPIIFaceGaze test set using our GazeGaussian, GazeNeRF, and Gaussian [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Qualitative ablation study on the ETH-XGaze dataset. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Improvement for gaze estimation accuracy. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Visualization of transformed two-stream Gaussians after [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 9
Figure 9. Figure 9: Example of a failure case. impacts. Thus, developing trustworthy techniques to discern real from fake content is crucial. We firmly oppose any unau￾thorized or harmful use of this technology and highlight the need to address ethical issues in its implementation. While …
Figure 8
Figure 8. Figure 8: Face morphing results on the ETH-XGaze dataset. [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 10
Figure 10. Figure 10: Additional qualitative ablation study on the ETH-XGaze dataset. [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Cross-dataset visualization on MPIIFaceGaze. [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: Cross-dataset visualization on ColumbiaGaze. [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]
Figure 13
Figure 13. Figure 13: Cross-dataset visualization on GazeCapture. [PITH_FULL_IMAGE:figures/full_fig_p016_13.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. GraphAvatar: Compact Head Avatars with GNN-Generated 3D Gaussians

    cs.CV 2024-12 conditional novelty 7.0 of 10

    Head avatars are produced by graph-neural-network-generated 3D Gaussians, cutting model size to about 10 MB and improving reported image quality over prior Gaussian-splatting avatars.

  2. OmniIndoor3D: Comprehensive Indoor 3D Reconstruction

    cs.CV 2025-05 conditional novelty 5.0 of 10

    OmniIndoor3D jointly optimizes appearance, geometry, and panoptic labels in a single set of 3D Gaussians initialized from RGB-D camera depth, reporting state-of-the-art numbers on ScanNet and ScanNet++.

  3. ManipDreamer: Boosting Robotic Manipulation World Model with Action Tree and Visual Guidance

    cs.RO 2025-04 reject novelty 3.0 of 10

    ManipDreamer conditions a robot-manipulation video diffusion model on action-tree instruction embeddings and multi-modal visual guidance, reporting modest gains over RoboDreamer that are undercut by evaluation inconsi...

Reference graph

Works this paper leans on

70 extracted references · 59 canonical work pages · cited by 3 Pith papers

  1. [1]

    Conversational gaze aversion for humanlike robots

    Sean Andrist, Xiang Zhi Tan, Michael Gleicher, and Bilge Mutlu. Conversational gaze aversion for humanlike robots. In Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction, pages 25–32, 2014. 1

    work page 2014

  2. [2]

    Diffusiontalker: Person- alization and acceleration for speech-driven 3d face diffuser

    Peng Chen, Xiaobao Wei, Ming Lu, Yitong Zhu, Naiming Yao, Xingyu Xiao, and Hui Chen. Diffusiontalker: Person- alization and acceleration for speech-driven 3d face diffuser. arXiv preprint arXiv:2311.16565, 2023. 2

    arXiv 2023

  3. [3]

    Gaze estimation using transformer

    Yihua Cheng and Feng Lu. Gaze estimation using transformer. In 2022 26th International Conference on Pattern Recognition (ICPR), pages 3341–3347. IEEE, 2022. 1

    work page 2022

  4. [4]

    Appearance-based gaze estimation with deep learning: A review and benchmark

    Yihua Cheng, Haofei Wang, Yiwei Bao, and Feng Lu. Appearance-based gaze estimation with deep learning: A review and benchmark. IEEE Transactions on Pattern Analy- sis and Machine Intelligence, 2024. 1

    work page 2024

  5. [5]

    Gaze manipulation for one-to-one teleconferencing

    Criminisi, Shotton, Blake, and Torr. Gaze manipulation for one-to-one teleconferencing. In Proceedings Ninth IEEE International Conference on Computer Vision , pages 191– 198 vol.1, 2003. 2

    work page 2003

  6. [6]

    Capture, learning, and synthesis of 3D speaking styles

    Daniel Cudeiro, Timo Bolkart, Cassidy Laidlaw, Anurag Ran- jan, and Michael Black. Capture, learning, and synthesis of 3D speaking styles. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 10101–10111,

    work page

  7. [7]

    Headgas: Real-time animatable head avatars via 3d gaussian splatting

    Helisa Dhamo, Yinyu Nie, Arthur Moreau, Jifei Song, Richard Shaw, Yiren Zhou, and Eduardo P ´erez-Pellitero. Headgas: Real-time animatable head avatars via 3d gaussian splatting. arXiv preprint arXiv:2312.02902, 2023. 2

    arXiv 2023

  8. [8]

    Faceformer: Speech-driven 3d facial animation with transformers

    Yingruo Fan, Zhaojiang Lin, Jun Saito, Wenping Wang, and Taku Komura. Faceformer: Speech-driven 3d facial animation with transformers. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 18770–18780, 2022. 2

    work page 2022

Show all 70 references
  1. [9]

    Dynamic neural radiance fields for monocular 4d facial avatar reconstruction

    Guy Gafni, Justus Thies, Michael Zollhofer, and Matthias Nießner. Dynamic neural radiance fields for monocular 4d facial avatar reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8649–8658, 2021. 2

    2021

  2. [10]

    Deepwarp: Photorealistic image resyn- thesis for gaze manipulation

    Yaroslav Ganin, Daniil Kononenko, Diana Sungatullina, and Victor Lempitsky. Deepwarp: Photorealistic image resyn- thesis for gaze manipulation. In Computer Vision – ECCV 2016, pages 311–326, Cham, 2016. Springer International Publishing. 1, 2

    2016

  3. [11]

    Gaze correction with a single webcam

    Dominik Giger, Jean-Charles Bazin, Claudia Kuster, Tiberiu Popa, and Markus Gross. Gaze correction with a single webcam. IEEE International Conference on Multimedia & Expo, 2014. 2

    2014

  4. [12]

    Neural head avatars from monocular rgb videos

    Philip-William Grassal, Malte Prinzler, Titus Leistner, Carsten Rother, Matthias Nießner, and Justus Thies. Neural head avatars from monocular rgb videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18653–18664, 2022. 1

    2022

  5. [13]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 5

    2016

  6. [14]

    Photo-realistic monocular gaze redirection using generative adversarial networks

    Zhe He, Adrian Spurr, Xucong Zhang, and Otmar Hilliges. Photo-realistic monocular gaze redirection using generative adversarial networks. In IEEE International Conference on Computer Vision (ICCV). IEEE, 2019. 2

    2019

  7. [16]

    Headnerf: A real-time nerf-based parametric head model

    Yang Hong, Bo Peng, Haiyao Xiao, Ligang Liu, and Juyong Zhang. Headnerf: A real-time nerf-based parametric head model. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 20374–20384,

  8. [17]

    S3gaussian: Self-supervised street gaussians for autonomous driving

    Nan Huang, Xiaobao Wei, Wenzhao Zheng, Pengju An, Ming Lu, Wei Zhan, Masayoshi Tomizuka, Kurt Keutzer, and Shanghang Zhang. S3gaussian: Self-supervised street gaussians for autonomous driving. arXiv preprint arXiv:2405.20323, 2024. 1

    2024 arXiv

  9. [18]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1, 2023. 1, 2, 3

    2023

  10. [19]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR, 2015. 1

    2015

  11. [20]

    Learning to look up: Realtime monocular gaze correction using machine learning

    Daniil Kononenko and Victor Lempitsky. Learning to look up: Realtime monocular gaze correction using machine learning. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4667–4675, 2015. 2

    2015

  12. [21]

    Eye tracking for everyone

    Kyle Krafka, Aditya Khosla, Petr Kellnhofer, Harini Kannan, Suchendra Bhandarkar, Wojciech Matusik, and Antonio Tor- ralba. Eye tracking for everyone. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2176–2184, 2016. 5, 1

    2016

  13. [22]

    Gaze correction for home video conferencing

    Claudia Kuster, Tiberiu Popa, Jean-Charles Bazin, Craig Gots- man, and Markus Gross. Gaze correction for home video conferencing. ACM Trans. Graph., 31(6), 2012. 2

    2012

  14. [23]

    Eyenerf: a hy- brid representation for photorealistic synthesis, animation and relighting of human eyes

    Gengyan Li, Abhimitra Meka, Franziska Mueller, Marcel C Buehler, Otmar Hilliges, and Thabo Beeler. Eyenerf: a hy- brid representation for photorealistic synthesis, animation and relighting of human eyes. ACM Transactions on Graphics (TOG), 41(4):1–16, 2022. 2

    2022

  15. [24]

    Learning a model of facial shape and expression from 4d scans

    Tianye Li, Timo Bolkart, Michael J Black, Hao Li, and Javier Romero. Learning a model of facial shape and expression from 4d scans. ACM Trans. Graph., 36(6):194–1, 2017. 2

    2017

  16. [25]

    Scaffold-gs: Structured 3d gaussians for view-adaptive rendering

    Tao Lu, Mulin Yu, Linning Xu, Yuanbo Xiangli, Limin Wang, Dahua Lin, and Bo Dai. Scaffold-gs: Structured 3d gaussians for view-adaptive rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 20654–20664, 2024. 1

    2024

  17. [26]

    Gaussianhair: Hair modeling and rendering with light-aware gaussians

    Haimin Luo, Min Ouyang, Zijun Zhao, Suyi Jiang, Longwen Zhang, Qixuan Zhang, Wei Yang, Lan Xu, and Jingyi Yu. Gaussianhair: Hair modeling and rendering with light-aware gaussians. arXiv preprint arXiv:2402.10483, 2024. 3

    2024 arXiv

  18. [27]

    Eye gaze tracking based driver monitoring sys- tem

    Annu George Mavely, JE Judith, PA Sahal, and Steffy Ann Kuruvilla. Eye gaze tracking based driver monitoring sys- tem. In 2017 IEEE international conference on circuits and systems (ICCS), pages 364–367. IEEE, 2017. 1

    2017

  19. [28]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthe- sis, 2020. 1, 2

    2020

  20. [29]

    Optimizing vr for all users through adap- tive focus displays

    Nitish Padmanaban, Robert Konrad, Emily A Cooper, and Gordon Wetzstein. Optimizing vr for all users through adap- tive focus displays. In ACM SIGGRAPH 2017 Talks, pages 1–2, 2017. 1

    2017

  21. [30]

    Few-shot adaptive gaze estimation

    Seonwook Park, Shalini De Mello, Pavlo Molchanov, Umar Iqbal, Otmar Hilliges, and Jan Kautz. Few-shot adaptive gaze estimation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9368–9377, 2019. 2

    2019

  22. [31]

    Selftalk: A self- supervised commutative training diagram to comprehend 3d talking faces

    Ziqiao Peng, Yihao Luo, Yue Shi, Hao Xu, Xiangyu Zhu, Hongyan Liu, Jun He, and Zhaoxin Fan. Selftalk: A self- supervised commutative training diagram to comprehend 3d talking faces. In Proceedings of the 31st ACM International Conference on Multimedia, pages 5292–5301, 2023. 2

    2023

  23. [32]

    Emotalk: Speech-driven emotional disentanglement for 3d face anima- tion

    Ziqiao Peng, Haoyu Wu, Zhenbo Song, Hao Xu, Xiangyu Zhu, Jun He, Hongyan Liu, and Zhaoxin Fan. Emotalk: Speech-driven emotional disentanglement for 3d face anima- tion. In Proceedings of the IEEE/CVF International Confer- ence on Computer Vision, pages 20687–20697, 2023. 2

    2023

  24. [33]

    Gaussiana- vatars: Photorealistic head avatars with rigged 3d gaussians

    Shenhan Qian, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Simon Giebenhain, and Matthias Nießner. Gaussiana- vatars: Photorealistic head avatars with rigged 3d gaussians. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20299–...

    2024

  25. [34]

    Eye gaze correction with a single webcam based on eye-replacement

    Yalun Qin, Kuo-Chin Lien, Matthew Turk, and Tobias H¨ollerer. Eye gaze correction with a single webcam based on eye-replacement. In Advances in Visual Computing, pages 599–609, Cham, 2015. Springer International Publishing. 2

    2015

  26. [35]

    Generating 3d faces using convolutional mesh au- toencoders

    Anurag Ranjan, Timo Bolkart, Soubhik Sanyal, and Michael J Black. Generating 3d faces using convolutional mesh au- toencoders. In Proceedings of the European conference on computer vision (ECCV), pages 704–720, 2018. 2

    2018

  27. [36]

    Gazenerf: 3d-aware gaze redirection with neural radiance fields

    Alessandro Ruzzi, Xiangwei Shi, Xi Wang, Gengyan Li, Shalini De Mello, Hyung Jin Chang, Xucong Zhang, and Otmar Hilliges. Gazenerf: 3d-aware gaze redirection with neural radiance fields. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 9676...

    2023

  28. [37]

    Deep marching tetrahedra: a hybrid represen- tation for high-resolution 3d shape synthesis

    Tianchang Shen, Jun Gao, Kangxue Yin, Ming-Yu Liu, and Sanja Fidler. Deep marching tetrahedra: a hybrid represen- tation for high-resolution 3d shape synthesis. Advances in Neural Information Processing Systems, 34:6087–6101, 2021. 3

    2021

  29. [38]

    Eyeopener: Editing eyes in the wild

    Zhixin Shu, Eli Shechtman, Dimitris Samaras, and Sunil Hadap. Eyeopener: Editing eyes in the wild. ACM Trans. Graph., 36(1), 2016. 2

    2016

  30. [39]

    Smith, Q

    B.A. Smith, Q. Yin, S.K. Feiner, and S.K. Nayar. Gaze Locking: Passive Eye Contact Detection for Human?Object Interaction. In ACM Symposium on User Interface Software and Technology (UIST), pages 271–280, 2013. 5, 1

    2013

  31. [40]

    Learning-by-synthesis for appearance-based 3d gaze estima- tion

    Yusuke Sugano, Yasuyuki Matsushita, and Yoichi Sato. Learning-by-synthesis for appearance-based 3d gaze estima- tion. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1821–1828, 2014. 5

    2014

  32. [41]

    Nonlinear 3d face morphable model

    Luan Tran and Xiaoming Liu. Nonlinear 3d face morphable model. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7346–7355, 2018. 2

    2018

  33. [42]

    High-fidelity eye animatable neural radi- ance fields for human face

    Hengfei Wang, Zhongqun Zhang, Yihua Cheng, and Hyung Jin Chang. High-fidelity eye animatable neural radi- ance fields for human face. arXiv preprint arXiv:2308.00773,

    arXiv

  34. [43]

    Facex-zoo: A pytorch toolbox for face recognition

    Jun Wang, Yinglu Liu, Yibo Hu, Hailin Shi, and Tao Mei. Facex-zoo: A pytorch toolbox for face recognition. In Pro- ceedings of the 29th ACM International Conference on Multi- media, pages 3779–3782, 2021. 5

    2021

  35. [44]

    Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction

    Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. arXiv preprint arXiv:2106.10689, 2021. 1

    2021 arXiv

  36. [45]

    Plgs: Robust panoptic lifting with 3d gaussian splatting

    Yu Wang, Xiaobao Wei, Ming Lu, and Guoliang Kang. Plgs: Robust panoptic lifting with 3d gaussian splatting. arXiv preprint arXiv:2410.17505, 2024. 1

    2024 arXiv

  37. [46]

    Nto3d: Neural target object 3d reconstruction with segment anything

    Xiaobao Wei, Renrui Zhang, Jiarui Wu, Jiaming Liu, Ming Lu, Yandong Guo, and Shanghang Zhang. Nto3d: Neural target object 3d reconstruction with segment anything. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20352–20362, 2024. 1

    2024

  38. [47]

    Gazedirector: Fully articulated eye gaze redirection in video, 2017

    Erroll Wood, Tadas Baltrusaitis, Louis-Philippe Morency, Peter Robinson, and Andreas Bulling. Gazedirector: Fully articulated eye gaze redirection in video, 2017. 2

    2017

  39. [48]

    Controllable continuous gaze redirection

    Weihao Xia, Yujiu Yang, Jing-Hao Xue, and Wensen Feng. Controllable continuous gaze redirection. In Proceedings of the 28th ACM International Conference on Multimedia, pages 1782–1790, 2020. 2

    2020

  40. [49]

    Flashavatar: High-fidelity head avatar with efficient gaussian embedding

    Jun Xiang, Xuan Gao, Yudong Guo, and Juyong Zhang. Flashavatar: High-fidelity head avatar with efficient gaussian embedding. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 1, 2

    2024

  41. [50]

    Learning a general- ized gaze estimator from gaze-consistent feature

    Mingjie Xu, Haofei Wang, and Feng Lu. Learning a general- ized gaze estimator from gaze-consistent feature. In Proceed- ings of the AAAI conference on artificial intelligence, pages 3027–3035, 2023. 1

    2023

  42. [51]

    Gaussian head avatar: Ultra high-fidelity head avatar via dynamic gaussians

    Yuelang Xu, Benwang Chen, Zhe Li, Hongwen Zhang, Lizhen Wang, Zerong Zheng, and Yebin Liu. Gaussian head avatar: Ultra high-fidelity head avatar via dynamic gaussians. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 1, 2, 3, 4, 5

    2024

  43. [52]

    Nerf-gaze: A head-eye redirection parametric model for gaze estimation

    Pengwei Yin, Jingjing Wang, Jiawu Dai, and Xiaojun Wu. Nerf-gaze: A head-eye redirection parametric model for gaze estimation. In ICASSP 2024-2024 IEEE International Confer- ence on Acoustics, Speech and Signal Processing (ICASSP), pages 2760–2764. IEEE, 2024. 1, 2, 5

    2024

  44. [53]

    Unsupervised representation learning for gaze estimation

    Yu Yu and Jean-Marc Odobez. Unsupervised representation learning for gaze estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7314–7324, 2020. 1

    2020

  45. [54]

    Gazeonce: Real-time multi-person gaze estimation

    Mingfang Zhang, Yunfei Liu, and Feng Lu. Gazeonce: Real-time multi-person gaze estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4197–4206, 2022. 1

    2022

  46. [55]

    Appearance-based gaze estimation in the wild

    Xucong Zhang, Yusuke Sugano, Mario Fritz, and Andreas Bulling. Appearance-based gaze estimation in the wild. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4511–4520, 2015. 5, 1

    2015

  47. [56]

    It’s written all over your face: Full-face appearance- based gaze estimation

    Xucong Zhang, Yusuke Sugano, Mario Fritz, and Andreas Bulling. It’s written all over your face: Full-face appearance- based gaze estimation. In Computer Vision and Pattern Recog- nition Workshops (CVPRW), 2017 IEEE Conference on, pages 2299–2308. IEEE, 2017. 5, 1

    2017

  48. [57]

    Revis- iting data normalization for appearance-based gaze estimation

    Xucong Zhang, Yusuke Sugano, and Andreas Bulling. Revis- iting data normalization for appearance-based gaze estimation. In Proc. International Symposium on Eye Tracking Research and Applications (ETRA), pages 12:1–12:9, 2018. 1, 5

    2018

  49. [58]

    Eth-xgaze: A large scale dataset for gaze estimation under extreme head pose and gaze variation

    Xucong Zhang, Seonwook Park, Thabo Beeler, Derek Bradley, Siyu Tang, and Otmar Hilliges. Eth-xgaze: A large scale dataset for gaze estimation under extreme head pose and gaze variation. In European Conference on Computer Vision (ECCV), 2020. 5, 1

    2020

  50. [59]

    Self-learning transformations for improving gaze and head redirection

    Yufeng Zheng, Seonwook Park, Xucong Zhang, Shalini De Mello, and Otmar Hilliges. Self-learning transformations for improving gaze and head redirection. In Neural Information Processing Systems (NeurIPS), 2020. 2, 5

    2020

  51. [60]

    Pointavatar: Deformable point- based head avatars from videos

    Yufeng Zheng, Wang Yifan, Gordon Wetzstein, Michael J Black, and Otmar Hilliges. Pointavatar: Deformable point- based head avatars from videos. In Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition, pages 21057–21067, 2023. 1, 2

    2023

  52. [61]

    Towards metrical reconstruction of human faces

    Wojciech Zielonka, Timo Bolkart, and Justus Thies. Towards metrical reconstruction of human faces. In European con- ference on computer vision, pages 250–269. Springer, 2022. 2

    2022

  53. [62]

    Instant volumetric head avatars

    Wojciech Zielonka, Timo Bolkart, and Justus Thies. Instant volumetric head avatars. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4574–4584, 2023. 1, 2

    2023

  54. [63]

    Using modified bisenet for face parsing in pytorch,

    zllrunning. Using modified bisenet for face parsing in pytorch,

  55. [65]

    Overview The supplementary material encompasses the subsequent components. • Video for continuous gaze redirection • Implementation details • Dataset and pre-processing details • Supplementary experiments – Ablation study on cross-dataset – Comparison with the FLAME-based meth...

  56. [66]

    continuous gaze redirection.mp4

    Video for continuous gaze redirection Please refer to the video“continuous gaze redirection.mp4” in the supplementary material for continuous gaze redirec- tion results on the ETH-Xgaze. The side-by-side visualiza- tion showcases smooth transitions and high-quality novel gaze ...

  57. [67]

    Implementation details We use the Adam optimizer [19], with a learning rate that follows an exponential decay schedule, starting at 1 × 10−4. We use the VGG-based network pre-trained on ImageNet, as provided by the GazeNeRF [36] implementation, and fine- tune it on the ETH-XGa...

  58. [68]

    ETH-XGaze [58] is a large-scale gaze estimation dataset featuring high-resolution images across a wide range of head poses and gaze directions

    Dataset and pre-processing details Following the baseline GazeNeRF [36], all experiments are conducted on four widely used datasets. ETH-XGaze [58] is a large-scale gaze estimation dataset featuring high-resolution images across a wide range of head poses and gaze directions. ...

  59. [69]

    Supplementary experiments 11.1. Ablation study on cross-dataset To further validate the effectiveness of each proposed com- ponent, we conduct an ablation study on the cross-dataset evaluation to assess the generalization capability of our full pipeline. As shown in Tab. 4, th...

  60. [70]

    Supplementary visualization 12.1. Visualization for transformed Gaussians To demonstrate the advantages of GazeGaussian’s explicit control of head pose and gaze direction for head and eye regions, we visualize the Gaussians after deformation from the canonical space. As shown ...

  61. [71]

    Example of a failure case

    Ethical considerations and limitations Our approach allows for the creation of lifelike portrait videos that may be exploited to spread misinformation, sway public opinion, and erode trust in media, with grave societal GazeGaussianTarget Image GazeGaussianTarget Image Figure 9...

  62. [2019]

    5, 1 GazeGaussian: High-Fidelity Gaze Redirection with 3D Gaussian Splatting Supplementary Material

    https://github.com/zllrunning/face- parsing.PyTorch. 5, 1 GazeGaussian: High-Fidelity Gaze Redirection with 3D Gaussian Splatting Supplementary Material

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.