Pith. sign in

REVIEW 3 major objections 6 minor 71 references

Low-Rank Head Avatar Personalization with Registers

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Personalizing a generic head-avatar model with low-rank adaptation misses fine identity details, and this paper argues a 3D register module that guides the LoRA training signal closes that gap without any inference-time cost.

desk verdict A training-only 3D register module for LoRA-based head avatar personalization—plausible and decently supported, but the evaluation needs more rigor before I'd fully trust the numbers. read the letter →

arxiv 2506.01935 v1 pith:JI44TYDD submitted 2025-06-02 cs.CV

Sai Tanmay Reddy Chakkera , Aggelina Chatziagapi , Md Moniruzzaman , Chen-Ping Yu , Yi-Hsuan Tsai , Dimitris Samaras This is my paper
classification cs.CV
keywords headavatarpersonalizationlow-rankadaptationLoRA3DregistersDINOv2features3DMMfacialdetailpreservationtalkinggeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Generic head-avatar models trained on many identities produce clean but generic faces, losing the wrinkles, tattoos, skin texture, and other high-frequency marks that make a person recognizable. The paper claims that adding a small trainable Register Module during low-rank (LoRA) adaptation teaches the model where to look in the source image's DINOv2 features, so those identity-specific details survive personalization. The module is discarded at inference, so the adapted model runs at the same speed and parameter budget as the generic model. To test this, the authors collect a new dataset, RareFace-50, of talking videos of people with distinctive facial details, and report better LPIPS and identity-preservation scores than vanilla LoRA, meta-learning, and the generic baseline.

What carries the argument

The Register Module. It is a learnable 3D feature space: D=512 dimensional embeddings are rigged to the vertices of a FLAME 3DMM mesh, and for each training frame the visible vertices are projected with the tracked camera pose onto the DINOv2 feature plane (296×296 pixels, 256 channels). Unprojected pixels inside the face contour are filled by inverse-distance interpolation over the k=11 nearest projected vertices, background pixels by a learned embedding, and the dense map is processed by two small CNN encoders (Eproc-1 and Eproc-2). The resulting feature map is added to the source image's dense DINOv2 features and trained with an MSE loss toward the driving frame's DINOv2 features plus a pairwise-cosine regularizer on the vertex embeddings. This amplifies the face region and suppresses background in the DINOv2 feature space, giving the LoRA layers a sharper learning signal for identity-specific details during adaptation; the module is removed at inference.

What would settle it

On a handful of identities, replace the tracked 3DMM fit with ground-truth mesh pose and expression (available from synthetic renders or manual fits) and rerun the same adaptation; if the register's improvement over vanilla LoRA disappears or shrinks sharply when tracking is perfect, then the reported gains are correcting tracker noise rather than storing identity-specific features.

Watch

Extended reading notes

Core claim

The paper's central claim is that LoRA fine-tuning of a generic head-avatar model fails to preserve an unseen person's high-frequency identity details, and that this can be fixed by a training-only 3D Register Module that conditions the source image's DINOv2 features on the driving frame's 3DMM geometry. The module rigs learnable embeddings to the vertices of a FLAME mesh, projects the visible vertices into the feature plane using the tracked camera pose, and adds the interpolated feature map to the source DINOv2 features before the reconstruction branch sees them. The authors report that this guidance makes LoRA capture wrinkles, tattoos, skin texture, and blemishes that vanilla LoRA, LoRA with meta-learning, and the GAGAvatar baseline miss, on both the existing VFHQ test set and the new RareFace-50 dataset. The Register Module and its losses are used only during adaptation; at inference the LoRA offsets are merged into the base model and the module is discarded, so the adapted avatar runs at the generic model's speed with roughly 23.2M additional training parameters.

Load-bearing premise

The entire method leans on the 3DMM tracker's pose, shape, and visibility estimates being accurate enough that the projected register features land on the true identity details of the source face; if the fit is noisy or fails on extreme expressions or views, the adaptation signal misleads LoRA.

Editorial extensions

If this is right

  • An unseen identity can be personalized from a single source image with roughly 35 minutes of adaptation, then animated in real time with the same resource cost as the generic model.
  • On both VFHQ Test and RareFace-50, the adapted model reports lower LPIPS and better identity preservation (ACD) than vanilla LoRA, meta-learned LoRA, and the GAGAvatar baseline.
  • The Register Module's design is not tied to GAGAvatar's renderer; it operates on DINOv2 dense features, so any avatar model built on DINOv2 features could use the same adaptation recipe.
  • Adaptation remains data-efficient: reducing the adaptation video from 4 seconds to 1 second degrades quality only slightly, so the method works from very short clips.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the mechanism is as claimed, the projection and interpolation could be replaced by a learned pose-conditioned spatial mask, which would extend the register idea beyond faces to any object with a parametric model.
  • The same training-only register recipe should transfer to other generative personalization settings where a generic prior must retain rare high-frequency attributes, such as hand texture, clothing detail, or full-body identity.
  • A direct test the paper leaves open is to keep the Register Module at inference: if feeding freg instead of the raw DINOv2 features recovers the reported side/back-view failures, then the module's value is not fully absorbed into the merged LoRA weights.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a Register Module for low-rank personalization of a generic head avatar model (GAGAvatar). The module rigs learnable embeddings to 3DMM vertices, projects visible vertices onto the camera plane using the driving image's pose, interpolates within an alpha-shape, and adds the resulting feature map to the source image's DINOv2 features during adaptation. The adapted features are supervised by an MSE loss against the driving image's DINOv2 features and a regularization loss on the embeddings. At inference, the Register Module is discarded and only the merged LoRA weights are used. The authors collect a new dataset, RareFace-50, of 50 identities with distinctive facial details such as wrinkles and tattoos, and report quantitative improvements in LPIPS and ACD over GAGAvatar, vanilla LoRA, and a meta-learning baseline, together with ablations and a user study.

Significance. The 3D register mechanism is a novel idea for personalizing generic head avatars, and the motivation is clear. The method is parameter-efficient (23.2M adaptation parameters, about 11% of GAGAvatar), adds no inference overhead, and the paper includes a new dataset, ablations, and a user study. If the reported improvements are robust, the method would be a practical recipe for adapting generic avatars to unseen identities. However, the current evaluation leaves open the possibility that the gains come from regularization or the auxiliary losses rather than from the claimed 3D feature-space mechanism, and the LPIPS metric protocol is insufficiently specified.

major comments (3)
  1. [Sec. 4.4, App. B.5] The LPIPS evaluation is performed on 'challenging patches' selected from predicted frames, but the main text does not state how these patches are chosen, whether selection is based on the predicted or source image, or whether the identical patch set is used for all methods. This is load-bearing because Tables 1 and 2 report these LPIPS values as the primary quantitative evidence of visual quality. Please specify the patch selection rule, report full-frame and random-crop LPIPS as a sanity check, and provide per-identity confidence intervals or significance tests.
  2. [Sec. 3.3, Eq. (7)-(8), App. B.4.1] The Register Module is active only during adaptation; at inference, the pipeline directly passes f_src_dense to the reconstruction branch. The paper does not isolate whether the observed improvement comes from the 3D feature-space mechanism or from the auxiliary losses L_feat and L_reg acting as a regularizer on LoRA training. The ablations in Table 1 replace the module with Gaussian noise or learnable embeddings but keep the same loss structure, so they do not rule out a regularization effect. Please add a control in which the 3D projection and alpha-shape interpolation are replaced by a non-3D feature map while keeping L_feat and L_reg, and compare the resulting merged LoRA weights at inference. If the improvement is not explained by the 3D mechanism, the paper's central mechanistic claim should be revised accordingly.
  3. [Table 2] Table 2 shows that vanilla LoRA on VFHQ Test (LPIPS 0.2666) is worse than the unadapted GAGAvatar baseline (LPIPS 0.2540). The paper does not comment on this degradation, yet it is directly relevant to the claim that the Register Module 'enhances' LoRA. The improvement of the full method over GAGAvatar could be interpreted as preventing LoRA-induced degradation rather than adding positive identity-specific detail. Please report results for adaptation without LoRA (only the Register Module), with LoRA plus the auxiliary losses but without the 3D projection, and with LoRA alone under identical training budgets, along with error bars or significance tests across the 50 identities.
minor comments (6)
  1. [Eq. (9)] In Eq. (9), the expression pcos(e) is not fully defined; please write the summation explicitly over ordered pairs i ≠ j so that the subtraction of n(V) is transparent.
  2. [Sec. 3.2] The sentence 'we use the same rank r = 32 for all comparisons' is qualified by 'except ablations described in the supplementary material'; please specify which ablations use a different rank and what rank they use.
  3. [Sec. 4.1] Please clarify whether the adaptation uses one or both of the two videos per identity and whether the evaluation is performed on the same videos used for adaptation, as this affects the interpretation of the results.
  4. [Sec. 4.4, App. B.4.2] The paper implements a Reptile-based meta-learning baseline but refers to it as MetaPortrait [60]; please clarify which components of the original MetaPortrait algorithm are retained and state explicitly how the comparison is made fair.
  5. [Fig. 2, Fig. 3] The figures are dense and the text is small; please consider enlarging the figures and using vector graphics for readability.
  6. [Sec. 1, Limitations] The statement that the method 'faithfully captures unseen faces' is stronger than the evidence, given the acknowledged limitations for extreme side or back views; please soften the claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the Register Module is an auxiliary training component evaluated by external rendering metrics.

full rationale

The paper's derivation chain is not circular. LoRA adaptation follows the standard decomposition W_adapt = W + BA (Eq. 1); the Register Module forward pass (Eqs. 2-7) constructs f_reg from 3DMM-projected vertex embeddings plus source DINOv2 features, and Eq. 8 (L_feat = ||f_dri_dense - f_reg||^2) is an auxiliary training loss, not a restatement of the LPIPS/ACD evaluation metrics, which are computed on rendered frames with external perceptual and ArcFace embeddings. The fact that the module is discarded at inference ('During inference, we skip the Register Module and directly pass f_src_dense') raises an empirical attribution question about whether the training-time benefit transfers through LoRA gradients, and Table 2 shows vanilla LoRA can degrade GAGAvatar, so the module might act partly as a regularizer; this is a causal-isolation weakness, not a circular reduction. The self-citations in related work are not load-bearing for the central claim, and Appendix F.1's admission that 3DMM fitting noise can propagate is a robustness limitation, not evidence of circularity. No equation or fitted parameter is re-labeled as a prediction; the comparisons against noise and learnable-embedding ablations are external empirical checks.

Assumptions & free parameters 6 free parameters · 4 assumptions · 1 invented entities

The central claim rests on the availability of accurate 3DMM tracking, the informativeness of DINOv2 features for identity details, and the hypothesis that a training-only feature augmentation transfers to merged LoRA weights. These are domain assumptions rather than mathematical axioms; there are no new physical entities, and the free parameters are hand-chosen hyperparameters (rank, neighbors, embedding dimension, loss weights).

free parameters (6)
  • LoRA rank r = 32
    Chosen for all experiments; not swept. The central comparison uses this fixed rank.
  • Number of nearest neighbors k = 11
    Used for inverse-distance interpolation of projected 3DMM vertex features; chosen by hand.
  • Embedding dimension D = 512
    Dimensionality of per-vertex embeddings in the Register Module; chosen by hand.
  • Feature loss weight lambda_feat = 2
    Weight for L_feat in L_register; chosen by hand.
  • Regularization loss weight lambda_reg = 20
    Weight for L_reg in L_register; chosen by hand.
  • Alpha-shape parameter alpha = 0.065
    Controls contour polygon of projected vertices; fixed value in preprocessing.
assumptions (4)
  • domain assumption DINOv2 features encode identity-specific high-frequency details needed for avatar personalization.
    The method augments source DINOv2 features with register features to improve LoRA. If DINOv2 features lack these details, the module cannot recover them. Invoked in Sec 3.3 and Fig 4.
  • domain assumption 3DMM fitting provides accurate mesh, pose, and camera parameters for computing visible vertices and projection.
    Equations 2-5 project 3DMM vertices to the camera plane; noisy or inaccurate fitting would corrupt the register feature map. The authors acknowledge this in Appendix F.1.
  • domain assumption Supervising the register output with the driving image's DINOv2 features is a valid training signal for identity detail preservation.
    The L_feat loss (Eq. 8) pulls the register output toward DINOv2 features of the driving frame; this is an auxiliary task assumed to improve the LoRA update.
  • ad hoc to paper Adding the register feature map to source DINOv2 features during adaptation produces LoRA weights that generalize at inference without the module.
    The Register Module is used only during training and discarded at inference; the claim that the benefit persists in LoRA weights alone is an unproven architectural hypothesis (Sec 3.3).
invented entities (1)
  • Register Module (3D feature-space registers)
    purpose: Stores and repurposes identity-specific facial details in a learnable 3D feature space during adaptation, improving LoRA learning signals.
    A new architectural component; it is not a physical entity and has no external falsifiable prediction outside the paper's own evaluations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Low-Rank Head Avatar Personalization with Registers." pith.science (2026). https://pith.science/paper/JI44TYDD

@misc{pith2026250601935,
  author       = {Pith},
  title        = {Pith review of: Low-Rank Head Avatar Personalization with Registers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JI44TYDD}},
  note         = {Machine review of arXiv:2506.01935}
}
read the original abstract

We introduce a novel method for low-rank personalization of a generic model for head avatar generation. Prior work proposes generic models that achieve high-quality face animation by leveraging large-scale datasets of multiple identities. However, such generic models usually fail to synthesize unique identity-specific details, since they learn a general domain prior. To adapt to specific subjects, we find that it is still challenging to capture high-frequency facial details via popular solutions like low-rank adaptation (LoRA). This motivates us to propose a specific architecture, a Register Module, that enhances the performance of LoRA, while requiring only a small number of parameters to adapt to an unseen identity. Our module is applied to intermediate features of a pre-trained model, storing and re-purposing information in a learnable 3D feature space. To demonstrate the efficacy of our personalization method, we collect a dataset of talking videos of individuals with distinctive facial details, such as wrinkles and tattoos. Our approach faithfully captures unseen faces, outperforming existing methods quantitatively and qualitatively. We will release the code, models, and dataset to the public.

Figures

Figures reproduced from arXiv: 2506.01935 by the authors.

Figure 1
Figure 1. Our method personalizes and adapts a generic head avatar model using LoRA, while [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Illustration of our Register Module in a generic avatar animation model. During adaptation, [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Illustration of our Register Module. We propose a Register Module that learns features [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Visualization of learned features by the Register Module on our RareFace-50 dataset. We [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Ablation results on the VFHQ Test dataset. We observe that our method performs better in [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Personalized head avatar generation on VFHQ Test (row 1, 2, and 3) and RareFace-50 [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: User Study. Preference (%) in terms of identity preservation, and visual quality, comparing LoRA [26], and our method. We conduct a user study to qualitatively and subjectively compare our method against LoRA (see Sec. E for details). The results of our user study are …
Figure 8
Figure 8. Figure 8: We show facial details that are well captured by our method with Register Module, such [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Additional results of personalized head avatar generation on VFHQ Test. Please zoom in [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Additional results of personalized head avatar generation on our RareFace-50 dataset. [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: Visualization of learned features by the Register Module on our RareFace-50 dataset. We [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]
Figure 12
Figure 12. Figure 12: Visualization of learned features by the Register Module on our RareFace-50 dataset. We [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]
Figure 13
Figure 13. Figure 13: Visualization of learned features by the Register Module on our RareFace-50 dataset. We [PITH_FULL_IMAGE:figures/full_fig_p022_13.png]
Figure 14
Figure 14. Figure 14: Visualization of learned features by the Register Module on our RareFace-50 dataset. We [PITH_FULL_IMAGE:figures/full_fig_p022_14.png]
Figure 15
Figure 15. Figure 15: Visualization of learned features by the Register Module on our RareFace-50 dataset. We [PITH_FULL_IMAGE:figures/full_fig_p022_15.png]
Figure 16
Figure 16. Figure 16: User Study Interface. We ask each user to watch 8 videos and answer which method [PITH_FULL_IMAGE:figures/full_fig_p023_16.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

71 extracted references · 56 canonical work pages

  1. [1]

    Beit: Bert pre-training of image transformers

    Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image transformers. arXiv preprint arXiv:2106.08254, 2021. 3

    arXiv 2021

  2. [2]

    A morphable model for the synthesis of 3d faces

    V olker Blanz and Thomas Vetter. A morphable model for the synthesis of 3d faces. InProceedings of the 26th annual conference on Computer graphics and interactive techniques, pages 187–194, 1999. 3

    work page 1999

  3. [3]

    Marlin: Masked autoencoder for facial video representation learning

    Zhixi Cai, Shreya Ghosh, Kalin Stefanov, Abhinav Dhall, Jianfei Cai, Hamid Rezatofighi, Reza Haffari, and Munawar Hayat. Marlin: Masked autoencoder for facial video representation learning. In CVPR, 2023. 19

    work page 2023

  4. [4]

    Jean: Joint expression and audio-guided nerf-based talking face generation

    Sai Tanmay Reddy Chakkera, Aggelina Chatziagapi, and Dimitris Samaras. Jean: Joint expression and audio-guided nerf-based talking face generation. arXiv preprint arXiv:2409.12156, 2024. 3

    arXiv 2024

  5. [5]

    Efficient geometry-aware 3d generative adversarial networks

    Eric R Chan, Connor Z Lin, Matthew A Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas J Guibas, Jonathan Tremblay, Sameh Khamis, et al. Efficient geometry-aware 3d generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16123–16133, 2022. 3

    work page 2022

  6. [6]

    minlora: a minimal pytorch library that allows you to apply lora to any pytorch model

    Jonathan Chang and James Kelly. minlora: a minimal pytorch library that allows you to apply lora to any pytorch model. https://github.com/cccntu/minLoRA. Accessed: 2025-05-17. 14

    work page 2025

  7. [7]

    MI-NeRF: Learning a Single Face NeRF from Multiple Identities

    Aggelina Chatziagapi, Grigorios G Chrysos, and Dimitris Samaras. Mi-nerf: Learning a single face nerf from multiple identities. arXiv preprint arXiv:2403.19920, 2024. 3

    work page Pith review arXiv 2024

  8. [8]

    Gaussiantalker: Real-time talking head synthesis with 3d gaussian splatting

    Kyusun Cho, Joungbin Lee, Heeji Yoon, Yeobin Hong, Jaehoon Ko, Sangjun Ahn, and Seungryong Kim. Gaussiantalker: Real-time talking head synthesis with 3d gaussian splatting. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 10985–10994, 2024. 1, 3

    work page 2024

Show all 71 references
  1. [9]

    Generalizable and animatable gaussian head avatar

    Xuangeng Chu and Tatsuya Harada. Generalizable and animatable gaussian head avatar. In The Thirty- eighth Annual Conference on Neural Information Processing Systems, 2024. 1, 2, 3, 4, 6, 8, 15, 16

    2024

  2. [10]

    Gpavatar: Generalizable and precise head avatar from image (s)

    Xuangeng Chu, Yu Li, Ailing Zeng, Tianyu Yang, Lijian Lin, Yunfei Liu, and Tatsuya Harada. Gpavatar: Generalizable and precise head avatar from image (s). arXiv preprint arXiv:2401.10215, 2024. 3

    2024 arXiv

  3. [11]

    Vision transformers need registers

    Timothée Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers. arXiv preprint arXiv:2309.16588, 2023. 2, 3, 4, 8

    2023 arXiv

  4. [12]

    Arcface: Additive angular margin loss for deep face recognition

    Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In CVPR, 2019. 8, 15

    2019

  5. [13]

    BERT: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of...

    2019

  6. [14]

    Headgas: Real-time animatable head avatars via 3d gaussian splatting

    Helisa Dhamo, Yinyu Nie, Arthur Moreau, Jifei Song, Richard Shaw, Yiren Zhou, and Eduardo Pérez- Pellitero. Headgas: Real-time animatable head avatars via 3d gaussian splatting. In European Conference on Computer Vision, pages 459–476. Springer, 2024. 3

    2024

  7. [15]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...

    2021

  8. [16]

    Bakedavatar: Baking neural fields for real-time head avatar synthesis

    Hao-Bin Duan, Miao Wang, Jin-Chuan Shi, Xu-Chuan Chen, and Yan-Pei Cao. Bakedavatar: Baking neural fields for real-time head avatar synthesis. ACM Trans. Graph., 42(6), sep 2023. 3

    2023

  9. [17]

    Edelsbrunner, D

    H. Edelsbrunner, D. Kirkpatrick, and R. Seidel. On the shape of a set of points in the plane. IEEE Transactions on Information Theory, 29(4):551–559, 1983. 5 10

    1983

  10. [18]

    Dynamic neural radiance fields for monocular 4d facial avatar reconstruction, 2020

    Guy Gafni, Justus Thies, Michael Zollhöfer, and Matthias Nießner. Dynamic neural radiance fields for monocular 4d facial avatar reconstruction, 2020. 1, 3

    2020

  11. [19]

    Portrait neural radiance fields from a single image

    Chen Gao, Yichang Shih, Wei-Sheng Lai, Chia-Kai Liang, and Jia-Bin Huang. Portrait neural radiance fields from a single image. arXiv preprint arXiv:2012.05903, 2020. 3

    2012 arXiv

  12. [20]

    Automatic face reenactment

    Pablo Garrido, Levi Valgaerts, Ole Rehmsen, Thorsten Thormahlen, Patrick Perez, and Christian Theobalt. Automatic face reenactment. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4217–4224, 2014. 1, 3

    2014

  13. [21]

    Vdub: Modifying face video of actors for plausible visual alignment to a dubbed audio track

    Pablo Garrido, Levi Valgaerts, Hamid Sarmadi, Ingmar Steiner, Kiran Varanasi, Patrick Perez, and Christian Theobalt. Vdub: Modifying face video of actors for plausible visual alignment to a dubbed audio track. In Computer graphics forum, volume 34, pages 193–204. Wiley Online ...

    2015

  14. [22]

    Understanding the difficulty of training deep feedforward neural networks

    Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Yee Whye Teh and Mike Titterington, editors,Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, volume 9 ofProceedings o...

    2010

  15. [23]

    Insightface repository model zoo

    Jia Guo, Jiankang Deng, Xiang An, Jack Yu, and Baris Gecer. Insightface repository model zoo. 15

  16. [24]

    Long short-term memory

    Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780,

  17. [25]

    Parameter-efficient transfer learning for nlp

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Ges- mundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. In International conference on machine learning, pages 2790–2799. PMLR, 2019. 3

    2019

  18. [26]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. 2, 3, 4, 8, 9, 14, 16

    2022

  19. [27]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), 2023. 1, 3

    2023

  20. [28]

    Deep video portraits

    Hyeongwoo Kim, Pablo Garrido, Ayush Tewari, Weipeng Xu, Justus Thies, Matthias Niessner, Patrick Pérez, Christian Richardt, Michael Zollhöfer, and Christian Theobalt. Deep video portraits. ACM Transactions on Graphics (TOG), 37(4):1–14, 2018. 1, 3

    2018

  21. [29]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017. 15

    2017

  22. [30]

    Diffusionavatars: Deferred diffusion for high-fidelity 3d head avatars

    Tobias Kirschstein, Simon Giebenhain, and Matthias Nießner. Diffusionavatars: Deferred diffusion for high-fidelity 3d head avatars. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5481–5492, 2024. 3

    2024

  23. [31]

    Gghead: Fast and generalizable 3d gaussian heads

    Tobias Kirschstein, Simon Giebenhain, Jiapeng Tang, Markos Georgopoulos, and Matthias Nießner. Gghead: Fast and generalizable 3d gaussian heads. In SIGGRAPH Asia 2024 Conference Papers, pages 1–11, 2024. 3

    2024

  24. [32]

    Instag: Learning personalized 3d talking head from few-second video

    Jiahe Li, Jiawei Zhang, Xiao Bai, Jin Zheng, Jun Zhou, and Lin Gu. Instag: Learning personalized 3d talking head from few-second video. arXiv preprint arXiv:2502.20387, 2025. 3

    2025 arXiv

  25. [33]

    Tianye Li, Timo Bolkart, Michael. J. Black, Hao Li, and Javier Romero. Learning a model of facial shape and expression from 4D scans. ACM Transactions on Graphics, (Proc. SIGGRAPH Asia), 36(6):194:1– 194:17, 2017. 5, 14

    2017

  26. [34]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020. 3

    2020

  27. [35]

    Reptile: a scalable metalearning algorithm

    Alex Nichol and John Schulman. Reptile: a scalable metalearning algorithm. arXiv preprint arXiv:1803.02999, 2(3):4, 2018. 15

    2018 arXiv

  28. [36]

    Mystyle: A personalized generative prior

    Yotam Nitzan, Kfir Aberman, Qiurui He, Orly Liba, Michal Yarom, Yossi Gandelsman, Inbar Mosseri, Yael Pritch, and Daniel Cohen-Or. Mystyle: A personalized generative prior. arXiv preprint arXiv:2203.17272,

    arXiv

  29. [37]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 2 11

    2023 arXiv

  30. [38]

    Barron, Sofien Bouaziz, Dan B Goldman, Steven M

    Keunhong Park, Utkarsh Sinha, Jonathan T. Barron, Sofien Bouaziz, Dan B Goldman, Steven M. Seitz, and Ricardo Martin-Brualla. Nerfies: Deformable neural radiance fields. ICCV, 2021. 1, 3

    2021

  31. [39]

    Hypernerf: A higher-dimensional representation for topologi- cally varying neural radiance fields

    Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Martin-Brualla, and Steven M Seitz. Hypernerf: A higher-dimensional representation for topologi- cally varying neural radiance fields. arXiv preprint arXiv:2106.13228, 2021. 1, 3

    2021 arXiv

  32. [40]

    Namboodiri, and C.V

    K R Prajwal, Rudrabha Mukhopadhyay, Vinay P. Namboodiri, and C.V . Jawahar. A lip sync expert is all you need for speech to lip generation in the wild. InProceedings of the 28th ACM International Conference on Multimedia, page 484–492, 2020. 1, 3

    2020

  33. [41]

    Ganimation: One-shot anatomically consistent facial animation

    Albert Pumarola, Antonio Agudo, Aleix M Martinez, Alberto Sanfeliu, and Francesc Moreno-Noguer. Ganimation: One-shot anatomically consistent facial animation. International Journal of Computer Vision, 128(3):698–713, 2020. 1, 3

    2020

  34. [42]

    D-nerf: Neural radiance fields for dynamic scenes

    Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer. D-nerf: Neural radiance fields for dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10318–10327, 2021. 1, 3

    2021

  35. [43]

    My3dgen: A scalable personalized 3d generative model

    Luchao Qi, Jiaye Wu, Annie N Wang, Shengze Wang, and Roni Sengupta. My3dgen: A scalable personalized 3d generative model. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 961–972. IEEE, 2025. 3

    2025

  36. [44]

    Gaussianavatars: Photorealistic head avatars with rigged 3d gaussians

    Shenhan Qian, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Simon Giebenhain, and Matthias Nießner. Gaussianavatars: Photorealistic head avatars with rigged 3d gaussians. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20299–20...

    2024

  37. [45]

    Detecting deepfakes without seeing any

    Tal Reiss, Bar Cavia, and Yedid Hoshen. Detecting deepfakes without seeing any. arXiv preprint arXiv:2311.01458, 2023. 19

    2023 arXiv

  38. [46]

    Talklora: Low-rank adaptation for speech-driven animation

    Jack Saunders and Vinay Namboodiri. Talklora: Low-rank adaptation for speech-driven animation. arXiv preprint arXiv:2408.13714, 2024. 2, 3, 8

    2024 arXiv

  39. [47]

    First order motion model for image animation

    Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe. First order motion model for image animation. In Conference on Neural Information Processing Systems (NeurIPS), December 2019. 3

    2019

  40. [48]

    End-to-end memory networks

    Sainbayar Sukhbaatar, Jason Weston, Rob Fergus, et al. End-to-end memory networks. Advances in neural information processing systems, 28, 2015. 3

    2015

  41. [49]

    Face2face: Real-time face capture and reenactment of rgb videos

    Justus Thies, Michael Zollhofer, Marc Stamminger, Christian Theobalt, and Matthias Nießner. Face2face: Real-time face capture and reenactment of rgb videos. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2387–2395, 2016. 1, 3

    2016

  42. [50]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. 3

    2017

  43. [51]

    End-to-end speech-driven realistic facial animation with temporal gans

    Konstantinos V ougioukas, Stavros Petridis, and Maja Pantic. End-to-end speech-driven realistic facial animation with temporal gans. In CVPRW, 2019. 8

    2019

  44. [52]

    Realistic speech-driven facial animation with gans

    Konstantinos V ougioukas, Stavros Petridis, and Maja Pantic. Realistic speech-driven facial animation with gans. International Journal of Computer Vision, 128(5):1398–1413, 2020. 1, 3

    2020

  45. [53]

    Gaussianhead: High-fidelity head avatars with learnable gaussian derivation

    Jie Wang, Jiu-Cheng Xie, Xianyan Li, Feng Xu, Chi-Man Pun, and Hao Gao. Gaussianhead: High-fidelity head avatars with learnable gaussian derivation. IEEE Transactions on Visualization and Computer Graphics, 2025. 3

    2025

  46. [54]

    Memory networks

    Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks. arXiv preprint arXiv:1410.3916,

    arXiv

  47. [55]

    Vfhq: A high-quality dataset and benchmark for video face super-resolution

    Liangbin Xie, Xintao Wang, Honglun Zhang, Chao Dong, and Ying Shan. Vfhq: A high-quality dataset and benchmark for video face super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 657–666, 2022. 6

    2022

  48. [56]

    Learning frame-wise emotion intensity for audio-driven talking-head generation

    Jingyi Xu, Hieu Le, Zhixin Shu, Yang Wang, Yi-Hsuan Tsai, and Dimitris Samaras. Learning frame-wise emotion intensity for audio-driven talking-head generation. arXiv preprint arXiv:2409.19501, 2024. 3 12

    2024 arXiv

  49. [57]

    Vasa-1: Lifelike audio-driven talking faces generated in real time

    Sicheng Xu, Guojun Chen, Yu-Xiao Guo, Jiaolong Yang, Chong Li, Zhenyu Zang, Yizhong Zhang, Xin Tong, and Baining Guo. Vasa-1: Lifelike audio-driven talking faces generated in real time. arXiv preprint arXiv:2404.10667, 2024. 3

    2024 arXiv

  50. [58]

    Gaussian head avatar: Ultra high-fidelity head avatar via dynamic gaussians

    Yuelang Xu, Benwang Chen, Zhe Li, Hongwen Zhang, Lizhen Wang, Zerong Zheng, and Yebin Liu. Gaussian head avatar: Ultra high-fidelity head avatar via dynamic gaussians. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 1, 3

    2024

  51. [59]

    One2avatar: Generative implicit head avatar for few-shot user adaptation

    Zhixuan Yu, Ziqian Bai, Abhimitra Meka, Feitong Tan, Qiangeng Xu, Rohit Pandey, Sean Fanello, Hyun Soo Park, and Yinda Zhang. One2avatar: Generative implicit head avatar for few-shot user adaptation. arXiv preprint arXiv:2402.11909, 2024. 3

    2024 arXiv

  52. [60]

    Metaportrait: Identity-preserving talking head generation with fast personalized adaptation

    Bowen Zhang, Chenyang Qi, Pan Zhang, Bo Zhang, HsiangTao Wu, Dong Chen, Qifeng Chen, Yong Wang, and Fang Wen. Metaportrait: Identity-preserving talking head generation with fast personalized adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern R...

    2023

  53. [61]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3836–3847,

  54. [62]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018. 8, 15

    2018

  55. [63]

    Pose-controllable talking face generation by implicitly modularized audio-visual representation

    Hang Zhou, Yasheng Sun, Wayne Wu, Chen Change Loy, Xiaogang Wang, and Ziwei Liu. Pose-controllable talking face generation by implicitly modularized audio-visual representation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. 3

    2021

  56. [64]

    Webface260m: A benchmark unveiling the power of million-scale deep face recognition

    Zheng Zhu, Guan Huang, Jiankang Deng, Yun Ye, Junjie Huang, Xinze Chen, Jiagang Zhu, Tian Yang, Jiwen Lu, Dalong Du, and Jie Zhou. Webface260m: A benchmark unveiling the power of million-scale deep face recognition. In CVPR, 2021. 15

    2021

  57. [65]

    Instant volumetric head avatars

    Wojciech Zielonka, Timo Bolkart, and Justus Thies. Instant volumetric head avatars. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4574–4584, 2023. 3 Appendix The appendix is organized as follows:

    2023

  58. [66]

    Additional Ablation Study in Sec. A

  59. [67]

    Implementation Details in Sec. B

  60. [68]

    Dataset Collection Details in Sec. C

  61. [69]

    Additional Results in Sec. D

  62. [70]

    User Study Details in Sec. E

  63. [71]

    buffalo_l

    Discussion: Broader Impact, Limitations, and Ethical Considerations in Sec. F We strongly encourage the readers to watch our supplementary video. A Additional Ablation Study Table 3: Ablation study on losses to adapt with our method. In (a), we removeLf eatduring adaptation. I...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.