Pith. sign in

REVIEW 3 major objections 5 minor 84 references

Total-Editing: Head Avatar with Editable Appearance, Motion, and Lighting

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read A single neural radiance field pipeline edits a head avatar's appearance, motion, and lighting independently.

desk verdict Solid unified head-avatar system with a promising dataset, but the central relighting claim is never quantitatively validated and the architecture admits a real leakage channel. read the letter →

arxiv 2505.20582 v1 pith:OVW5UU6X submitted 2025-05-26 cs.CV

classification cs.CV
keywords facereenactmentportraitrelightingneuralradiancefieldintrinsicdecompositionlightmapestimationmovingleastsquaresdeformationheadavatarHDRenvironmentmap
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to prove that face reenactment and portrait relighting belong in one pipeline, not two. It presents Total-Editing, a single head-avatar framework that takes three independent inputs—an appearance portrait, a motion video, and a lighting source (another portrait or an HDR environment map)—and synthesizes a realistic animated portrait in which the light and shadow move with the head. The authors argue that current reenactment models bake lighting into facial texture and current relighters ignore motion, so a joint, illumination-aware 3D representation avoids the accumulated errors of running reenactment and relighting separately. If the claim holds, portrait editing reduces to choosing three sources, and effects like transferring one portrait's lighting onto another person's animation become direct operations.

What carries the argument

The load-bearing object is an intrinsically decomposed neural radiance field decoder. Its shading decoder splits the volume color into albedo, diffuse coefficient, specular coefficients, and a residual, then recombines them with the Phong reflection model, sampling diffuse and specular shading from pre-filtered lightmaps instead of regressing shading from appearance. The lightmaps are the lighting interface: HDR environment maps are baked into them directly, and a portrait image is converted into them by a transformer-based lightmap estimator trained on synthetic ground truth. The second mechanism is a moving-least-squares (MLS) deformation field derived from a parametric 3D face model, which warps tri-plane features and rotates decoded normals; MLS's globally averaged, distance-weighted transformations are what keep motion and shading spatially coherent.

What would settle it

Re-render one fixed identity through the same motion under two known HDR environment maps with very different directionality, and compare the estimated lightmaps and final shading against ground-truth images rendered under those same maps. If the estimated lightmaps fail to track the environment, or if a strong facial texture such as a beard remains dark under opposite lighting, the claimed albedo-shading disentanglement is not real.

Watch

Extended reading notes

Core claim

Total-Editing's central claim is that a neural radiance field can be decomposed into intrinsic components—density, surface normal, albedo, diffuse and specular shading coefficients, and a residual color—so that appearance, motion, and lighting are separately controllable in one synthesized 3D head. Diffuse and specular shading are not learned as part of the face texture; they are looked up from pre-filtered lightmaps, which are either baked from an HDR environment map or estimated from a portrait image by a lightmap estimator. Motion is carried by a moving-least-squares deformation field that warps volumetric features and rotates normals continuously, so shading follows the geometry as the head turns. The paper reports that this end-to-end design yields higher reconstruction quality, better identity preservation, and more accurate motion control than reenactment-only systems or two-stage reenactment-then-relighting pipelines, and it demonstrates illumination transfer, HDR relighting, and background replacement as applications.

Load-bearing premise

The lighting claim depends on the network truly separating permanent facial coloring (albedo) from light-and-shadow effects (shading); if the lightmap estimator instead reads texture such as a beard as shadow, the demonstrated lighting edits are correlations from training data, not genuine relighting.

Editorial extensions

If this is right

  • A user can take one portrait for identity, another video for motion, and either a portrait or an HDR map for lighting, and synthesize a video where the shading shifts naturally as the head moves.
  • Because reenactment and relighting are done in one pass, the compounding errors of two-stage reenactment-then-relighting pipelines are avoided.
  • Lighting can be transferred from one portrait onto another person's animated head without retraining.
  • The same representation supports placing an animated portrait into a new environment by baking that environment's HDR map and rendering matching backgrounds.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper chooses its final training scheme by reenactment validation performance rather than by a direct measure of albedo-shading disentanglement; a dedicated relighting benchmark with ground-truth HDR illumination would be the direct check on the framework's strongest claim.
  • The MLS deformation field is a generic warp, so the same mechanism could apply to articulated bodies or other objects where surface-field warping produces seams, not just heads.
  • Because the lightmap estimator is pretrained on synthetic data with ground truth, retraining it on physics-based renderings of other object classes could extend this style of control beyond faces.
  • One extension the paper does not develop is continuous control of light intensity or direction during animation, which would allow gradual lighting changes within a single motion sequence rather than one fixed environment.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces Total-Editing, a unified portrait editing framework that combines one-shot face reenactment, portrait relighting, and intrinsic decomposition in a neural radiance field. The method takes an appearance image, a motion source, and a lighting source (either a portrait image or an HDR environment map) and synthesizes a head avatar with controllable appearance, motion, and lighting. The color is decomposed via the Phong reflection model into diffuse and specular terms using pre-filtered lightmaps, an MLS-based deformation field replaces the Surface Field used in prior work, and training is performed on a new synthetic dataset of roughly 2M frames with ground-truth albedo/normal/HDR, together with the VFHQ real dataset. Experiments report reenactment quality metrics, ablations on deformation fields and data/regularization choices, and qualitative demonstrations of relighting and background replacement.

Significance. If confirmed, the paper would provide a useful step toward a single 3D-aware pipeline for both reenactment and relighting, and the promised release of a 2M-frame synthetic dataset with ground-truth intrinsic decompositions and HDR environments is a valuable community asset. The architecture is clearly motivated, and the reenactment results improve over Portrait4D-v2 on several metrics (CSIM, AED, APD, FID in cross-reenactment). However, the central claim of precise and controllable lighting is not quantitatively established: all numeric tables measure reenactment quality, not relighting accuracy or albedo-shading disentanglement, and the acknowledged leakage risk (albedo misinterpreted as shading) is never measured. The overstatement of 'outperforms all other approaches' relative to Table 1 also requires correction. These gaps are load-bearing for the manuscript's main novelty, so the paper needs major revision.

major comments (3)
  1. [4.2, Table 1] The statement in Sec. 4.2 that 'our end-to-end method outperforms all other approaches in the face reenactment task' is not supported by Table 1 in the self-reenactment setting: GPAvatar achieves higher PSNR (20.7 vs. 20.3) and SSIM (0.753 vs. 0.730), while Total-Editing wins on LPIPS, CSIM, AED, and APD. Because no error bars or significance tests are reported, the averaged differences may not be meaningful, and the global claim is too strong. Please qualify the statement to the specific metrics and settings where the method actually leads, and discuss the trade-off with PSNR/SSIM.
  2. [4.3, Eq. (18), Supp. A] The paper's central novelty is lighting control, yet no quantitative evaluation of relighting accuracy or of albedo-shading disentanglement is provided. Tables 1-3 measure only reenactment quality; in Table 1 the lighting source is the appearance image, and in Table 3 the editing target is the motion frame itself (or a random frame from the same video), so lighting transfer is never compared against ground-truth relit images. The residual decoder delta_c in Eq. (18) can compensate for incorrect lightmaps, and on real data neither albedo nor lightmaps have ground-truth supervision (Supplement A states that L_a and L_S are applied only to synthetic data). Sec. 4.3 explicitly admits that without regularization the lightmap estimator can misinterpret albedo as shading, e.g., beards as shadows, yet the chosen regularization (Random Ilit, row 5 of Table 3) is selected by reenactment metrics rather than by a disentanglement measure. I recommend adding a quantitative relighting benchmark (e.g., light-stage data or a held-out synthetic test set with known HDR), a measure of albedo stability under varying lighting sources, and a comparison with at least one dedicated relighting baseline in the same setting.
  3. [3.4, 4.3] The dependence of the output on the lighting source is not directly measured anywhere in the paper. Because the lightmap estimator and the shading decoder are co-trained on real data without lighting ground truth, the model could in principle memorize correlations between the appearance image and the target frame rather than perform physical relighting. A direct disentanglement test would help: fix appearance and motion, vary only the lighting source (portrait image or HDR map), and report the pixel-wise or perceptual difference in the face region; conversely, fix lighting and motion and vary appearance to verify that the estimated lightmaps do not absorb identity. This experiment would make the 'precise control over lighting' claim falsifiable and is currently missing.
minor comments (5)
  1. [3.2, Eq. (12)] The MLS fall-off exponent alpha is fixed at 1.0 with no sensitivity analysis; a small sweep (e.g., 0.5, 1.0, 2.0) would help establish that the gains over Surface Fields are not tied to this particular choice.
  2. [3.1.1] The symbol n is used both for the surface normal and for the shininess exponent n in {1,16,32,64}, which is confusing in Eqs. (3) and (8); consider renaming the shininess exponent to m or q.
  3. [4.1, Table 3] In Sec. 4.1, 'resolution 5122' should be '512x512'; the caption of Table 3 does not define the meaning of the check marks in the Synthetic and Real columns.
  4. [4.3] The two paragraphs that begin 'Impact of ...' both reference 'Rows 1-3 of Tab. 3', which makes it easy to lose track of which rows are being discussed; renumber the rows in the table or restructure the text for clarity.
  5. [Supp. D] For the two-stage reenactment-relighting pipelines, the table would benefit from a footnote clarifying that the reported numbers are computed only on the valid cropped areas, since this may affect comparability with the end-to-end method.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central reenactment claim is benchmarked against external baselines, and the lighting-control gap is a validation limitation, not a derivation that reduces to its inputs.

full rationale

Total-Editing does not claim to derive analytic results from first principles; it is an end-to-end learned system. The lighting path is either physically baked from HDR environment maps (Sec. 3.1.2) or estimated by a network from a portrait image (Sec. 3.4), and neither path is defined in terms of the final editing frames. The real-data training protocol sets the editing target to the motion source while taking the lighting source from another frame of the same video, so lighting transfer is not quantitatively validated against ground truth; this is a correctness and evaluation gap, not a circular reduction. The lightmap estimator is pre-trained with synthetic ground-truth lightmaps and normals (Eq. 22), and the albedo and lightmap losses are applied on synthetic data; this is supervised evidence from an external, self-created benchmark, not a fit-then-predict rename. The paper builds on prior work [16, 17, 67] by non-overlapping authors and compares against external baselines on VFHQ-Test, so the reenactment claim is independently supported. The acknowledged albedo/shading ambiguity in Sec. 4.3 is an identifiability concern, and choosing Row 5 of Table 3 by validation performance is a standard model-selection procedure rather than circularity. No equation equates a predicted quantity to a fitted input by construction, and no load-bearing argument relies on self-citation.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The central claim rests mostly on learned components and empirical training choices rather than analytic derivation. The listed parameters and assumptions cover the hand-set values and domain assumptions that could break the claimed appearance, motion, and lighting disentanglement.

free parameters (2)
  • MLS fall-off exponent alpha = 1.0
    Set by hand in Eq. (12); controls the smoothness and locality of the moving least squares deformation field.
  • Shininess exponents for specular lightmaps = {1, 16, 32, 64}
    Hand-chosen set in Eq. (3) to represent specular lobes; no sensitivity analysis or optimization is reported.
assumptions (6)
  • domain assumption Phong reflection model with the ambient term omitted approximates face appearance under arbitrary lighting
    Used in Sec. 3.1.1, Eqs. (1)-(3), as the basis for color decomposition; real faces have subsurface scattering and complex reflectance not captured by Phong.
  • domain assumption Prefiltered Phong lightmaps preserve enough lighting detail for relighting
    Sec. 3.1.2 replaces full environment integrals with baked lightmaps; the approximation error introduced by this reduction is not quantified.
  • ad hoc to paper The synthetic 2M-frame rendered dataset transfers to real portraits
    Sec. 4.1 uses synthetic data for pretraining and joint training; no cross-dataset quantitative validation beyond VFHQ is provided.
  • domain assumption Expression features from the off-the-shelf encoder are appearance-free enough for de-enactment and re-enactment
    Sec. 3.2 adopts the expression encoder from [67]; the paper inherits this property without verifying it for lighting-neutral feature extraction.
  • standard math The standard volumetric rendering equation is valid for this head avatar setting
    Eqs. (5)-(6) in Sec. 3.1.3; widely accepted in the NeRF literature but not re-derived here.
  • standard math MLS rigid transformation solved via SVD yields continuous deformation fields
    Sec. 3.2 Eqs. (11)-(13) rely on the known MLS formulation from [83].

how reviews work

0 comments
Cite this review

Pith. "Pith review of Total-Editing: Head Avatar with Editable Appearance, Motion, and Lighting." pith.science (2026). https://pith.science/paper/OVW5UU6X

@misc{pith2026250520582,
  author       = {Pith},
  title        = {Pith review of: Total-Editing: Head Avatar with Editable Appearance, Motion, and Lighting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OVW5UU6X}},
  note         = {Machine review of arXiv:2505.20582}
}
read the original abstract

Face reenactment and portrait relighting are essential tasks in portrait editing, yet they are typically addressed independently, without much synergy. Most face reenactment methods prioritize motion control and multiview consistency, while portrait relighting focuses on adjusting shading effects. To take advantage of both geometric consistency and illumination awareness, we introduce Total-Editing, a unified portrait editing framework that enables precise control over appearance, motion, and lighting. Specifically, we design a neural radiance field decoder with intrinsic decomposition capabilities. This allows seamless integration of lighting information from portrait images or HDR environment maps into synthesized portraits. We also incorporate a moving least squares based deformation field to enhance the spatiotemporal coherence of avatar motion and shading effects. With these innovations, our unified framework significantly improves the quality and realism of portrait editing results. Further, the multi-source nature of Total-Editing supports more flexible applications, such as illumination transfer from one portrait to another, or portrait animation with customized backgrounds.

Figures

Figures reproduced from arXiv: 2505.20582 by the authors.

Figure 1
Figure 1. Our Total-Editing enables geometry-and-illumination-aware portrait editing with appearance sources (top-right corner), motion [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Face reenactment under uneven lighting. Existing models like [17] couple facial textures and lighting, resulting in fixed light and shadow that do not adapt to head movements. In contrast, our model provides more realistic portrait shading. • We contribute a physically rendered synthetic dataset for general portrait editing, featuring 2M frames that capture diverse subjects, views, poses, expressions, and lighting e… view at source ↗
Figure 3
Figure 3. The framework of Total-Editing. Sec. 3.2: Our pipeline learns to encode appearance and motion sources Iapp, Imot, neutralize the expression from Iapp and reapply the expression from Imot to obtain a fused feature F. After generating canonical space geometry and shading tri-planes Tgeo, Tshd, the neck pose is handled by warping features with moving least squares based deformation fields R, T . Sec. 3.4: For the light… view at source ↗
Figures from the paper (15 more)
Figure 5
Figure 5. Figure 5: Shading module architecture. The shading feature tshd sampled from a position in the shading tri-plane is first de￾coded into normal n, albedo a, and additional features w for super-resolution. The viewing direction v is then reflected with the normal n, resulting in a…
Figure 6
Figure 6. Figure 6: Lightmap estimator architecture. During pre-training with synthetic data, we adopt a U-Net to encode lighting source Ilit and decode pixel-wise normal N. The intermediate feature zlit is used to decode diffuse and specular lightmaps Sd, {Ss(n)}, querying with embedded …
Figure 7
Figure 7. Figure 7: Qualitative comparison of cross-reenactment on VFHQ-Test. For GPAvatar [10] (c), Real3DPortrait [74] (e), and Portrait4D￾v2 [17] (g), we use appearance sources (a) and motion sources (b) as inputs. For additional relighting with PortraitRelighting [6], we use (a) as th…
Figure 8
Figure 8. Figure 8: Qualitative comparison of self-reenactment on VFHQ-Test. We employ the same input settings as in [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: More applications. Total-Editing enables flexible ap￾plications such as animatable portraits with background changing. Limitation discussion and future work. The current for￾mulation of Total-Editing does not account for visibility and self-occlusion, making it difficu…
Figure 10
Figure 10. Figure 10: Samples from one identity of the Lumos [75] dataset. We refer to each column as a unique subject, since they have different appearances and accessories. demonstrating the effectiveness of Total-Editing in both motion and lighting control. Further, we explore two down￾…
Figure 11
Figure 11. Figure 11: Additional cross-reenactment results on the VFHQ dataset. ing spatial and/or temporal coherence. D. Evaluation Details In Tab. 1, we exclude both generated and ground truth back￾grounds from metric calculations, focusing solely on the quality of the portrait regions. …
Figure 12
Figure 12. Figure 12: Cross-reenactment results on the HTDF dataset. 3 [PITH_FULL_IMAGE:figures/full_fig_p015_12.png]
Figure 13
Figure 13. Figure 13: Cross-reenactment results on the HTDF dataset (continued). (a) Appearance (b) Motion (c) GPA (d) R3DP (e) P4Dv2 (f) Ours [PITH_FULL_IMAGE:figures/full_fig_p016_13.png]
Figure 14
Figure 14. Figure 14: Self-reenactment results on the HTDF dataset. 4 [PITH_FULL_IMAGE:figures/full_fig_p016_14.png]
Figure 15
Figure 15. Figure 15: Cross-reenactment results on the VFHQ dataset with HDR environment maps as lighting sources. 5 [PITH_FULL_IMAGE:figures/full_fig_p017_15.png]
Figure 16
Figure 16. Figure 16: Cross-reenactment results on the VFHQ dataset with portrait images as lighting sources. 6 [PITH_FULL_IMAGE:figures/full_fig_p018_16.png]
Figure 17
Figure 17. Figure 17: Multi-view subset of our synthetic data. It incorporates 50K subjects. Each is rendered in 2 environments with 10 views. 7 [PITH_FULL_IMAGE:figures/full_fig_p019_17.png]
Figure 18
Figure 18. Figure 18: Video subset of our synthetic data. It includes 10K subjects. Each is rendered in 10 environments with 10 poses/expressions. 8 [PITH_FULL_IMAGE:figures/full_fig_p020_18.png]
Figure 19
Figure 19. Figure 19: More subjects in our synthetic data. Subjects are with randomized poses, expressions, hairstyles, skin types, accessories, etc. 9 [PITH_FULL_IMAGE:figures/full_fig_p021_19.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

84 extracted references · 67 canonical work pages

  1. [1]

    Styleflow: Attribute-conditioned exploration of stylegan- generated images using conditional continuous normalizing flows

    Rameen Abdal, Peihao Zhu, Niloy J Mitra, and Peter Wonka. Styleflow: Attribute-conditioned exploration of stylegan- generated images using conditional continuous normalizing flows. ACM Transactions on Graphics (ToG) , 40(3):1–21,

  2. [2]

    Audio-Visual Face Reenactment

    Madhav Agarwal et al. Audio-visual face reenactment. arXiv preprint arXiv:2210.02755, 2022. 1

  3. [3]

    Rignerf: Fully controllable neu- ral 3d portraits

    ShahRukh Athar, Zexiang Xu, Kalyan Sunkavalli, Eli Shechtman, and Zhixin Shu. Rignerf: Fully controllable neu- ral 3d portraits. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20364–20373, 2022. 2

  4. [4]

    Generative neu- ral articulated radiance fields

    Alexander Bergman, Petr Kellnhofer, Wang Yifan, Eric Chan, David Lindell, and Gordon Wetzstein. Generative neu- ral articulated radiance fields. Advances in Neural Informa- tion Processing Systems, 35:19900–19916, 2022. 2, 4, 5

  5. [5]

    A morphable model for the synthesis of 3d faces

    V olker Blanz and Thomas Vetter. A morphable model for the synthesis of 3d faces. In Proceedings of the 26th an- nual conference on Computer graphics and interactive tech- niques, pages 187–194. ACM Press/Addison-Wesley Pub- lishing Co., 1999. 2

  6. [6]

    Real-time 3d-aware portrait video relighting

    Ziqi Cai, Kaiwen Jiang, Shu-Yu Chen, Yu-Kun Lai, Hongbo Fu, Boxin Shi, and Lin Gao. Real-time 3d-aware portrait video relighting. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 6221–6231, 2024. 6, 7, 8, 2

  7. [7]

    Sira: Re- lightable avatars from a single image

    Pol Caselles, Eduard Ramon, Jaime Garcia, Xavier Giro-i Nieto, Francesc Moreno-Noguer, and Gil Triginer. Sira: Re- lightable avatars from a single image. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 775–784, 2023. 3

  8. [8]

    Efficient geometry-aware 3d generative adversarial networks

    Eric R Chan, Connor Z Lin, Matthew A Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas J Guibas, Jonathan Tremblay, Sameh Khamis, et al. Efficient geometry-aware 3d generative adversarial networks. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16123–16133, 2022. 5, 1

Show all 84 references
  1. [9]

    Generalizable and animatable gaussian head avatar

    Xuangeng Chu and Tatsuya Harada. Generalizable and animatable gaussian head avatar. arXiv preprint arXiv:2410.07971, 2024. 2

  2. [10]

    Gpavatar: Generaliz- able and precise head avatar from image (s)

    Xuangeng Chu, Yu Li, Ailing Zeng, Tianyu Yang, Lijian Lin, Yunfei Liu, and Tatsuya Harada. Gpavatar: Generaliz- able and precise head avatar from image (s). arXiv preprint arXiv:2401.10215, 2024. 2, 7, 8

  3. [11]

    Torrance

    Robert L Cook and Kenneth E. Torrance. A reflectance model for computer graphics. ACM Transactions on Graph- ics (ToG), 1(1):7–24, 1982. 3

  4. [12]

    Acquiring the reflectance field of a human face

    Paul Debevec, Tim Hawkins, Chris Tchou, Haarm-Pieter Duiker, Westley Sarokin, and Mark Sagar. Acquiring the reflectance field of a human face. In Proceedings of the 27th annual conference on Computer graphics and interac- tive techniques, pages 145–156, 2000. 3

  5. [13]

    Lu- migan: Unconditional generation of relightable 3d human faces

    Boyang Deng, Yifan Wang, and Gordon Wetzstein. Lu- migan: Unconditional generation of relightable 3d human faces. In 2024 International Conference on 3D Vision (3DV), pages 302–312. IEEE, 2024. 3, 8

  6. [14]

    Arcface: Additive angular margin loss for deep face recognition

    Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 4690–4699, 2019. 6, 1

  7. [15]

    Accurate 3d face reconstruction with weakly-supervised learning: From single image to image set

    Yu Deng, Jiaolong Yang, Sicheng Xu, Dong Chen, Yunde Jia, and Xin Tong. Accurate 3d face reconstruction with weakly-supervised learning: From single image to image set. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition workshops , pages 0–0...

  8. [16]

    Portrait4d: Learning one-shot 4d head avatar synthesis using synthetic data

    Yu Deng, Duomin Wang, Xiaohang Ren, Xingyu Chen, and Baoyuan Wang. Portrait4d: Learning one-shot 4d head avatar synthesis using synthetic data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7119–7130, 2024. 2, 3, 4, 5

  9. [17]

    Portrait4d-v2: Pseudo multi-view data creates better 4d head synthesizer

    Yu Deng, Duomin Wang, and Baoyuan Wang. Portrait4d-v2: Pseudo multi-view data creates better 4d head synthesizer. arXiv preprint arXiv:2403.13570, 2024. 2, 3, 4, 5, 6, 7, 8

  10. [18]

    Split: Single portrait lighting estimation via a tetrad of face intrinsics

    Fan Fei, Yean Cheng, Yongjie Zhu, Qian Zheng, Si Li, Gang Pan, and Boxin Shi. Split: Single portrait lighting estimation via a tetrad of face intrinsics. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023. 3

  11. [19]

    Dynamic neural radiance fields for monocular 4d facial avatar reconstruction

    Guy Gafni, Justus Thies, Michael Zollh ¨ofer, and Matthias Nießner. Dynamic neural radiance fields for monocular 4d facial avatar reconstruction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8649–8658, 2021. 2

  12. [20]

    Generative adversarial nets

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014. 2

  13. [21]

    Project starline

    Google. Project starline. https://en.wikipedia. org/wiki/Project_Starline, 2021. 1

  14. [22]

    High- fidelity relightable monocular portrait animation with lighting-controllable video diffusion model

    Mingtao Guo, Guanyu Xing, and Yanli Liu. High- fidelity relightable monocular portrait animation with lighting-controllable video diffusion model. arXiv preprint arXiv:2502.19894, 2025. 3

  15. [23]

    Realistic, hardware-accelerated shading and lighting

    Wolfgang Heidrich and Hans-Peter Seidel. Realistic, hardware-accelerated shading and lighting. In Proceedings of the 26th annual conference on Computer graphics and in- teractive techniques, pages 171–178, 1999. 3

  16. [24]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 6

  17. [25]

    Depth-aware generative adversarial network for talking head video generation

    Fa-Ting Hong, Longhao Zhang, Li Shen, and Dan Xu. Depth-aware generative adversarial network for talking head video generation. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 3397–3406, 2022. 2

  18. [26]

    Towards high fidelity face relight- ing with realistic shadows

    Andrew Hou, Ze Zhang, Michel Sarkis, Ning Bi, Yiying Tong, and Xiaoming Liu. Towards high fidelity face relight- ing with realistic shadows. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 14719–14728, 2021. 3 9

  19. [27]

    Face relighting with geometrically consistent shadows

    Andrew Hou, Michel Sarkis, Ning Bi, Yiying Tong, and Xi- aoming Liu. Face relighting with geometrically consistent shadows. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4217–4226,

  20. [28]

    Geometry-aware single-image full-body human relight- ing

    Chaonan Ji, Tao Yu, Kaiwen Guo, Jingxin Liu, and Yebin Liu. Geometry-aware single-image full-body human relight- ing. In European Conference on Computer Vision , pages 388–405. Springer, 2022. 3

  21. [29]

    Nerf- facelighting: Implicit and disentangled face lighting rep- resentation leveraging generative prior in neural radiance fields

    Kaiwen Jiang, Shu-Yu Chen, Hongbo Fu, and Lin Gao. Nerf- facelighting: Implicit and disentangled face lighting rep- resentation leveraging generative prior in neural radiance fields. ACM Transactions on Graphics , 42(3):1–18, 2023. 3

  22. [30]

    Analyzing and improv- ing the image quality of stylegan

    Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improv- ing the image quality of stylegan. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8110–8119, 2020. 2

  23. [31]

    A unified approach to prefiltered envi- ronment maps

    Jan Kautz, Pere-Pau V ´azquez, Wolfgang Heidrich, and Hans-Peter Seidel. A unified approach to prefiltered envi- ronment maps. In Rendering Techniques 2000: Proceedings of the Eurographics Workshop in Brno, Czech Republic, June 26–28, 2000 11, pages 185–196. Springer, 2000. 3

  24. [32]

    Realistic one-shot mesh-based head avatars

    Taras Khakhulin, Vanessa Sklyarova, Victor Lempitsky, and Egor Zakharov. Realistic one-shot mesh-based head avatars. In European Conference on Computer Vision , pages 345–

  25. [33]

    Sapiens: Foundation for human vision mod- els

    Rawal Khirodkar, Timur Bagautdinov, Julieta Martinez, Su Zhaoen, Austin James, Peter Selednik, Stuart Anderson, and Shunsuke Saito. Sapiens: Foundation for human vision mod- els. arXiv preprint arXiv:2408.12569, 2024. 1

  26. [34]

    Switchlight: Co-design of physics- driven architecture and pre-training framework for human portrait relighting

    Hoon Kim, Minje Jang, Wonjun Yoon, Jisoo Lee, Donghyun Na, and Sanghyun Woo. Switchlight: Co-design of physics- driven architecture and pre-training framework for human portrait relighting. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition,...

  27. [35]

    Gradient-based learning applied to document recog- nition

    Yann LeCun, L ´eon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recog- nition. Proceedings of the IEEE , 86(11):2278–2324, 1998. 2

  28. [36]

    One-shot high-fidelity talking- head synthesis with deformable neural radiance field

    Weichuang Li, Longhao Zhang, Dong Wang, Bin Zhao, Zhi- gang Wang, Mulin Chen, Bang Zhang, Zhongjian Wang, Liefeng Bo, and Xuelong Li. One-shot high-fidelity talking- head synthesis with deformable neural radiance field. InPro- ceedings of the IEEE/CVF Conference on Computer Vi...

  29. [37]

    Generalizable one-shot 3d neural head avatar

    Xueting Li, Shalini De Mello, Sifei Liu, Koki Nagano, Umar Iqbal, and Jan Kautz. Generalizable one-shot 3d neural head avatar. Advances in Neural Information Processing Systems, 36, 2024. 2

  30. [38]

    3d gan inversion for controllable portrait image animation

    Connor Z Lin, David B Lindell, Eric R Chan, and Gordon Wetzstein. 3d gan inversion for controllable portrait image animation. arXiv preprint arXiv:2203.13441, 2022. 6

  31. [39]

    Edgerelight360: Text- conditioned 360-degree hdr image generation for real-time on-device video portrait relighting

    Min-Hui Lin, Mahesh Reddy, Guillaume Berger, Michel Sarkis, Fatih Porikli, and Ning Bi. Edgerelight360: Text- conditioned 360-degree hdr image generation for real-time on-device video portrait relighting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern...

  32. [40]

    Otavatar: One-shot talking face avatar with control- lable tri-plane rendering

    Zhiyuan Ma, Xiangyu Zhu, Guo-Jun Qi, Zhen Lei, and Lei Zhang. Otavatar: One-shot talking face avatar with control- lable tri-plane rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 16901–16910, 2023. 2

  33. [41]

    Lightpainter: interactive por- trait relighting with freehand scribble

    Yiqun Mei, He Zhang, Xuaner Zhang, Jianming Zhang, Zhixin Shu, Yilin Wang, Zijun Wei, Shi Yan, HyunJoon Jung, and Vishal M Patel. Lightpainter: interactive por- trait relighting with freehand scribble. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Re...

  34. [42]

    Holo-relighting: Controllable volumetric portrait relighting from a single image

    Yiqun Mei, Yu Zeng, He Zhang, Zhixin Shu, Xuaner Zhang, Sai Bi, Jianming Zhang, HyunJoon Jung, and Vishal M Patel. Holo-relighting: Controllable volumetric portrait relighting from a single image. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recog...

  35. [43]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. In European conference on computer vision , pages 405–421. Springer, 2020. 2

  36. [44]

    Learning physics-guided face re- lighting under directional light

    Thomas Nestmeyer, Jean-Franc ¸ois Lalonde, Iain Matthews, and Andreas Lehrmann. Learning physics-guided face re- lighting under directional light. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5124–5133, 2020. 3

  37. [45]

    Total relighting: learning to relight portraits for background replacement

    Rohit Pandey, Sergio Orts-Escolano, Chloe Legendre, Chris- tian Haene, Sofien Bouaziz, Christoph Rhemann, Paul E De- bevec, and Sean Ryan Fanello. Total relighting: learning to relight portraits for background replacement. ACM Trans. Graph., 40(4):43–1, 2021. 3

  38. [46]

    Relightify: Re- lightable 3d faces from a single image via diffusion models

    Foivos Paraperas Papantoniou, Alexandros Lattas, Stylianos Moschoglou, and Stefanos Zafeiriou. Relightify: Re- lightable 3d faces from a single image via diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8806–8817, 2023. 3

  39. [47]

    Barron, Sofien Bouaziz, Dan B

    Keunhong Park, Utkarsh Sinha, Jonathan T. Barron, Sofien Bouaziz, Dan B. Goldman, Steven M. Seitz, and Ricardo Martin-Brualla. Nerfies: Deformable neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 5865–5874, 2021. 2

  40. [48]

    Barron, Sofien Bouaziz, Dan B

    Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T. Barron, Sofien Bouaziz, Dan B. Goldman, Ricardo Martin- Brualla, and Steven M. Seitz. Hypernerf: A higher- dimensional representation for topologically varying neural radiance fields. ACM Transactions on Graphics (TOG) , ...

  41. [49]

    Illumination for computer generated pic- tures

    Bui Tuong Phong. Illumination for computer generated pic- tures. In Seminal graphics: pioneering efforts that shaped the field, pages 95–101. 1998. 3

  42. [50]

    Difareli: Diffusion face relighting

    Puntawat Ponglertnapakorn, Nontawat Tritrong, and Supa- sorn Suwajanakorn. Difareli: Diffusion face relighting. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22646–22657, 2023. 3 10

  43. [51]

    Relitalk: Relightable talking portrait generation from a single video

    Haonan Qiu, Zhaoxi Chen, Yuming Jiang, Hang Zhou, Xi- angyu Fan, Lei Yang, Wayne Wu, and Ziwei Liu. Relitalk: Relightable talking portrait generation from a single video. International Journal of Computer Vision, pages 1–16, 2024. 3

  44. [52]

    An efficient represen- tation for irradiance environment maps

    Ravi Ramamoorthi and Pat Hanrahan. An efficient represen- tation for irradiance environment maps. In Proceedings of the 28th annual conference on Computer graphics and inter- active techniques, pages 497–500, 2001. 3

  45. [53]

    Facelit: Neural 3d relightable faces

    Anurag Ranjan, Kwang Moo Yi, Jen-Hao Rick Chang, and Oncel Tuzel. Facelit: Neural 3d relightable faces. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8619–8628, 2023. 3

  46. [54]

    Lite2relight: 3d-aware single image portrait relight- ing

    Pramod Rao, Gereon Fox, Abhimitra Meka, Mallikar- jun BR, Fangneng Zhan, Tim Weyrich, Bernd Bickel, Hanspeter Pfister, Wojciech Matusik, Mohamed Elgharib, et al. Lite2relight: 3d-aware single image portrait relight- ing. In ACM SIGGRAPH 2024 Conference Papers , pages 1–12, 2024. 3

  47. [55]

    Relightful harmonization: Lighting-aware portrait background replacement

    Mengwei Ren, Wei Xiong, Jae Shin Yoon, Zhixin Shu, Jianming Zhang, HyunJoon Jung, Guido Gerig, and He Zhang. Relightful harmonization: Lighting-aware portrait background replacement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6...

  48. [56]

    Adaptive perturbation learning for unsupervised dis- entangling of appearance and motion

    Yang Ren, Jie Liu, Xinwei Jiang, Xiaodan Liang, and Liang Lin. Adaptive perturbation learning for unsupervised dis- entangling of appearance and motion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14318–14327, 2021. 2

  49. [57]

    Relightable gaussian codec avatars

    Shunsuke Saito, Gabriel Schwartz, Tomas Simon, Junxuan Li, and Giljoo Nam. Relightable gaussian codec avatars. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 130–141, 2024. 3

  50. [58]

    Image de- formation using moving least squares

    Scott Schaefer, Travis McPhail, and Joe Warren. Image de- formation using moving least squares. In ACM SIGGRAPH 2006 Papers, pages 533–540. 2006. 2

  51. [59]

    First order motion model for image animation

    Aliaksandr Siarohin, St ´ephane Lathuili`ere, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe. First order motion model for image animation. Advances in neural information processing systems, 32, 2019. 2

  52. [60]

    Single image portrait relighting

    Tiancheng Sun, Jonathan T Barron, Yun-Ta Tsai, Zexiang Xu, Xueming Yu, Graham Fyffe, Christoph Rhemann, Jay Busch, Paul Debevec, and Ravi Ramamoorthi. Single image portrait relighting. ACM Transactions on Graphics (TOG) , 38(4):1–12, 2019. 3

  53. [61]

    Nelf: Neural light-transport field for portrait view synthesis and relighting

    Tiancheng Sun, Kai-En Lin, Sai Bi, Zexiang Xu, and Ravi Ramamoorthi. Nelf: Neural light-transport field for portrait view synthesis and relighting. arXiv preprint arXiv:2107.12351, 2021. 1, 3

  54. [62]

    Resolution-robust large mask inpainting with fourier convolutions

    Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky. Resolution-robust large mask inpainting with fourier convolutions. InProceedings of the IEEE/CVF winter...

  55. [63]

    3DFaceShop: Explic- itly controllable 3d-aware portrait generation

    Junshu Tang, Bo Zhang, Binxin Yang, Ting Zhang, Dong Chen, Lizhuang Ma, and Fang Wen. 3DFaceShop: Explic- itly controllable 3d-aware portrait generation. IEEE Trans- actions on Visualization and Computer Graphics, 2023. 2

  56. [64]

    Neural voice pup- petry: Audio-driven facial reenactment

    Justus Thies, Michael Zollh ¨ofer, Marc Stamminger, Chris- tian Theobalt, and Matthias Nießner. Neural voice pup- petry: Audio-driven facial reenactment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9382–9391, 2019. 1

  57. [65]

    Non-rigid neural radi- ance fields: Reconstruction and novel view synthesis of a deforming scene from monocular video

    Edgar Tretschk, Ayush Tewari, Vladislav Golyanik, Michael Zollh¨ofer, and Christian Theobalt. Non-rigid neural radi- ance fields: Reconstruction and novel view synthesis of a deforming scene from monocular video. In Proceedings of the IEEE/CVF International Conference on Compu...

  58. [66]

    Real-time radiance fields for single-image portrait view synthesis

    Alex Trevithick, Matthew Chan, Michael Stengel, Eric Chan, Chao Liu, Zhiding Yu, Sameh Khamis, Ravi Ramamoorthi, and Koki Nagano. Real-time radiance fields for single-image portrait view synthesis. 2023. 3, 4

  59. [67]

    Progressive disentangled representation learning for fine-grained controllable talking head synthesis

    Duomin Wang, Yu Deng, Zixin Yin, Heung-Yeung Shum, and Baoyuan Wang. Progressive disentangled representation learning for fine-grained controllable talking head synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17979–17989, ...

  60. [68]

    One-shot free-view neural talking-head synthesis for video conferenc- ing

    Ting-Chun Wang, Arun Mallya, and Ming-Yu Liu. One-shot free-view neural talking-head synthesis for video conferenc- ing. arXiv preprint arXiv:2011.15126, 2020. 1

  61. [69]

    Single image portrait relighting via explicit mul- tiple reflectance channel modeling

    Zhibo Wang, Xin Yu, Ming Lu, Quan Wang, Chen Qian, and Feng Xu. Single image portrait relighting via explicit mul- tiple reflectance channel modeling. ACM Transactions on Graphics (ToG), 39(6):1–13, 2020. 3

  62. [70]

    Perfor- mance relighting and reflectance transformation with time- multiplexed illumination

    Andreas Wenger, Andrew Gardner, Chris Tchou, Jonas Unger, Tim Hawkins, and Paul Debevec. Perfor- mance relighting and reflectance transformation with time- multiplexed illumination. ACM Transactions on Graphics (TOG), 24(3):756–764, 2005. 1, 3

  63. [71]

    Vfhq: A high-quality dataset and bench- mark for video face super-resolution

    Liangbin Xie, Xintao Wang, Honglun Zhang, Chao Dong, and Ying Shan. Vfhq: A high-quality dataset and bench- mark for video face super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 657–666, 2022. 6, 1

  64. [72]

    Omniavatar: Geometry-guided controllable 3d head synthesis

    Hongyi Xu, Guoxian Song, Zihang Jiang, Jianfeng Zhang, Yichun Shi, Jing Liu, Wanchun Ma, Jiashi Feng, and Lin- jie Luo. Omniavatar: Geometry-guided controllable 3d head synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pag...

  65. [73]

    PV3D: A 3d generative model for portrait video generation

    Zhongcong Xu, Jianfeng Zhang, Junhao Liew, Wenqing Zhang, Song Bai, Jiashi Feng, and Mike Zheng Shou. PV3D: A 3d generative model for portrait video generation. In Pro- ceedings of the Tenth International Conference on Learning Representations (ICLR), 2023. 2

  66. [74]

    Real3d-portrait: One-shot realistic 3d talking portrait synthesis

    Zhenhui Ye, Tianyun Zhong, Yi Ren, Jiaqi Yang, Weichuang Li, Jiawei Huang, Ziyue Jiang, Jinzheng He, Rongjie Huang, Jinglin Liu, et al. Real3d-portrait: One-shot realistic 3d talking portrait synthesis. arXiv preprint arXiv:2401.08503,

  67. [75]

    Learning to relight portrait images via a virtual light stage and synthetic-to-real adaptation

    Yu-Ying Yeh, Koki Nagano, Sameh Khamis, Jan Kautz, Ming-Yu Liu, and Ting-Chun Wang. Learning to relight portrait images via a virtual light stage and synthetic-to-real adaptation. ACM Transactions on Graphics (TOG) , 41(6): 1–21, 2022. 1, 3

  68. [76]

    Nofa: Nerf-based one-shot facial avatar recon- struction

    Wangbo Yu, Yanbo Fan, Yong Zhang, Xuan Wang, Fei Yin, Yunpeng Bai, Yan-Pei Cao, Ying Shan, Yang Wu, Zhongqian Sun, et al. Nofa: Nerf-based one-shot facial avatar recon- struction. In ACM SIGGRAPH 2023 Conference Proceed- ings, pages 1–12, 2023. 2

  69. [77]

    Dinet: Deformation inpainting network for realistic face visually dubbing on high resolution video

    Zhenyu Yu et al. Dinet: Deformation inpainting network for realistic face visually dubbing on high resolution video. arXiv preprint arXiv:2304.10212, 2023. 1

  70. [78]

    Few-shot adversarial learning of realistic neural talking head models

    Egor Zakharov, Aliaksandra Shysheya, Egor Burkov, and Victor Lempitsky. Few-shot adversarial learning of realistic neural talking head models. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 9459– 9468, 2019. 2

  71. [79]

    Neural video portrait relighting in real-time via con- sistency modeling

    Longwen Zhang, Qixuan Zhang, Minye Wu, Jingyi Yu, and Lan Xu. Neural video portrait relighting in real-time via con- sistency modeling. In Proceedings of the IEEE/CVF interna- tional conference on computer vision, pages 802–812, 2021. 3

  72. [80]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 6, 1

  73. [81]

    Flow-guided one-shot talking face generation with a high-resolution audio-visual dataset

    Zhimeng Zhang, Lincheng Li, Yu Ding, and Changjie Fan. Flow-guided one-shot talking face generation with a high-resolution audio-visual dataset. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3661–3670, 2021. 1

  74. [82]

    Deep single-image portrait relighting

    Hao Zhou, Sunil Hadap, Kalyan Sunkavalli, and David W Jacobs. Deep single-image portrait relighting. In Proceed- ings of the IEEE/CVF international conference on computer vision, pages 7194–7202, 2019. 3

  75. [83]

    3d deformation using moving least squares

    Yuanchen Zhu and Steven J Gortler. 3d deformation using moving least squares. 2007. 2, 4

  76. [84]

    Controllable radiance fields for dynamic face syn- thesis

    Peiye Zhuang, Liqian Ma, Sanmi Koyejo, and Alexander Schwing. Controllable radiance fields for dynamic face syn- thesis. In Proceedings of the 2022 International Conference on 3D Vision (3DV), 2022. 2 12 Total-Editing: Head Avatar with Editable Appearance, Motion, and Lighting...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.