REVIEW 3 major objections 5 minor 84 references
Total-Editing: Head Avatar with Editable Appearance, Motion, and Lighting
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A single neural radiance field pipeline edits a head avatar's appearance, motion, and lighting independently.
desk verdict Solid unified head-avatar system with a promising dataset, but the central relighting claim is never quantitatively validated and the architecture admits a real leakage channel. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is an intrinsically decomposed neural radiance field decoder. Its shading decoder splits the volume color into albedo, diffuse coefficient, specular coefficients, and a residual, then recombines them with the Phong reflection model, sampling diffuse and specular shading from pre-filtered lightmaps instead of regressing shading from appearance. The lightmaps are the lighting interface: HDR environment maps are baked into them directly, and a portrait image is converted into them by a transformer-based lightmap estimator trained on synthetic ground truth. The second mechanism is a moving-least-squares (MLS) deformation field derived from a parametric 3D face model, which warps tri-plane features and rotates decoded normals; MLS's globally averaged, distance-weighted transformations are what keep motion and shading spatially coherent.
What would settle it
Re-render one fixed identity through the same motion under two known HDR environment maps with very different directionality, and compare the estimated lightmaps and final shading against ground-truth images rendered under those same maps. If the estimated lightmaps fail to track the environment, or if a strong facial texture such as a beard remains dark under opposite lighting, the claimed albedo-shading disentanglement is not real.
Extended reading notes
Core claim
Total-Editing's central claim is that a neural radiance field can be decomposed into intrinsic components—density, surface normal, albedo, diffuse and specular shading coefficients, and a residual color—so that appearance, motion, and lighting are separately controllable in one synthesized 3D head. Diffuse and specular shading are not learned as part of the face texture; they are looked up from pre-filtered lightmaps, which are either baked from an HDR environment map or estimated from a portrait image by a lightmap estimator. Motion is carried by a moving-least-squares deformation field that warps volumetric features and rotates normals continuously, so shading follows the geometry as the head turns. The paper reports that this end-to-end design yields higher reconstruction quality, better identity preservation, and more accurate motion control than reenactment-only systems or two-stage reenactment-then-relighting pipelines, and it demonstrates illumination transfer, HDR relighting, and background replacement as applications.
Load-bearing premise
The lighting claim depends on the network truly separating permanent facial coloring (albedo) from light-and-shadow effects (shading); if the lightmap estimator instead reads texture such as a beard as shadow, the demonstrated lighting edits are correlations from training data, not genuine relighting.
Editorial extensions
If this is right
- A user can take one portrait for identity, another video for motion, and either a portrait or an HDR map for lighting, and synthesize a video where the shading shifts naturally as the head moves.
- Because reenactment and relighting are done in one pass, the compounding errors of two-stage reenactment-then-relighting pipelines are avoided.
- Lighting can be transferred from one portrait onto another person's animated head without retraining.
- The same representation supports placing an animated portrait into a new environment by baking that environment's HDR map and rendering matching backgrounds.
Reading between the lines
- The paper chooses its final training scheme by reenactment validation performance rather than by a direct measure of albedo-shading disentanglement; a dedicated relighting benchmark with ground-truth HDR illumination would be the direct check on the framework's strongest claim.
- The MLS deformation field is a generic warp, so the same mechanism could apply to articulated bodies or other objects where surface-field warping produces seams, not just heads.
- Because the lightmap estimator is pretrained on synthetic data with ground truth, retraining it on physics-based renderings of other object classes could extend this style of control beyond faces.
- One extension the paper does not develop is continuous control of light intensity or direction during animation, which would allow gradual lighting changes within a single motion sequence rather than one fixed environment.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Total-Editing, a unified portrait editing framework that combines one-shot face reenactment, portrait relighting, and intrinsic decomposition in a neural radiance field. The method takes an appearance image, a motion source, and a lighting source (either a portrait image or an HDR environment map) and synthesizes a head avatar with controllable appearance, motion, and lighting. The color is decomposed via the Phong reflection model into diffuse and specular terms using pre-filtered lightmaps, an MLS-based deformation field replaces the Surface Field used in prior work, and training is performed on a new synthetic dataset of roughly 2M frames with ground-truth albedo/normal/HDR, together with the VFHQ real dataset. Experiments report reenactment quality metrics, ablations on deformation fields and data/regularization choices, and qualitative demonstrations of relighting and background replacement.
Significance. If confirmed, the paper would provide a useful step toward a single 3D-aware pipeline for both reenactment and relighting, and the promised release of a 2M-frame synthetic dataset with ground-truth intrinsic decompositions and HDR environments is a valuable community asset. The architecture is clearly motivated, and the reenactment results improve over Portrait4D-v2 on several metrics (CSIM, AED, APD, FID in cross-reenactment). However, the central claim of precise and controllable lighting is not quantitatively established: all numeric tables measure reenactment quality, not relighting accuracy or albedo-shading disentanglement, and the acknowledged leakage risk (albedo misinterpreted as shading) is never measured. The overstatement of 'outperforms all other approaches' relative to Table 1 also requires correction. These gaps are load-bearing for the manuscript's main novelty, so the paper needs major revision.
major comments (3)
- [4.2, Table 1] The statement in Sec. 4.2 that 'our end-to-end method outperforms all other approaches in the face reenactment task' is not supported by Table 1 in the self-reenactment setting: GPAvatar achieves higher PSNR (20.7 vs. 20.3) and SSIM (0.753 vs. 0.730), while Total-Editing wins on LPIPS, CSIM, AED, and APD. Because no error bars or significance tests are reported, the averaged differences may not be meaningful, and the global claim is too strong. Please qualify the statement to the specific metrics and settings where the method actually leads, and discuss the trade-off with PSNR/SSIM.
- [4.3, Eq. (18), Supp. A] The paper's central novelty is lighting control, yet no quantitative evaluation of relighting accuracy or of albedo-shading disentanglement is provided. Tables 1-3 measure only reenactment quality; in Table 1 the lighting source is the appearance image, and in Table 3 the editing target is the motion frame itself (or a random frame from the same video), so lighting transfer is never compared against ground-truth relit images. The residual decoder delta_c in Eq. (18) can compensate for incorrect lightmaps, and on real data neither albedo nor lightmaps have ground-truth supervision (Supplement A states that L_a and L_S are applied only to synthetic data). Sec. 4.3 explicitly admits that without regularization the lightmap estimator can misinterpret albedo as shading, e.g., beards as shadows, yet the chosen regularization (Random Ilit, row 5 of Table 3) is selected by reenactment metrics rather than by a disentanglement measure. I recommend adding a quantitative relighting benchmark (e.g., light-stage data or a held-out synthetic test set with known HDR), a measure of albedo stability under varying lighting sources, and a comparison with at least one dedicated relighting baseline in the same setting.
- [3.4, 4.3] The dependence of the output on the lighting source is not directly measured anywhere in the paper. Because the lightmap estimator and the shading decoder are co-trained on real data without lighting ground truth, the model could in principle memorize correlations between the appearance image and the target frame rather than perform physical relighting. A direct disentanglement test would help: fix appearance and motion, vary only the lighting source (portrait image or HDR map), and report the pixel-wise or perceptual difference in the face region; conversely, fix lighting and motion and vary appearance to verify that the estimated lightmaps do not absorb identity. This experiment would make the 'precise control over lighting' claim falsifiable and is currently missing.
minor comments (5)
- [3.2, Eq. (12)] The MLS fall-off exponent alpha is fixed at 1.0 with no sensitivity analysis; a small sweep (e.g., 0.5, 1.0, 2.0) would help establish that the gains over Surface Fields are not tied to this particular choice.
- [3.1.1] The symbol n is used both for the surface normal and for the shininess exponent n in {1,16,32,64}, which is confusing in Eqs. (3) and (8); consider renaming the shininess exponent to m or q.
- [4.1, Table 3] In Sec. 4.1, 'resolution 5122' should be '512x512'; the caption of Table 3 does not define the meaning of the check marks in the Synthetic and Real columns.
- [4.3] The two paragraphs that begin 'Impact of ...' both reference 'Rows 1-3 of Tab. 3', which makes it easy to lose track of which rows are being discussed; renumber the rows in the table or restructure the text for clarity.
- [Supp. D] For the two-stage reenactment-relighting pipelines, the table would benefit from a footnote clarifying that the reported numbers are computed only on the valid cropped areas, since this may affect comparability with the end-to-end method.
Circularity Check
No significant circularity: the central reenactment claim is benchmarked against external baselines, and the lighting-control gap is a validation limitation, not a derivation that reduces to its inputs.
full rationale
Total-Editing does not claim to derive analytic results from first principles; it is an end-to-end learned system. The lighting path is either physically baked from HDR environment maps (Sec. 3.1.2) or estimated by a network from a portrait image (Sec. 3.4), and neither path is defined in terms of the final editing frames. The real-data training protocol sets the editing target to the motion source while taking the lighting source from another frame of the same video, so lighting transfer is not quantitatively validated against ground truth; this is a correctness and evaluation gap, not a circular reduction. The lightmap estimator is pre-trained with synthetic ground-truth lightmaps and normals (Eq. 22), and the albedo and lightmap losses are applied on synthetic data; this is supervised evidence from an external, self-created benchmark, not a fit-then-predict rename. The paper builds on prior work [16, 17, 67] by non-overlapping authors and compares against external baselines on VFHQ-Test, so the reenactment claim is independently supported. The acknowledged albedo/shading ambiguity in Sec. 4.3 is an identifiability concern, and choosing Row 5 of Table 3 by validation performance is a standard model-selection procedure rather than circularity. No equation equates a predicted quantity to a fitted input by construction, and no load-bearing argument relies on self-citation.
Assumptions & free parameters
free parameters (2)
- MLS fall-off exponent alpha =
1.0
- Shininess exponents for specular lightmaps =
{1, 16, 32, 64}
assumptions (6)
- domain assumption Phong reflection model with the ambient term omitted approximates face appearance under arbitrary lighting
- domain assumption Prefiltered Phong lightmaps preserve enough lighting detail for relighting
- ad hoc to paper The synthetic 2M-frame rendered dataset transfers to real portraits
- domain assumption Expression features from the off-the-shelf encoder are appearance-free enough for de-enactment and re-enactment
- standard math The standard volumetric rendering equation is valid for this head avatar setting
- standard math MLS rigid transformation solved via SVD yields continuous deformation fields
Cite this review
Pith. "Pith review of Total-Editing: Head Avatar with Editable Appearance, Motion, and Lighting." pith.science (2026). https://pith.science/paper/OVW5UU6X
@misc{pith2026250520582,
author = {Pith},
title = {Pith review of: Total-Editing: Head Avatar with Editable Appearance, Motion, and Lighting},
year = {2026},
howpublished = {\url{https://pith.science/paper/OVW5UU6X}},
note = {Machine review of arXiv:2505.20582}
}
read the original abstract
Face reenactment and portrait relighting are essential tasks in portrait editing, yet they are typically addressed independently, without much synergy. Most face reenactment methods prioritize motion control and multiview consistency, while portrait relighting focuses on adjusting shading effects. To take advantage of both geometric consistency and illumination awareness, we introduce Total-Editing, a unified portrait editing framework that enables precise control over appearance, motion, and lighting. Specifically, we design a neural radiance field decoder with intrinsic decomposition capabilities. This allows seamless integration of lighting information from portrait images or HDR environment maps into synthesized portraits. We also incorporate a moving least squares based deformation field to enhance the spatiotemporal coherence of avatar motion and shading effects. With these innovations, our unified framework significantly improves the quality and realism of portrait editing results. Further, the multi-source nature of Total-Editing supports more flexible applications, such as illumination transfer from one portrait to another, or portrait animation with customized backgrounds.
Figures
Figures from the paper (15 more)
Reference graph
Works this paper leans on
-
[1]
Styleflow: Attribute-conditioned exploration of stylegan- generated images using conditional continuous normalizing flows
Rameen Abdal, Peihao Zhu, Niloy J Mitra, and Peter Wonka. Styleflow: Attribute-conditioned exploration of stylegan- generated images using conditional continuous normalizing flows. ACM Transactions on Graphics (ToG) , 40(3):1–21,
-
[2]
Madhav Agarwal et al. Audio-visual face reenactment. arXiv preprint arXiv:2210.02755, 2022. 1
work page Pith review arXiv 2022
-
[3]
Rignerf: Fully controllable neu- ral 3d portraits
ShahRukh Athar, Zexiang Xu, Kalyan Sunkavalli, Eli Shechtman, and Zhixin Shu. Rignerf: Fully controllable neu- ral 3d portraits. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20364–20373, 2022. 2
2022
-
[4]
Generative neu- ral articulated radiance fields
Alexander Bergman, Petr Kellnhofer, Wang Yifan, Eric Chan, David Lindell, and Gordon Wetzstein. Generative neu- ral articulated radiance fields. Advances in Neural Informa- tion Processing Systems, 35:19900–19916, 2022. 2, 4, 5
2022
-
[5]
A morphable model for the synthesis of 3d faces
V olker Blanz and Thomas Vetter. A morphable model for the synthesis of 3d faces. In Proceedings of the 26th an- nual conference on Computer graphics and interactive tech- niques, pages 187–194. ACM Press/Addison-Wesley Pub- lishing Co., 1999. 2
1999
-
[6]
Real-time 3d-aware portrait video relighting
Ziqi Cai, Kaiwen Jiang, Shu-Yu Chen, Yu-Kun Lai, Hongbo Fu, Boxin Shi, and Lin Gao. Real-time 3d-aware portrait video relighting. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 6221–6231, 2024. 6, 7, 8, 2
work page 2024
-
[7]
Sira: Re- lightable avatars from a single image
Pol Caselles, Eduard Ramon, Jaime Garcia, Xavier Giro-i Nieto, Francesc Moreno-Noguer, and Gil Triginer. Sira: Re- lightable avatars from a single image. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 775–784, 2023. 3
work page 2023
-
[8]
Efficient geometry-aware 3d generative adversarial networks
Eric R Chan, Connor Z Lin, Matthew A Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas J Guibas, Jonathan Tremblay, Sameh Khamis, et al. Efficient geometry-aware 3d generative adversarial networks. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16123–16133, 2022. 5, 1
work page 2022
Show all 84 references
-
[9]
Generalizable and animatable gaussian head avatar
Xuangeng Chu and Tatsuya Harada. Generalizable and animatable gaussian head avatar. arXiv preprint arXiv:2410.07971, 2024. 2
2024 arXiv
-
[10]
Gpavatar: Generaliz- able and precise head avatar from image (s)
Xuangeng Chu, Yu Li, Ailing Zeng, Tianyu Yang, Lijian Lin, Yunfei Liu, and Tatsuya Harada. Gpavatar: Generaliz- able and precise head avatar from image (s). arXiv preprint arXiv:2401.10215, 2024. 2, 7, 8
2024 arXiv
-
[11]
Torrance
Robert L Cook and Kenneth E. Torrance. A reflectance model for computer graphics. ACM Transactions on Graph- ics (ToG), 1(1):7–24, 1982. 3
1982
-
[12]
Acquiring the reflectance field of a human face
Paul Debevec, Tim Hawkins, Chris Tchou, Haarm-Pieter Duiker, Westley Sarokin, and Mark Sagar. Acquiring the reflectance field of a human face. In Proceedings of the 27th annual conference on Computer graphics and interac- tive techniques, pages 145–156, 2000. 3
2000
-
[13]
Lu- migan: Unconditional generation of relightable 3d human faces
Boyang Deng, Yifan Wang, and Gordon Wetzstein. Lu- migan: Unconditional generation of relightable 3d human faces. In 2024 International Conference on 3D Vision (3DV), pages 302–312. IEEE, 2024. 3, 8
2024
-
[14]
Arcface: Additive angular margin loss for deep face recognition
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 4690–4699, 2019. 6, 1
2019
-
[15]
Accurate 3d face reconstruction with weakly-supervised learning: From single image to image set
Yu Deng, Jiaolong Yang, Sicheng Xu, Dong Chen, Yunde Jia, and Xin Tong. Accurate 3d face reconstruction with weakly-supervised learning: From single image to image set. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition workshops , pages 0–0...
2019
-
[16]
Portrait4d: Learning one-shot 4d head avatar synthesis using synthetic data
Yu Deng, Duomin Wang, Xiaohang Ren, Xingyu Chen, and Baoyuan Wang. Portrait4d: Learning one-shot 4d head avatar synthesis using synthetic data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7119–7130, 2024. 2, 3, 4, 5
2024
-
[17]
Portrait4d-v2: Pseudo multi-view data creates better 4d head synthesizer
Yu Deng, Duomin Wang, and Baoyuan Wang. Portrait4d-v2: Pseudo multi-view data creates better 4d head synthesizer. arXiv preprint arXiv:2403.13570, 2024. 2, 3, 4, 5, 6, 7, 8
2024 arXiv
-
[18]
Split: Single portrait lighting estimation via a tetrad of face intrinsics
Fan Fei, Yean Cheng, Yongjie Zhu, Qian Zheng, Si Li, Gang Pan, and Boxin Shi. Split: Single portrait lighting estimation via a tetrad of face intrinsics. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023. 3
2023
-
[19]
Dynamic neural radiance fields for monocular 4d facial avatar reconstruction
Guy Gafni, Justus Thies, Michael Zollh ¨ofer, and Matthias Nießner. Dynamic neural radiance fields for monocular 4d facial avatar reconstruction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8649–8658, 2021. 2
2021
-
[20]
Generative adversarial nets
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014. 2
2014
-
[21]
Project starline
Google. Project starline. https://en.wikipedia. org/wiki/Project_Starline, 2021. 1
2021
-
[22]
High- fidelity relightable monocular portrait animation with lighting-controllable video diffusion model
Mingtao Guo, Guanyu Xing, and Yanli Liu. High- fidelity relightable monocular portrait animation with lighting-controllable video diffusion model. arXiv preprint arXiv:2502.19894, 2025. 3
2025 arXiv
-
[23]
Realistic, hardware-accelerated shading and lighting
Wolfgang Heidrich and Hans-Peter Seidel. Realistic, hardware-accelerated shading and lighting. In Proceedings of the 26th annual conference on Computer graphics and in- teractive techniques, pages 171–178, 1999. 3
1999
-
[24]
Gans trained by a two time-scale update rule converge to a local nash equilib- rium
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 6
2017
-
[25]
Depth-aware generative adversarial network for talking head video generation
Fa-Ting Hong, Longhao Zhang, Li Shen, and Dan Xu. Depth-aware generative adversarial network for talking head video generation. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 3397–3406, 2022. 2
2022
-
[26]
Towards high fidelity face relight- ing with realistic shadows
Andrew Hou, Ze Zhang, Michel Sarkis, Ning Bi, Yiying Tong, and Xiaoming Liu. Towards high fidelity face relight- ing with realistic shadows. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 14719–14728, 2021. 3 9
2021
-
[27]
Face relighting with geometrically consistent shadows
Andrew Hou, Michel Sarkis, Ning Bi, Yiying Tong, and Xi- aoming Liu. Face relighting with geometrically consistent shadows. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4217–4226,
-
[28]
Geometry-aware single-image full-body human relight- ing
Chaonan Ji, Tao Yu, Kaiwen Guo, Jingxin Liu, and Yebin Liu. Geometry-aware single-image full-body human relight- ing. In European Conference on Computer Vision , pages 388–405. Springer, 2022. 3
2022
-
[29]
Nerf- facelighting: Implicit and disentangled face lighting rep- resentation leveraging generative prior in neural radiance fields
Kaiwen Jiang, Shu-Yu Chen, Hongbo Fu, and Lin Gao. Nerf- facelighting: Implicit and disentangled face lighting rep- resentation leveraging generative prior in neural radiance fields. ACM Transactions on Graphics , 42(3):1–18, 2023. 3
2023
-
[30]
Analyzing and improv- ing the image quality of stylegan
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improv- ing the image quality of stylegan. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8110–8119, 2020. 2
2020
-
[31]
A unified approach to prefiltered envi- ronment maps
Jan Kautz, Pere-Pau V ´azquez, Wolfgang Heidrich, and Hans-Peter Seidel. A unified approach to prefiltered envi- ronment maps. In Rendering Techniques 2000: Proceedings of the Eurographics Workshop in Brno, Czech Republic, June 26–28, 2000 11, pages 185–196. Springer, 2000. 3
2000
-
[32]
Realistic one-shot mesh-based head avatars
Taras Khakhulin, Vanessa Sklyarova, Victor Lempitsky, and Egor Zakharov. Realistic one-shot mesh-based head avatars. In European Conference on Computer Vision , pages 345–
-
[33]
Sapiens: Foundation for human vision mod- els
Rawal Khirodkar, Timur Bagautdinov, Julieta Martinez, Su Zhaoen, Austin James, Peter Selednik, Stuart Anderson, and Shunsuke Saito. Sapiens: Foundation for human vision mod- els. arXiv preprint arXiv:2408.12569, 2024. 1
2024 arXiv
-
[34]
Switchlight: Co-design of physics- driven architecture and pre-training framework for human portrait relighting
Hoon Kim, Minje Jang, Wonjun Yoon, Jisoo Lee, Donghyun Na, and Sanghyun Woo. Switchlight: Co-design of physics- driven architecture and pre-training framework for human portrait relighting. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition,...
2024
-
[35]
Gradient-based learning applied to document recog- nition
Yann LeCun, L ´eon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recog- nition. Proceedings of the IEEE , 86(11):2278–2324, 1998. 2
1998
-
[36]
One-shot high-fidelity talking- head synthesis with deformable neural radiance field
Weichuang Li, Longhao Zhang, Dong Wang, Bin Zhao, Zhi- gang Wang, Mulin Chen, Bang Zhang, Zhongjian Wang, Liefeng Bo, and Xuelong Li. One-shot high-fidelity talking- head synthesis with deformable neural radiance field. InPro- ceedings of the IEEE/CVF Conference on Computer Vi...
2023
-
[37]
Generalizable one-shot 3d neural head avatar
Xueting Li, Shalini De Mello, Sifei Liu, Koki Nagano, Umar Iqbal, and Jan Kautz. Generalizable one-shot 3d neural head avatar. Advances in Neural Information Processing Systems, 36, 2024. 2
2024
-
[38]
3d gan inversion for controllable portrait image animation
Connor Z Lin, David B Lindell, Eric R Chan, and Gordon Wetzstein. 3d gan inversion for controllable portrait image animation. arXiv preprint arXiv:2203.13441, 2022. 6
2022 arXiv
-
[39]
Edgerelight360: Text- conditioned 360-degree hdr image generation for real-time on-device video portrait relighting
Min-Hui Lin, Mahesh Reddy, Guillaume Berger, Michel Sarkis, Fatih Porikli, and Ning Bi. Edgerelight360: Text- conditioned 360-degree hdr image generation for real-time on-device video portrait relighting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern...
2024
-
[40]
Otavatar: One-shot talking face avatar with control- lable tri-plane rendering
Zhiyuan Ma, Xiangyu Zhu, Guo-Jun Qi, Zhen Lei, and Lei Zhang. Otavatar: One-shot talking face avatar with control- lable tri-plane rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 16901–16910, 2023. 2
2023
-
[41]
Lightpainter: interactive por- trait relighting with freehand scribble
Yiqun Mei, He Zhang, Xuaner Zhang, Jianming Zhang, Zhixin Shu, Yilin Wang, Zijun Wei, Shi Yan, HyunJoon Jung, and Vishal M Patel. Lightpainter: interactive por- trait relighting with freehand scribble. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Re...
2023
-
[42]
Holo-relighting: Controllable volumetric portrait relighting from a single image
Yiqun Mei, Yu Zeng, He Zhang, Zhixin Shu, Xuaner Zhang, Sai Bi, Jianming Zhang, HyunJoon Jung, and Vishal M Patel. Holo-relighting: Controllable volumetric portrait relighting from a single image. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recog...
2024
-
[43]
Nerf: Representing scenes as neural radiance fields for view syn- thesis
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. In European conference on computer vision , pages 405–421. Springer, 2020. 2
2020
-
[44]
Learning physics-guided face re- lighting under directional light
Thomas Nestmeyer, Jean-Franc ¸ois Lalonde, Iain Matthews, and Andreas Lehrmann. Learning physics-guided face re- lighting under directional light. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5124–5133, 2020. 3
2020
-
[45]
Total relighting: learning to relight portraits for background replacement
Rohit Pandey, Sergio Orts-Escolano, Chloe Legendre, Chris- tian Haene, Sofien Bouaziz, Christoph Rhemann, Paul E De- bevec, and Sean Ryan Fanello. Total relighting: learning to relight portraits for background replacement. ACM Trans. Graph., 40(4):43–1, 2021. 3
2021
-
[46]
Relightify: Re- lightable 3d faces from a single image via diffusion models
Foivos Paraperas Papantoniou, Alexandros Lattas, Stylianos Moschoglou, and Stefanos Zafeiriou. Relightify: Re- lightable 3d faces from a single image via diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8806–8817, 2023. 3
2023
-
[47]
Barron, Sofien Bouaziz, Dan B
Keunhong Park, Utkarsh Sinha, Jonathan T. Barron, Sofien Bouaziz, Dan B. Goldman, Steven M. Seitz, and Ricardo Martin-Brualla. Nerfies: Deformable neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 5865–5874, 2021. 2
2021
-
[48]
Barron, Sofien Bouaziz, Dan B
Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T. Barron, Sofien Bouaziz, Dan B. Goldman, Ricardo Martin- Brualla, and Steven M. Seitz. Hypernerf: A higher- dimensional representation for topologically varying neural radiance fields. ACM Transactions on Graphics (TOG) , ...
2021
-
[49]
Illumination for computer generated pic- tures
Bui Tuong Phong. Illumination for computer generated pic- tures. In Seminal graphics: pioneering efforts that shaped the field, pages 95–101. 1998. 3
1998
-
[50]
Difareli: Diffusion face relighting
Puntawat Ponglertnapakorn, Nontawat Tritrong, and Supa- sorn Suwajanakorn. Difareli: Diffusion face relighting. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22646–22657, 2023. 3 10
2023
-
[51]
Relitalk: Relightable talking portrait generation from a single video
Haonan Qiu, Zhaoxi Chen, Yuming Jiang, Hang Zhou, Xi- angyu Fan, Lei Yang, Wayne Wu, and Ziwei Liu. Relitalk: Relightable talking portrait generation from a single video. International Journal of Computer Vision, pages 1–16, 2024. 3
2024
-
[52]
An efficient represen- tation for irradiance environment maps
Ravi Ramamoorthi and Pat Hanrahan. An efficient represen- tation for irradiance environment maps. In Proceedings of the 28th annual conference on Computer graphics and inter- active techniques, pages 497–500, 2001. 3
2001
-
[53]
Facelit: Neural 3d relightable faces
Anurag Ranjan, Kwang Moo Yi, Jen-Hao Rick Chang, and Oncel Tuzel. Facelit: Neural 3d relightable faces. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8619–8628, 2023. 3
2023
-
[54]
Lite2relight: 3d-aware single image portrait relight- ing
Pramod Rao, Gereon Fox, Abhimitra Meka, Mallikar- jun BR, Fangneng Zhan, Tim Weyrich, Bernd Bickel, Hanspeter Pfister, Wojciech Matusik, Mohamed Elgharib, et al. Lite2relight: 3d-aware single image portrait relight- ing. In ACM SIGGRAPH 2024 Conference Papers , pages 1–12, 2024. 3
2024
-
[55]
Relightful harmonization: Lighting-aware portrait background replacement
Mengwei Ren, Wei Xiong, Jae Shin Yoon, Zhixin Shu, Jianming Zhang, HyunJoon Jung, Guido Gerig, and He Zhang. Relightful harmonization: Lighting-aware portrait background replacement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6...
2024
-
[56]
Adaptive perturbation learning for unsupervised dis- entangling of appearance and motion
Yang Ren, Jie Liu, Xinwei Jiang, Xiaodan Liang, and Liang Lin. Adaptive perturbation learning for unsupervised dis- entangling of appearance and motion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14318–14327, 2021. 2
2021
-
[57]
Relightable gaussian codec avatars
Shunsuke Saito, Gabriel Schwartz, Tomas Simon, Junxuan Li, and Giljoo Nam. Relightable gaussian codec avatars. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 130–141, 2024. 3
2024
-
[58]
Image de- formation using moving least squares
Scott Schaefer, Travis McPhail, and Joe Warren. Image de- formation using moving least squares. In ACM SIGGRAPH 2006 Papers, pages 533–540. 2006. 2
2006
-
[59]
First order motion model for image animation
Aliaksandr Siarohin, St ´ephane Lathuili`ere, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe. First order motion model for image animation. Advances in neural information processing systems, 32, 2019. 2
2019
-
[60]
Single image portrait relighting
Tiancheng Sun, Jonathan T Barron, Yun-Ta Tsai, Zexiang Xu, Xueming Yu, Graham Fyffe, Christoph Rhemann, Jay Busch, Paul Debevec, and Ravi Ramamoorthi. Single image portrait relighting. ACM Transactions on Graphics (TOG) , 38(4):1–12, 2019. 3
2019
-
[61]
Nelf: Neural light-transport field for portrait view synthesis and relighting
Tiancheng Sun, Kai-En Lin, Sai Bi, Zexiang Xu, and Ravi Ramamoorthi. Nelf: Neural light-transport field for portrait view synthesis and relighting. arXiv preprint arXiv:2107.12351, 2021. 1, 3
2021 arXiv
-
[62]
Resolution-robust large mask inpainting with fourier convolutions
Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky. Resolution-robust large mask inpainting with fourier convolutions. InProceedings of the IEEE/CVF winter...
2022
-
[63]
3DFaceShop: Explic- itly controllable 3d-aware portrait generation
Junshu Tang, Bo Zhang, Binxin Yang, Ting Zhang, Dong Chen, Lizhuang Ma, and Fang Wen. 3DFaceShop: Explic- itly controllable 3d-aware portrait generation. IEEE Trans- actions on Visualization and Computer Graphics, 2023. 2
2023
-
[64]
Neural voice pup- petry: Audio-driven facial reenactment
Justus Thies, Michael Zollh ¨ofer, Marc Stamminger, Chris- tian Theobalt, and Matthias Nießner. Neural voice pup- petry: Audio-driven facial reenactment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9382–9391, 2019. 1
2019
-
[65]
Non-rigid neural radi- ance fields: Reconstruction and novel view synthesis of a deforming scene from monocular video
Edgar Tretschk, Ayush Tewari, Vladislav Golyanik, Michael Zollh¨ofer, and Christian Theobalt. Non-rigid neural radi- ance fields: Reconstruction and novel view synthesis of a deforming scene from monocular video. In Proceedings of the IEEE/CVF International Conference on Compu...
2021
-
[66]
Real-time radiance fields for single-image portrait view synthesis
Alex Trevithick, Matthew Chan, Michael Stengel, Eric Chan, Chao Liu, Zhiding Yu, Sameh Khamis, Ravi Ramamoorthi, and Koki Nagano. Real-time radiance fields for single-image portrait view synthesis. 2023. 3, 4
2023
-
[67]
Progressive disentangled representation learning for fine-grained controllable talking head synthesis
Duomin Wang, Yu Deng, Zixin Yin, Heung-Yeung Shum, and Baoyuan Wang. Progressive disentangled representation learning for fine-grained controllable talking head synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17979–17989, ...
2023
-
[68]
One-shot free-view neural talking-head synthesis for video conferenc- ing
Ting-Chun Wang, Arun Mallya, and Ming-Yu Liu. One-shot free-view neural talking-head synthesis for video conferenc- ing. arXiv preprint arXiv:2011.15126, 2020. 1
2011 arXiv
-
[69]
Single image portrait relighting via explicit mul- tiple reflectance channel modeling
Zhibo Wang, Xin Yu, Ming Lu, Quan Wang, Chen Qian, and Feng Xu. Single image portrait relighting via explicit mul- tiple reflectance channel modeling. ACM Transactions on Graphics (ToG), 39(6):1–13, 2020. 3
2020
-
[70]
Perfor- mance relighting and reflectance transformation with time- multiplexed illumination
Andreas Wenger, Andrew Gardner, Chris Tchou, Jonas Unger, Tim Hawkins, and Paul Debevec. Perfor- mance relighting and reflectance transformation with time- multiplexed illumination. ACM Transactions on Graphics (TOG), 24(3):756–764, 2005. 1, 3
2005
-
[71]
Vfhq: A high-quality dataset and bench- mark for video face super-resolution
Liangbin Xie, Xintao Wang, Honglun Zhang, Chao Dong, and Ying Shan. Vfhq: A high-quality dataset and bench- mark for video face super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 657–666, 2022. 6, 1
2022
-
[72]
Omniavatar: Geometry-guided controllable 3d head synthesis
Hongyi Xu, Guoxian Song, Zihang Jiang, Jianfeng Zhang, Yichun Shi, Jing Liu, Wanchun Ma, Jiashi Feng, and Lin- jie Luo. Omniavatar: Geometry-guided controllable 3d head synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pag...
2023
-
[73]
PV3D: A 3d generative model for portrait video generation
Zhongcong Xu, Jianfeng Zhang, Junhao Liew, Wenqing Zhang, Song Bai, Jiashi Feng, and Mike Zheng Shou. PV3D: A 3d generative model for portrait video generation. In Pro- ceedings of the Tenth International Conference on Learning Representations (ICLR), 2023. 2
2023
-
[74]
Real3d-portrait: One-shot realistic 3d talking portrait synthesis
Zhenhui Ye, Tianyun Zhong, Yi Ren, Jiaqi Yang, Weichuang Li, Jiawei Huang, Ziyue Jiang, Jinzheng He, Rongjie Huang, Jinglin Liu, et al. Real3d-portrait: One-shot realistic 3d talking portrait synthesis. arXiv preprint arXiv:2401.08503,
-
[75]
Learning to relight portrait images via a virtual light stage and synthetic-to-real adaptation
Yu-Ying Yeh, Koki Nagano, Sameh Khamis, Jan Kautz, Ming-Yu Liu, and Ting-Chun Wang. Learning to relight portrait images via a virtual light stage and synthetic-to-real adaptation. ACM Transactions on Graphics (TOG) , 41(6): 1–21, 2022. 1, 3
2022
-
[76]
Nofa: Nerf-based one-shot facial avatar recon- struction
Wangbo Yu, Yanbo Fan, Yong Zhang, Xuan Wang, Fei Yin, Yunpeng Bai, Yan-Pei Cao, Ying Shan, Yang Wu, Zhongqian Sun, et al. Nofa: Nerf-based one-shot facial avatar recon- struction. In ACM SIGGRAPH 2023 Conference Proceed- ings, pages 1–12, 2023. 2
2023
-
[77]
Dinet: Deformation inpainting network for realistic face visually dubbing on high resolution video
Zhenyu Yu et al. Dinet: Deformation inpainting network for realistic face visually dubbing on high resolution video. arXiv preprint arXiv:2304.10212, 2023. 1
2023 arXiv
-
[78]
Few-shot adversarial learning of realistic neural talking head models
Egor Zakharov, Aliaksandra Shysheya, Egor Burkov, and Victor Lempitsky. Few-shot adversarial learning of realistic neural talking head models. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 9459– 9468, 2019. 2
2019
-
[79]
Neural video portrait relighting in real-time via con- sistency modeling
Longwen Zhang, Qixuan Zhang, Minye Wu, Jingyi Yu, and Lan Xu. Neural video portrait relighting in real-time via con- sistency modeling. In Proceedings of the IEEE/CVF interna- tional conference on computer vision, pages 802–812, 2021. 3
2021
-
[80]
The unreasonable effectiveness of deep features as a perceptual metric
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 6, 1
2018
-
[81]
Flow-guided one-shot talking face generation with a high-resolution audio-visual dataset
Zhimeng Zhang, Lincheng Li, Yu Ding, and Changjie Fan. Flow-guided one-shot talking face generation with a high-resolution audio-visual dataset. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3661–3670, 2021. 1
2021
-
[82]
Deep single-image portrait relighting
Hao Zhou, Sunil Hadap, Kalyan Sunkavalli, and David W Jacobs. Deep single-image portrait relighting. In Proceed- ings of the IEEE/CVF international conference on computer vision, pages 7194–7202, 2019. 3
2019
-
[83]
3d deformation using moving least squares
Yuanchen Zhu and Steven J Gortler. 3d deformation using moving least squares. 2007. 2, 4
2007
-
[84]
Controllable radiance fields for dynamic face syn- thesis
Peiye Zhuang, Liqian Ma, Sanmi Koyejo, and Alexander Schwing. Controllable radiance fields for dynamic face syn- thesis. In Proceedings of the 2022 International Conference on 3D Vision (3DV), 2022. 2 12 Total-Editing: Head Avatar with Editable Appearance, Motion, and Lighting...
2022
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.