REVIEW 4 major objections 4 minor 94 references
A two-stage model can relight any human head from flat-lit multi-view images or a single photo, without any light-stage capture of that person, by first estimating a flat-lit 3D Gaussian avatar and then mapping it to physically based reflec
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 12:18 UTC pith:Q4Z2ONKH
load-bearing objection The two-stage flat-lit-to-relightable idea is the real contribution; the cross-dataset lighting code is plausible but under-evidenced, and the 'any subject' claim outruns the experiments. the 4 major comments →
RelightAnyone: A Generalized Relightable 3D Gaussian Head Model
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper's central claim is that any subject from single- or multi-view images can be relit without OLAT data for that subject, by learning a mapping from flat-lit 3DGS avatars to relightable Gaussian parameters. The transfer works because the stages are decoupled: Stage 1 builds a multi-identity prior of heads under fixed lighting using an identity code and a per-dataset lighting code; Stage 2 reads reflectance parameters off the flat-lit colors from only 116 OLAT subjects. Two regularizers (albedo near the mean full-on texture, diffuse SH near monochromatic) keep the albedo/shading split meaningful. On the OLAT test split this design reaches 30.06 PSNR versus 25.49 single-stage.
What carries the argument
The load-bearing object is the per-dataset lighting code z_l in R^4. In Stage 1, the color decoder takes this code together with the identity code, so each dataset's particular flat illumination is factored into the code rather than baked into the identity representation; swapping codes re-renders the same subject under a different dataset's lighting, and the code assigned to the OLAT dataset D1 produces the exact full-on color condition the Stage 2 network was trained on. The second piece is the Stage 2 UNet: a shared encoder with two decoder branches predicts view-independent reflectance (diffuse albedo, spherical-harmonic radiance transfer, specular roughness) and view-dependent quantitie
Load-bearing premise
The load-bearing premise is that a single learned four-number lighting code per dataset can make subjects captured under very different rigs — differing light positions and spectra, camera color responses, and background bounce light — look to the Stage 2 network exactly as if all had been lit by D1's full-on condition; if a dataset's lighting cannot be absorbed into those four numbers, the flat-lit-to-reflectance mapping breaks for every subject in that dataset.
What would settle it
Capture a new subject once under flat-lit multi-view conditions and once under OLAT in a light stage — the paper notes in Section 4.3 that no such relit ground truth exists for its non-OLAT datasets, so its cross-dataset relighting claims currently rest on qualitative renders. Fit the model from the flat-lit frames alone, render under the measured OLAT light positions, and compare pixel-wise against the true one-light-at-a-time images. If the relit renders fail to reproduce hard shadows, specular highlights, and identity-correct albedo — or if the lighting codes cannot align a rig with strongl
If this is right
- Any subject appearing in a flat-lit multi-view dataset can be relit under arbitrary environment maps, including harsh outdoor light, without ever being captured under OLAT illumination.
- Flat-lit datasets can be converted into synthetic OLAT renders by passing their avatars through Stage 2, expanding the pool of relightable training data without new light-stage captures.
- A single in-the-wild photograph of an unseen person suffices to fit a view-consistent, relightable 3D Gaussian avatar, with appearance constrained by the learned multi-identity prior.
- The amount of expensive OLAT data required is small: the authors show the Stage 2 mapping already produces correct lighting when trained on 20 OLAT subjects, with more subjects improving albedo and identity fidelity.
- Because the identity prior is trained on cheap flat-lit data and the reflectance mapping on a small calibrated set, each stage can be scaled independently as new datasets become available.
Where Pith is reading between the lines
- Cross-dataset relighting is demonstrated qualitatively; the paper states no relit ground truth exists for the non-OLAT datasets (Section 4.3). The decisive test would be to capture a new subject in both a flat-lit multi-view rig and a light stage, fit from the flat-lit frames, and compare pixel-wise against the measured OLAT images.
- The paper's own failure cases — long hair becoming a texture-less cloud, blurred headscarf patterns, glasses without speculars, and neutral expressions only — all sit where the small OLAT set has no coverage, suggesting the transfer is reliable for appearance classes represented in the calibrated set, not for arbitrary materials or dynamics (Section 5 and supplementary).
- A four-number lighting code is a small lever for aligning rigs that differ in light positions, spectra, camera color response, and bounce light; probing its limits with a strongly colored or highly directional lighting rig would show how far self-supervised alignment can be pushed.
- The recipe of factoring per-dataset nuisance into a tiny auxiliary code and then learning a transfer from aligned appearance to physics-based parameters on a small calibrated set is portable to other appearance domains, such as bodies or hands, that share the same cost asymmetry between flat-lit and calibrated capture.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes RelightAnyone, a two-stage system for reconstructing and relighting head avatars from multi-view or single-image inputs without requiring OLAT (one-light-at-a-time) captures for the target subject. Stage 1 trains a multi-identity 3D Gaussian head model on several flat-lit datasets, using a learned low-dimensional dataset-specific lighting code to align different illumination conditions. Stage 2 trains a UNet that maps full-on Gaussian colors from Stage 1 to relightable RGCA parameters, using only a comparatively small OLAT dataset. The authors claim that the resulting model can relight any subject from a single image or multiple images, generalizing across datasets. Quantitative evaluations are provided for the OLAT dataset D1 (11 held-out subjects), comparing against 3D GAN-based baselines, along with ablations validating the two-stage design, the lighting code, and regularization terms. Qualitative results are shown for subjects from other datasets and in-the-wild images.
Significance. If the two-stage framework works as claimed, it would be an important practical step toward relightable avatars from easy-to-acquire flat-lit multi-view data, avoiding the expensive and time-consuming large-scale OLAT acquisition required by methods such as URAvatar. The core idea—learning a dataset-specific lighting code for self-supervised illumination alignment and a separate mapping from flat-lit to relightable parameters—is novel and well-motivated. The D1 results are strong (Table 2: PSNR 29.07 multi-view vs. 3DPR's 21.02), and the ablations (two-stage vs. single-stage, Lρ/Lmono, OLAT dataset size) are informative. However, the central generalization claim is currently supported only by qualitative evidence on subjects from non-OLAT datasets and in-the-wild images; no quantitative evaluation is provided for held-out subjects from D2–D4, which is a significant gap.
major comments (4)
- [4.2 Qualitative Results / Fig. 3] The multi-view generalization results are demonstrated explicitly on "training subjects from the flat-lit datasets (D2, D3, D4)" (Section 4.2). Because these identities are already seen by Stage 1, they do not test the ability to fit new identities from these datasets, which is the central claim of the paper. The only held-out identities are the 11 D1 test subjects, all from the same OLAT dataset. To support the claim that Stage 1 generalizes across identities from flat-lit datasets, the authors should hold out a subset of D2–D4 subjects and report fitting/relighting results on them.
- [3.2 Eq. (8), 4.3 Fig. 7/12] The dataset-specific lighting code z_l is trained with each subject observed under only one illumination. The Stage 1 reconstruction loss cannot enforce that swapping z_l(D2) to z_l(D1) produces the same subject under D1's full-on lighting; the network may simply encode dataset-specific color transforms. The paper itself states that "relit ground truth is not available for these subjects" and therefore gives only qualitative examples. A quantitative validation of the lighting alignment is needed, e.g., by holding out D1 subjects, synthesizing D2-like lighting from OLAT data, and measuring the residual after code swapping.
- [4.4 Table 2] All quantitative comparisons are restricted to the 11 D1 test subjects. The in-the-wild claims are supported only by qualitative examples (Fig. 3, rows 6-7). The reporting of Table 2 does not specify the exact input protocol for "Ours (single image)" vs. "Ours (multi-view)" (e.g., whether the same input image is used as the baselines, how many fitting iterations, and the environment map set). This makes it difficult to assess the fairness and generalizability of the reported gains. Please provide the protocol and ideally a quantitative in-the-wild evaluation or user study.
- [3.3 Model Fitting / 4.4 Comparisons] The comparison protocol for Table 2 is under-specified: it is unclear how the single-image and multi-view variants are fitted, whether the baselines use the same test images, and how the ground-truth relit images (image-based relighting) are generated. Since the baselines are single-image methods, the multi-view variant may be at an advantage, while the single-image variant's results depend heavily on the fitting procedure. Please provide the exact evaluation setup and consider reporting per-subject results.
minor comments (4)
- [4.1 Datasets] The text says "we processed the neutral expression for 127 subjects (116 for training and 11 for testing)". Please clarify whether these 11 test subjects are exactly the same as those used in the 3DPR paper's test set, and describe the split criterion.
- [Eq. (4)] The Gaussian color cf_k in Eq. (4) is not explicitly defined in the main text before Eq. (8); it is helpful to define it as the color predicted by Dc(zid, zl) under a specific lighting condition.
- [3.2, Eq. (9)] The loss terms Ll1, Lssim, Lgeo, Ls, and Lt are not all defined in the main text. A brief definition or a pointer to the supplementary would improve readability.
- [4.3 Lighting Code Ablation] In the ablation, the "without z_l" model uses Dc(zid) only, but the figure (Fig. 7) shows artifacts from across datasets. It would be clearer to explicitly state that in this ablation all datasets share the same decoder without a lighting code, and that no color calibration is applied for D2/D3/D4 in that case.
Circularity Check
No circularity found; the derivation is a learned two-stage mapping supervised by held-out OLAT ground truth, not a reduction of outputs to inputs.
full rationale
The paper's derivation chain is not circular by the standards of this review. Stage 1 (Eqs. 6-8) learns flat-lit Gaussian parameters from multi-view images with a dataset-specific lighting code z_l; Stage 2 (Eqs. 10-11) is a UNet that maps those full-on colors to RGCA relightable parameters. The Stage 2 mapping is not defined in terms of its own output: it is trained with an L1/SSIM loss against ground-truth OLAT renders of D1 subjects, and the central relighting claim is evaluated on held-out D1 test subjects in Table 1 and Table 2. The z_l code is optimized under the Stage 1 reconstruction loss (Eq. 9), not fitted to any relit ground truth, so the claim that swapping z_l aligns D2-D4 to D1 is an empirical extrapolation that could fail due to domain shift, not a tautology enforced by construction. No equation reduces the predicted relightable parameters to the input flat colors, and no fitted value is renamed as a prediction. The L_rho and L_mono regularizers constrain the albedo/shading decomposition but do not force the final relit render to equal any input; the render loss on OLAT ground truth remains the dominant supervision, with L_rho annealed from 10 to 0.01. There is no load-bearing self-citation chain: RGCA is used as an external, published parameterization, and the paper's own contribution is a new two-stage training scheme. The lack of quantitative relighting metrics on D2-D4 subjects is a validation gap and a correctness risk, not evidence of circularity. Therefore the appropriate finding is no significant circularity.
Axiom & Free-Parameter Ledger
free parameters (4)
- dataset-specific lighting code z_l =
4-dim vector, one per dataset (D1-D4)
- per-subject identity code z_id =
256-dim latent, one per subject; initialized at mean and optimized in fitting
- D2 color calibration matrix =
optimized 3x3 color correction matrix
- Regularization weights (lambda_rho, lambda_mono, and other stage losses) =
lambda_rho annealed 10->0.01; lambda_mono=0.01; lambda_l1=10; lambda_ssim=0.2; etc.
axioms (4)
- domain assumption The flat-lit (full-on) appearance of a head avatar is a sufficient intermediate representation for predicting relightable RGCA parameters.
- ad hoc to paper A single 4-dimensional dataset-specific lighting code z_l can fully align the illumination of each flat-lit dataset to the OLAT dataset's full-on condition.
- domain assumption The D1 OLAT training set (127 neutral subjects, 331 point lights) provides a universal relightable prior that generalizes to any identity, hairstyle, and accessory.
- domain assumption RGCA's reflectance parameterization (SH diffuse radiance transfer plus spherical-Gaussian specular with per-Gaussian visibility/normals) is sufficient to represent the appearance of heads and hair.
read the original abstract
3D Gaussian Splatting (3DGS) has become a standard approach to reconstruct and render photorealistic 3D head avatars. A major challenge is to relight the avatars to match any scene illumination. For high quality relighting, existing methods require subjects to be captured under complex time-multiplexed illumination, such as one-light-at-a-time (OLAT). We propose a new generalized relightable 3D Gaussian head model that can relight any subject observed in a single- or multi-view images without requiring OLAT data for that subject. Our core idea is to learn a mapping from flat-lit 3DGS avatars to corresponding relightable Gaussian parameters for that avatar. Our model consists of two stages: a first stage that models flat-lit 3DGS avatars without OLAT lighting, and a second stage that learns the mapping to physically-based reflectance parameters for high-quality relighting. This two-stage design allows us to train the first stage across diverse existing multi-view datasets without OLAT lighting ensuring cross-subject generalization, where we learn a dataset-specific lighting code for self-supervised lighting alignment. Subsequently, the second stage can be trained on a significantly smaller dataset of subjects captured under OLAT illumination. Together, this allows our method to generalize well and relight any subject from the first stage as if we had captured them under OLAT lighting. Furthermore, we can fit our model to unseen subjects from as little as a single image, allowing several applications in novel view synthesis and relighting for digital avatars.
Figures
Reference graph
Works this paper leans on
-
[1]
ScaffoldAvatar: High-fidelity gaussian avatars with patch expressions
Shivangi Aneja, Sebastian Weiss, Irene Baeza, Prashanth Chandran, Gaspard Zoss, Matthias Niessner, and Derek Bradley. ScaffoldAvatar: High-fidelity gaussian avatars with patch expressions. InProceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers, pages 1–11, New York, NY , USA, 2025. ACM. 1
2025
-
[2]
High-res facial appearance cap- ture from polarized smartphone images
Dejan Azinovi´c, Olivier Maury, Christophe Hery, Matthias Nießner, and Justus Thies. High-res facial appearance cap- ture from polarized smartphone images. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16836–16846, 2023. 3
2023
-
[3]
Shrisha Bharadwaj, Yufeng Zheng, Otmar Hilliges, Michael J Black, and Victoria Fernandez-Abrevaya. Flare: Fast learning of animatable and relightable mesh avatars.arXiv preprint arXiv:2310.17519, 2023. 3
Pith/arXiv arXiv 2023
-
[4]
Deep relightable appearance models for animatable faces.ACM Transactions on Graphics (ToG), 40(4):1–15, 2021
Sai Bi, Stephen Lombardi, Shunsuke Saito, Tomas Simon, Shih-En Wei, Kevyn Mcphail, Ravi Ramamoorthi, Yaser Sheikh, and Jason Saragih. Deep relightable appearance models for animatable faces.ACM Transactions on Graphics (ToG), 40(4):1–15, 2021. 2
2021
-
[5]
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023. 2
Pith/arXiv arXiv 2023
-
[6]
Align your latents: High-resolution video synthesis with la- tent diffusion models
Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dock- horn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with la- tent diffusion models. InProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pages 22563–22575, 2023. 2
2023
-
[7]
Preface: A data-driven volumetric prior for few-shot ultra high-resolution face synthesis
Marcel C B ¨uhler, Kripasindhu Sarkar, Tanmay Shah, Gengyan Li, Daoye Wang, Leonhard Helminger, Sergio Orts-Escolano, Dmitry Lagun, Otmar Hilliges, Thabo Beeler, et al. Preface: A data-driven volumetric prior for few-shot ultra high-resolution face synthesis. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 3402–3413, 2023. 5
2023
-
[8]
Efficient geometry-aware 3d generative adversarial networks
Eric R Chan, Connor Z Lin, Matthew A Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas J Guibas, Jonathan Tremblay, Sameh Khamis, et al. Efficient geometry-aware 3d generative adversarial networks. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16123–16133, 2022. 3
2022
-
[9]
Semantic deep face models
Prashanth Chandran, Derek Bradley, Markus Gross, and Thabo Beeler. Semantic deep face models. In2020 interna- tional conference on 3D vision (3DV), pages 345–354. IEEE,
-
[10]
Acquiring the reflectance field of a human face
Paul Debevec, Tim Hawkins, Chris Tchou, Haarm-Pieter Duiker, Westley Sarokin, and Mark Sagar. Acquiring the reflectance field of a human face. InProceedings of the 27th annual conference on Computer graphics and interactive techniques, pages 145–156, 2000. 1, 2, 8
2000
-
[11]
Lumigan: Unconditional generation of relightable 3d human faces
Boyang Deng, Yifan Wang, and Gordon Wetzstein. Lumigan: Unconditional generation of relightable 3d human faces. In 2024 International Conference on 3D Vision (3DV), pages 302–312. IEEE, 2024. 3
2024
-
[12]
Diffusion models beat gans on image synthesis.Advances in neural information processing systems, 34:8780–8794, 2021
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis.Advances in neural information processing systems, 34:8780–8794, 2021. 2
2021
-
[13]
Practical face reconstruction via differentiable ray tracing
Abdallah Dib, Gaurav Bharaj, Junghyun Ahn, C ´edric Th´ebault, Philippe Gosselin, Marco Romeo, and Louis Chevallier. Practical face reconstruction via differentiable ray tracing. InComputer Graphics Forum, pages 153–164. Wiley Online Library, 2021. 3
2021
-
[14]
Towards high fidelity monocular face reconstruction with rich reflectance using self-supervised learning and ray tracing
Abdallah Dib, Cedric Thebault, Junghyun Ahn, Philippe- Henri Gosselin, Christian Theobalt, and Louis Chevallier. Towards high fidelity monocular face reconstruction with rich reflectance using self-supervised learning and ray tracing. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12819–12829, 2021. 3
2021
-
[15]
Abdallah Dib, Junghyun Ahn, Cedric Thebault, Philippe- Henri Gosselin, and Louis Chevallier. S2f2: Self-supervised high fidelity face reconstruction from monocular image.arXiv preprint arXiv:2203.07732, 2022. 3
Pith/arXiv arXiv 2022
-
[16]
Near-instant capture of high-resolution facial geometry and reflectance
Graham Fyffe, Paul Graham, Borom Tunwattanapong, Ab- hijeet Ghosh, and Paul Debevec. Near-instant capture of high-resolution facial geometry and reflectance. InComputer Graphics Forum, pages 353–363. Wiley Online Library, 2016. 2
2016
-
[17]
Multiview face capture using polarized spherical gradient illumination
Abhijeet Ghosh, Graham Fyffe, Borom Tunwattanapong, Jay Busch, Xueming Yu, and Paul Debevec. Multiview face capture using polarized spherical gradient illumination. In Proceedings of the 2011 SIGGRAPH Asia Conference, pages 1–10, 2011
2011
-
[18]
The re- lightables: V olumetric performance capture of humans with realistic relighting.ACM Transactions on Graphics (ToG), 38 (6):1–19, 2019
Kaiwen Guo, Peter Lincoln, Philip Davidson, Jay Busch, Xueming Yu, Matt Whalen, Geoff Harvey, Sergio Orts- Escolano, Rohit Pandey, Jason Dourgarian, et al. The re- lightables: V olumetric performance capture of humans with realistic relighting.ACM Transactions on Graphics (ToG), 38 (6):1–19, 2019. 2
2019
-
[19]
Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yao- hui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning.arXiv preprint arXiv:2307.04725, 2023. 2
Pith/arXiv arXiv 2023
-
[20]
High-quality facial geometry and appearance capture at home
Yuxuan Han, Junfeng Lyu, and Feng Xu. High-quality facial geometry and appearance capture at home. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 697–707, 2024. 3
2024
-
[21]
Facial appearance capture at home with patch-level reflectance prior.ACM Transactions on Graphics (TOG), 44(4):1–16, 2025
Yuxuan Han, Junfeng Lyu, Kuan Sheng, Minghao Que, Qix- uan Zhang, Lan Xu, and Feng Xu. Facial appearance capture at home with patch-level reflectance prior.ACM Transactions on Graphics (TOG), 44(4):1–16, 2025. 3
2025
-
[22]
Unirelight: Learn- ing joint decomposition and synthesis for video relighting
Kai He, Ruofan Liang, Jacob Munkberg, Jon Hasselgren, Nandita Vijaykumar, Alexander Keller, Sanja Fidler, Igor Gilitschenski, Zan Gojcic, and Zian Wang. Unirelight: Learn- ing joint decomposition and synthesis for video relighting. arXiv preprint arXiv:2506.15673, 2025. 2
Pith/arXiv arXiv 2025
-
[23]
Diffrelight: Diffusion-based 9 facial performance relighting
Mingming He, Pascal Clausen, Ahmet Levent Tas ¸el, Li Ma, Oliver Pilarski, Wenqi Xian, Laszlo Rikker, Xueming Yu, Ryan Burgert, Ning Yu, et al. Diffrelight: Diffusion-based 9 facial performance relighting. InSIGGRAPH Asia 2024 Con- ference Papers, pages 1–12, 2024. 2
2024
-
[24]
Geometry-aware single-image full-body human relighting
Chaonan Ji, Tao Yu, Kaiwen Guo, Jingxin Liu, and Yebin Liu. Geometry-aware single-image full-body human relighting. InEuropean conference on computer vision, pages 388–405. Springer, 2022. 2
2022
-
[25]
Nerf- facelighting: Implicit and disentangled face lighting repre- sentation leveraging generative prior in neural radiance fields
Kaiwen Jiang, Shu-Yu Chen, Hongbo Fu, and Lin Gao. Nerf- facelighting: Implicit and disentangled face lighting repre- sentation leveraging generative prior in neural radiance fields. ACM Transactions on Graphics, 42(3):1–18, 2023. 3, 7, 8, 1
2023
-
[26]
A unified approach to prefiltered environment maps
Jan Kautz, Pere-Pau V´azquez, Wolfgang Heidrich, and Hans- Peter Seidel. A unified approach to prefiltered environment maps. InEurographics Workshop on Rendering Techniques, pages 185–196. Springer, 2000. 3
2000
-
[27]
3d gaussian splatting for real-time radiance field rendering.ACM Trans
Bernhard Kerbl, Georgios Kopanas, Thomas Leimk¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM Trans. Graph., 42(4):139–1, 2023. 1, 3
2023
-
[28]
Haircup: Hair compositional universal prior for 3d gaussian avatars
Byungjun Kim, Shunsuke Saito, Giljoo Nam, Tomas Simon, Jason Saragih, Hanbyul Joo, and Junxuan Li. Haircup: Hair compositional universal prior for 3d gaussian avatars. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9966–9976, 2025. 8
2025
-
[29]
Switchlight: Co-design of physics- driven architecture and pre-training framework for human portrait relighting
Hoon Kim, Minje Jang, Wonjun Yoon, Jisoo Lee, Donghyun Na, and Sanghyun Woo. Switchlight: Co-design of physics- driven architecture and pre-training framework for human portrait relighting. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR), pages 25096–25106, 2024. 2
2024
-
[30]
Nersemble: Multi-view radi- ance field reconstruction of human heads.ACM Trans
Tobias Kirschstein, Shenhan Qian, Simon Giebenhain, Tim Walter, and Matthias Nießner. Nersemble: Multi-view radi- ance field reconstruction of human heads.ACM Trans. Graph., 42(4), 2023. 2, 3, 5
2023
-
[31]
Lightit: Illumination modeling and control for diffusion models
Peter Kocsis, Julien Philip, Kalyan Sunkavalli, Matthias Nießner, and Yannick Hold-Geoffroy. Lightit: Illumination modeling and control for diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9359–9369, 2024. 2
2024
-
[32]
Avatarme: Realistically renderable 3d facial reconstruction” in-the-wild”
Alexandros Lattas, Stylianos Moschoglou, Baris Gecer, Stylianos Ploumpis, Vasileios Triantafyllou, Abhijeet Ghosh, and Stefanos Zafeiriou. Avatarme: Realistically renderable 3d facial reconstruction” in-the-wild”. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 760–769, 2020. 3
2020
-
[33]
Avatarme++: Facial shape and brdf inference with photorealistic rendering-aware gans.IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12):9269– 9284, 2021
Alexandros Lattas, Stylianos Moschoglou, Stylianos Ploumpis, Baris Gecer, Abhijeet Ghosh, and Stefanos Zafeiriou. Avatarme++: Facial shape and brdf inference with photorealistic rendering-aware gans.IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12):9269– 9284, 2021. 3
2021
-
[34]
Practical and scalable desktop-based high- quality facial capture
Alexandros Lattas, Yiming Lin, Jayanth Kannan, Ekin Ozturk, Luca Filipi, Giuseppe Claudio Guarnera, Gaurav Chawla, and Abhijeet Ghosh. Practical and scalable desktop-based high- quality facial capture. InEuropean Conference on Computer Vision, pages 522–537. Springer, 2022. 3
2022
-
[35]
Uravatar: Universal relightable gaussian codec avatars
Junxuan Li, Chen Cao, Gabriel Schwartz, Rawal Khirodkar, Christian Richardt, Tomas Simon, Yaser Sheikh, and Shun- suke Saito. Uravatar: Universal relightable gaussian codec avatars. InSIGGRAPH Asia 2024 Conference Papers, pages 1–11, 2024. 2, 3, 5, 6, 7
2024
-
[36]
Diffusionrenderer: Neural inverse and forward rendering with video diffusion models
Ruofan Liang, Zan Gojcic, Huan Ling, Jacob Munkberg, Jon Hasselgren, Zhi-Hao Lin, Jun Gao, Alexander Keller, Nandita Vijaykumar, Sanja Fidler, and Zian Wang. Diffusionrenderer: Neural inverse and forward rendering with video diffusion models. InThe IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2025. 2, 8
2025
-
[37]
Single-shot implicit morphable faces with consistent texture parameterization
Connor Lin, Koki Nagano, Jan Kautz, Eric Chan, Umar Iqbal, Leonidas Guibas, Gordon Wetzstein, and Sameh Khamis. Single-shot implicit morphable faces with consistent texture parameterization. InACM SIGGRAPH 2023 Conference Proceedings, pages 1–12, 2023. 3
2023
-
[38]
Real-time high-resolution background matting
Shanchuan Lin, Andrey Ryabtsev, Soumyadip Sengupta, Brian L Curless, Steven M Seitz, and Ira Kemelmacher- Shlizerman. Real-time high-resolution background matting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8762–8771, 2021. 5
2021
-
[39]
Mixture of volumetric primitives for efficient neural rendering.ACM Transactions on Graphics (ToG), 40(4):1–13, 2021
Stephen Lombardi, Tomas Simon, Gabriel Schwartz, Michael Zollhoefer, Yaser Sheikh, and Jason Saragih. Mixture of volumetric primitives for efficient neural rendering.ACM Transactions on Graphics (ToG), 40(4):1–13, 2021. 3
2021
-
[40]
Gsheadrelight: Fast relightability for 3d gaussian head synthesis
Henglei Lv, Bailin Deng, Jianzhu Guo, Xiaoqiang Liu, Pengfei Wan, Di Zhang, and Lin Gao. Gsheadrelight: Fast relightability for 3d gaussian head synthesis. InProceedings of the Special Interest Group on Computer Graphics and In- teractive Techniques Conference Conference Papers, pages 1–12, 2025. 3
2025
-
[41]
Rapid acqui- sition of specular and diffuse normal maps from polarized spherical gradient illumination.Rendering Techniques, 9(10): 2, 2007
Wan-Chun Ma, Tim Hawkins, Pieter Peers, Charles-Felix Chabert, Malte Weiss, Paul E Debevec, et al. Rapid acqui- sition of specular and diffuse normal maps from polarized spherical gradient illumination.Rendering Techniques, 9(10): 2, 2007. 2
2007
-
[42]
Codec avatar studio: Paired human captures for complete, driveable, and general- izable avatars.Advances in Neural Information Processing Systems, 37:83008–83023, 2024
Julieta Martinez, Emily Kim, Javier Romero, Timur Bagautdi- nov, Shunsuke Saito, Shoou-I Yu, Stuart Anderson, Michael Zollh¨ofer, Te-Li Wang, Shaojie Bai, et al. Codec avatar studio: Paired human captures for complete, driveable, and general- izable avatars.Advances in Neural Information Processing Systems, 37:83008–83023, 2024. 2, 3, 5
2024
-
[43]
Lightpainter: Interactive portrait relight- ing with freehand scribble
Yiqun Mei, He Zhang, Xuaner Zhang, Jianming Zhang, Zhixin Shu, Yilin Wang, Zijun Wei, Shi Yan, HyunJoon Jung, and Vishal M Patel. Lightpainter: Interactive portrait relight- ing with freehand scribble. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 195–205, 2023. 2
2023
-
[44]
Holo-relighting: Controllable volumetric portrait relighting from a single image
Yiqun Mei, Yu Zeng, He Zhang, Zhixin Shu, Xuaner Zhang, Sai Bi, Jianming Zhang, HyunJoon Jung, and Vishal M Patel. Holo-relighting: Controllable volumetric portrait relighting from a single image. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 4263–4273, 2024. 3
2024
-
[45]
Patel, and Paul Debevec
Yiqun Mei, Mingming He, Li Ma, Julien Philip, Wenqi Xian, David M George, Xueming Yu, Gabriel Dedic, Ahmet Levent Tas ¸el, Ning Yu, Vishal M. Patel, and Paul Debevec. Lux post facto: Learning portrait performance relighting with condi- tional video diffusion and a hybrid dataset. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco...
2025
-
[46]
Deep reflectance fields: high-quality facial reflectance field infer- ence from color gradient illumination.ACM Transactions on Graphics (TOG), 38(4):1–12, 2019
Abhimitra Meka, Christian Haene, Rohit Pandey, Michael Zollh¨ofer, Sean Fanello, Graham Fyffe, Adarsh Kowdle, Xueming Yu, Jay Busch, Jason Dourgarian, et al. Deep reflectance fields: high-quality facial reflectance field infer- ence from color gradient illumination.ACM Transactions on Graphics (TOG), 38(4):1–12, 2019. 2
2019
-
[47]
Deep relightable textures - vol- umetric performance capture with neural rendering
Abhimitra Meka, Rohit Pandey, Christian Haene, Sergio Orts- Escolano, Peter Barnum, Philip Davidson, Daniel Erickson, Yinda Zhang, Jonathan Taylor, Sofien Bouaziz, Chloe Leg- endre, Wan-Chun Ma, Ryan Overbeck, Thabo Beeler, Paul Debevec, Shahram Izadi, Christian Theobalt, Christoph Rhe- mann, and Sean Fanello. Deep relightable textures - vol- umetric perf...
2020
-
[48]
Nerf: Representing scenes as neural radiance fields for view syn- thesis.Communications of the ACM, 65(1):99–106, 2021
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis.Communications of the ACM, 65(1):99–106, 2021. 2
2021
-
[49]
Instant neural graphics primitives with a multires- olution hash encoding.ACM transactions on graphics (TOG), 41(4):1–15, 2022
Thomas M¨uller, Alex Evans, Christoph Schied, and Alexan- der Keller. Instant neural graphics primitives with a multires- olution hash encoding.ACM transactions on graphics (TOG), 41(4):1–15, 2022. 2
2022
-
[50]
Learning physics-guided face relight- ing under directional light
Thomas Nestmeyer, Jean-Franc ¸ois Lalonde, Iain Matthews, and Andreas Lehrmann. Learning physics-guided face relight- ing under directional light. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5124–5133, 2020. 2
2020
-
[51]
Total relighting: learning to relight portraits for background replacement.ACM Trans
Rohit Pandey, Sergio Orts-Escolano, Chloe Legendre, Chris- tian Haene, Sofien Bouaziz, Christoph Rhemann, Paul E De- bevec, and Sean Ryan Fanello. Total relighting: learning to relight portraits for background replacement.ACM Trans. Graph., 40(4):43–1, 2021. 2
2021
-
[52]
A diffusion approach to radiance field relighting using multi-illumination synthesis
Yohan Poirier-Ginter, Alban Gauthier, Julien Phillip, J-F Lalonde, and George Drettakis. A diffusion approach to radiance field relighting using multi-illumination synthesis. InComputer Graphics Forum, page e15147. Wiley Online Library, 2024. 2
2024
-
[53]
Difareli: Diffusion face relighting
Puntawat Ponglertnapakorn, Nontawat Tritrong, and Supa- sorn Suwajanakorn. Difareli: Diffusion face relighting. In Proceedings of the IEEE/CVF international conference on computer vision, pages 22646–22657, 2023. 2
2023
-
[54]
Vhap: Versatile head alignment with adaptive appearance priors, 2024
Shenhan Qian. Vhap: Versatile head alignment with adaptive appearance priors, 2024. 5, 3
2024
-
[55]
Gaussiana- vatars: Photorealistic head avatars with rigged 3d gaussians
Shenhan Qian, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Simon Giebenhain, and Matthias Nießner. Gaussiana- vatars: Photorealistic head avatars with rigged 3d gaussians. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20299–20309, 2024. 1, 5
2024
-
[56]
Neural shading fields for efficient facial inverse rendering
Gilles Rainer, Lewis Bridgeman, and Abhijeet Ghosh. Neural shading fields for efficient facial inverse rendering. InCom- puter Graphics Forum, page e14943. Wiley Online Library,
-
[57]
Facelit: Neural 3d relightable faces
Anurag Ranjan, Kwang Moo Yi, Jen-Hao Rick Chang, and Oncel Tuzel. Facelit: Neural 3d relightable faces. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8619–8628, 2023. 3
2023
-
[58]
V orf: V ol- umetric relightable faces
Pramod Rao, Mallikarjun BR, Gereon Fox, Tim Weyrich, Bernd Bickel, Hans-Peter Seidel, Hanspeter Pfister, Wojciech Matusik, Ayush Tewari, Christian Theobalt, et al. V orf: V ol- umetric relightable faces. In33rd British Machine Vision Conference, 2022. 3
2022
-
[59]
Lite2relight: 3d-aware single image portrait relighting
Pramod Rao, Gereon Fox, Abhimitra Meka, Mallikarjun BR, Fangneng Zhan, Tim Weyrich, Bernd Bickel, Hanspeter Pfis- ter, Wojciech Matusik, Mohamed Elgharib, et al. Lite2relight: 3d-aware single image portrait relighting. InACM SIG- GRAPH 2024 Conference Papers, pages 1–12, 2024. 3, 7, 8, 1
2024
-
[60]
3dpr: Single image 3d portrait relighting with generative priors
Pramod Rao, Abhimitra Meka, Xilong Zhou, Gereon Fox, Mallikarjun B R, Fangneng Zhan, Tim Weyrich, Bernd Bickel, Hanspeter Pfister, Wojciech Matusik, Thabo Beeler, Mo- hamed Elgharib, Marc Habermann, and Christian Theobalt. 3dpr: Single image 3d portrait relighting with generative priors. 2025. 2, 3, 5, 7, 8, 1
2025
-
[61]
Relightful harmonization: Lighting-aware portrait background replacement
Mengwei Ren, Wei Xiong, Jae Shin Yoon, Zhixin Shu, Jianming Zhang, HyunJoon Jung, Guido Gerig, and He Zhang. Relightful harmonization: Lighting-aware portrait background replacement. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6452–6462, 2024. 2
2024
-
[62]
Single-shot high-quality facial geometry and skin appearance capture.ACM Trans
J´er´emy Riviere, Paulo FU Gotardo, Derek Bradley, Abhijeet Ghosh, and Thabo Beeler. Single-shot high-quality facial geometry and skin appearance capture.ACM Trans. Graph., 39(4):81, 2020. 2
2020
-
[63]
High-resolution image synthesis with latent diffusion models
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2
2022
-
[64]
Relightable gaussian codec avatars
Shunsuke Saito, Gabriel Schwartz, Tomas Simon, Junxuan Li, and Giljoo Nam. Relightable gaussian codec avatars. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 130–141, 2024. 1, 3, 5
2024
-
[65]
Litnerf: Intrinsic radiance decomposition for high-quality view syn- thesis and relighting of faces
Kripasindhu Sarkar, Marcel C B ¨uhler, Gengyan Li, Daoye Wang, Delio Vicini, J ´er´emy Riviere, Yinda Zhang, Sergio Orts-Escolano, Paulo Gotardo, Thabo Beeler, et al. Litnerf: Intrinsic radiance decomposition for high-quality view syn- thesis and relighting of faces. InSIGGRAPH Asia 2023 Conference Papers, pages 1–11, 2023. 2
2023
-
[66]
Jonathan Schmidt, Simon Giebenhain, and Matthias Niessner. Becominglit: Relightable gaussian avatars with hybrid neural shading.arXiv preprint arXiv:2506.06271, 2025. 3
arXiv 2025
-
[67]
A light stage on every desk
Soumyadip Sengupta, Brian Curless, Ira Kemelmacher- Shlizerman, and Steven M Seitz. A light stage on every desk. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 2420–2429, 2021. 2
2021
-
[68]
Make-a-video: Text-to-video generation without text-video data.arXiv preprint arXiv:2209.14792,
Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data.arXiv preprint arXiv:2209.14792,
-
[69]
Single image portrait relighting.ACM Trans
Tiancheng Sun, Jonathan T Barron, Yun-Ta Tsai, Zexiang Xu, Xueming Yu, Graham Fyffe, Christoph Rhemann, Jay Busch, Paul E Debevec, and Ravi Ramamoorthi. Single image portrait relighting.ACM Trans. Graph., 38(4):79–1, 2019. 2 11
2019
-
[70]
Tiancheng Sun, Kai-En Lin, Sai Bi, Zexiang Xu, and Ravi Ra- mamoorthi. Nelf: Neural light-transport field for portrait view synthesis and relighting.arXiv preprint arXiv:2107.12351,
-
[71]
V olux-gan: A generative model for 3d face synthesis with hdri relighting
Feitong Tan, Sean Fanello, Abhimitra Meka, Sergio Orts- Escolano, Danhang Tang, Rohit Pandey, Jonathan Taylor, Ping Tan, and Yinda Zhang. V olux-gan: A generative model for 3d face synthesis with hdri relighting. InACM SIGGRAPH 2022 Conference Proceedings, pages 1–9, 2022. 3
2022
-
[72]
Gaussian- Heads: End-to-end learning of drivable gaussian head avatars from coarse-to-fine representations.ACM Trans
Kartik Teotia, Hyeongwoo Kim, Pablo Garrido, Marc Haber- mann, Mohamed Elgharib, and Christian Theobalt. Gaussian- Heads: End-to-end learning of drivable gaussian head avatars from coarse-to-fine representations.ACM Trans. Graph., 43 (6):1–12, 2024. 1
2024
-
[73]
Morf: Morphable radiance fields for multiview neural head modeling
Daoye Wang, Prashanth Chandran, Gaspard Zoss, Derek Bradley, and Paulo Gotardo. Morf: Morphable radiance fields for multiview neural head modeling. InACM SIGGRAPH 2022 Conference Proceedings, pages 1–9, 2022. 5
2022
-
[74]
Sunstage: Portrait reconstruction and relighting using the sun as a light stage
Yifan Wang, Aleksander Holynski, Xiuming Zhang, and Xu- aner Zhang. Sunstage: Portrait reconstruction and relighting using the sun as a light stage. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20792–20802, 2023. 3
2023
-
[75]
Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing, 13(4):600–612, 2004
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing, 13(4):600–612, 2004. 7
2004
-
[76]
Single image portrait relighting via explicit multiple reflectance channel modeling.ACM Transactions on Graphics (ToG), 39(6):1–13, 2020
Zhibo Wang, Xin Yu, Ming Lu, Quan Wang, Chen Qian, and Feng Xu. Single image portrait relighting via explicit multiple reflectance channel modeling.ACM Transactions on Graphics (ToG), 39(6):1–13, 2020. 2
2020
-
[77]
Analysis of human faces using a measurement-based skin reflectance model.ACM Transactions on Graphics (ToG), 25(3):1013– 1024, 2006
Tim Weyrich, Wojciech Matusik, Hanspeter Pfister, Bernd Bickel, Craig Donner, Chien Tu, Janet McAndless, Jinho Lee, Addy Ngan, Henrik Wann Jensen, et al. Analysis of human faces using a measurement-based skin reflectance model.ACM Transactions on Graphics (ToG), 25(3):1013– 1024, 2006. 2
2006
-
[78]
Improved lighting models for facial appearance capture.EG 2022-Short Papers, pages 5–8, 2022
Yingyan Xu, J´er´emy Riviere, Gaspard Zoss, Prashanth Chan- dran, Derek Bradley, and Paulo Gotardo. Improved lighting models for facial appearance capture.EG 2022-Short Papers, pages 5–8, 2022. 2
2022
-
[79]
Renerf: Re- lightable neural radiance fields with nearfield lighting
Yingyan Xu, Gaspard Zoss, Prashanth Chandran, Markus Gross, Derek Bradley, and Paulo Gotardo. Renerf: Re- lightable neural radiance fields with nearfield lighting. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22581–22591, 2023. 2
2023
-
[80]
Artist-friendly relightable and animatable neural heads
Yingyan Xu, Prashanth Chandran, Sebastian Weiss, Markus Gross, Gaspard Zoss, and Derek Bradley. Artist-friendly relightable and animatable neural heads. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2457–2467, 2024. 3
2024
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.