Pith. sign in

REVIEW 3 major objections 5 minor 45 references

Learning to Decouple the Lights for 3D Face Texture Modeling

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that facial albedo can be recovered free of external-occlusion shadows by modeling illumination as several decoupled light conditions, and that such textures re-render onto unoccluded faces more faithfully than textures…

desk verdict A genuinely useful decoupling idea with a mostly sound evaluation; the 'accurate textures' claim needs tightening, but it deserves a real review. read the letter →

arxiv 2412.08524 v1 pith:SGAY2E6F submitted 2024-12-11 cs.CV

Tianxin Huang , Zhenyu Zhang , Ying Tai , Gim Hee Lee This is my paper · ORCID
classification cs.CV
keywords 3Dfacereconstructiontexturemodelinglightdecouplingexternalocclusionshadowssphericalharmonicsilluminationneuralpriorsmorphablemodelrelighting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Faces photographed under a hat, hair, or other external occlusion carry shadows that most 3D face reconstruction methods fold into the recovered texture. This paper tries to establish that those shadows can instead be explained as extra local light conditions, learned by neural masks, so the texture stays clean. The claim is tested by extracting textures from shadowed source images and re-rendering them onto unoccluded target images of the same person, where the method reports higher PSNR and lower perceptual distance than every compared baseline. If correct, the result matters for downstream tasks such as face relighting and animation, because a texture separated from illumination can be reused under new lighting.

What carries the argument

The central object is the Light Decoupling pipeline: $n$ ray-traced renders under $n$ spherical-harmonics light conditions, mixed by neural masks into one synthetic image. The work is carried by the two spatial-temporal MLPs $f(\cdot)$ and $g(\cdot)$—one assigns each pixel to a light condition, the other separates face from surrounding environment—and by ACE, which prunes redundant conditions so the number of lights adapts during optimization. The prior losses $L_{GP}$, $L_{LP}$, and $L_{HP}$ are what keep the unconstrained texture/light split from drifting into implausible albedo.

What would settle it

Render a synthetic head with a known albedo texture, a hat casting a shadow, and known multi-light conditions; run the method on the rendering and compare the recovered diffuse albedo to the ground truth. If shadows leak into the texture, the albedo error in the shadowed region will remain high even when target-image PSNR is good—this would show the separation is driven by priors rather than by the multi-light representation.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a face under "unnatural" illumination can be modeled as the composition of several separate spherical-harmonics light conditions rather than one uniform light. An MLP $f(\cdot)$ predicts per-pixel masks assigning facial regions to light conditions; a second MLP $g(\cdot)$ predicts which pixels belong to the face; and an Adaptive Condition Estimation step drops masks that cover too little area. The final rendered face is the mask-weighted sum of faces rendered under each surviving light, and three prior losses pull the texture toward global color consistency, local smoothness, and face-recognition plausibility. The paper takes its target-synthesis numbers—PSNR 29.22 and LPIPS 6.36 versus 25.03 and 7.19 for the best baseline—as evidence that the recovered textures are accurate enough to be transplanted onto an unoccluded face.

Load-bearing premise

The load-bearing premise is that the image-matching objective, helped by hand-chosen plausibility rules, can correctly decide which pixels are facial skin color and which are lighting effects; nothing physically measures the occluder or the light, so the recovered texture is only as unbiased as those rules.

Editorial extensions

If this is right

  • Textures extracted from shadowed source images can be re-rendered onto unoccluded target images, matching the target more closely than any single-illumination baseline (Target PSNR 29.22 vs 25.03).
  • For video input, sharing texture, shape, and illumination across frames gives consistently better texture modeling than running single-image methods frame by frame.
  • Adding 2D shadow-removal or deocclusion pre-processing to baselines does not close the gap; treating shadows as illumination preserves facial details such as beards, which deocclusion tends to remove.
  • Raising the number of spherical-harmonics bands in a single global light cannot replace decoupling: a single 18-band SH reaches only 25.34 PSNR, while the decoupled model reaches 29.22.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension the authors do not explore is applying the same "occluder shadow as an extra light condition" idea to albedo recovery for non-face objects, such as hands or clothing, whenever the occluder geometry is unknown.
  • Because the separation is driven by priors, a stronger validity test would compare recovered albedo against ground-truth albedo on synthetic faces with known lighting; the paper only evaluates re-rendering, so the texture itself is never checked directly.
  • The AlbedoMM initialization limits high-frequency detail (the paper's own failure cases show lost wrinkles); replacing it with a non-linear generative texture prior is a natural next step that the paper names as future work.
  • A stronger downstream test of whether the decoupling truly removed shadows would re-render the recovered texture onto a different person's face shape and check whether shadow contamination re-appears; the paper's evaluation only transplants textures onto the same identity.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper addresses 3D face texture modeling from images captured under unnatural illumination caused by external occlusions (e.g., hats, hair). The central idea is to replace the traditional single-uniform-illumination assumption with a composition of multiple per-region spherical-harmonics light conditions, where an MLP predicts soft masks that assign face regions to different light conditions (Light Decoupling). A second MLP predicts a face-region mask to exclude direct occlusions. The optimization is carried out in three stages, with photometric, landmark, segmentation-distillation, and mask regularizers, followed by texture refinement under global, local, and human-face priors (LGP, LLP, LHP). Experiments on Voxceleb2 image pairs and video sequences, CelebAMask-HQ, and a shadow-removal dataset compare the method with D3DFR, CPEM, NextFace, NextFace*, and FFHQ-UV, using a texture-transfer protocol in which source textures are re-rendered on unoccluded target images and scored with PSNR, SSIM, and LPIPS. The paper reports substantial gains on the target-synthesis metric, e.g., Target PSNR 29.22 vs 25.03 for the best baseline in Table 1.

Significance. If the central claim is substantiated, the method is a useful step for 3D face reconstruction in the presence of external occlusions and complicated shadows, which is a realistic and under-addressed scenario. The evaluation protocol is genuinely non-circular: textures are transferred to independent target images and compared against ground truth, rather than merely fitting the input. The paper also provides code and videos, and it includes several ablations (Tables 3-4, 10-12) and comparisons against shadow-removal and deocclusion baselines. However, the headline claim of recovering 'accurate textures' rests on an identifiability assumption that is not directly validated: the decomposition into albedo and multiple light conditions is determined by hand-chosen priors, and the reported metrics are re-rendering metrics, not per-pixel albedo errors. The evaluation sets are small and lack statistical reliability measures. These issues limit, but do not destroy, the contribution; with additional direct validation or a carefully reframed claim, the work could be acceptable.

major comments (3)
  1. [Sec. 3.3-3.4, Eqs. (2)-(8)] The decomposition I_R = Σ I_Rs ⊙ M_L is underdetermined: the photometric loss L_pho (Eq. 2) constrains only the composite rendered image, so the split between facial albedo and the decoupled light conditions is enforced entirely by the hand-chosen priors L_GP, L_LP, L_HP and the mask regularizers L_area and L_bin, not by any physical measurement of illumination or known albedo. The target-image synthesis test in Sec. 4.1 is an indirect proxy: a smooth, shadow-free texture that approximates the target identity can achieve high PSNR/LPIPS even if it is not the true albedo, and the paper's own failure-case discussion (A.12) reports loss of high-frequency details such as wrinkles, which is consistent with the AlbedoMM-anchored priors pulling the texture toward the initialization. To support the 'accurate textures' wording in the conclusion, the authors should validate against ground-truth albedo (e.g., synthetic faces rendered from known albedo and illumination, or a dataset with measured reflectance), or explicitly reframe the contribution as re-rendering quality rather than albedo accuracy. This concern is the central load-bearing point for the paper's main claim.
  2. [Sec. 4.1, Tables 1 and 2] The evaluation sets are small and self-constructed (38 image pairs, 24 video pairs), and no error bars, per-subject breakdowns, or significance tests are reported. The reported margin in Target PSNR (29.22 vs 25.03 in Table 1; 29.15 vs 24.35 in Table 2) is large, but without variance estimates it is impossible to rule out that the gap is dominated by a few favorable subjects or by the specific selection of source images with occlusions. Please report standard deviations or confidence intervals, and ideally paired per-identity differences, to demonstrate that the improvement is consistent across the test set rather than an artifact of averaging.
  3. [Sec. 4.4, Table 5] The claim that 'our method still outperforms other methods under faces with diverse shadows' is not uniformly supported by Table 5. On the first row (presumably source reconstruction), NextFace achieves LPIPS 5.26 versus 6.29 for the proposed method, and PSNR is essentially tied (32.10 vs 32.13). The paper should label the two rows explicitly (e.g., Source and Target, matching Tables 1-2) and either temper the stated claim or explain why the source-reconstruction LPIPS regression is acceptable given the method's goals. As written, the conclusion that the method 'consistently surpasses' baselines is too strong relative to the data in this table.
minor comments (5)
  1. [Throughout] There are several typos and grammatical slips: 'adpatively' (Sec. 3.2), 'texutures' in the sentence before Eq. (8), 'an regularization' (Sec. 3.3), and 'Alg. 3' in A.8 should be 'Alg. 1'.
  2. [Eqs. (4)-(5)] The normalization in L_area and L_bin is written ambiguously: the expressions '1/|I_in| X 1/n ...' and '1/|I_in| X 1/n_L ...' mix a spatial average with a sum over masks. Please rewrite these equations with clear indices (e.g., separate the sum over masks from the pixel average) so the reader can verify the intended scale.
  3. [Sec. 4.1] The texture-transfer protocol says 'we optimize source and target images separately following Sec. 3.' Please clarify whether all three stages are run independently for each image and how the final texture variable T is extracted for transfer (in particular, whether the Stage-3 texture is used and how the target's illumination is kept fixed).
  4. [Table 5] The two rows of Table 5 are unlabeled. Please add row labels (e.g., 'Source' and 'Target') to match Tables 1-2, since the paper currently refers to them only implicitly in the text.
  5. [A.5] The phrase '9-bands SH' and later '9, 12, 15, 18 bands' is confusing: spherical harmonics are typically described by the number of coefficients (e.g., 9 for order 2) or by the band order. Please clarify whether these numbers denote SH coefficient counts, band orders, or something else.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the central texture-quality claim is tested by an external target-transfer benchmark rather than by construction.

full rationale

The paper's central claim is evaluated by an external transfer benchmark (Sec 4.1): textures are optimized on occluded source images, then re-rendered onto unoccluded target images of the same identity and compared with ground-truth targets. The target image is not used when fitting the source texture, so the metric does not reduce to a fitted value. The photometric loss (Eq. 2) combined with the light-mask decomposition (Sec 3.3) does leave the texture/illumination split underdetermined, and the priors (Eqs. 6-8) pull the texture toward the AlbedoMM initialization; this is an identifiability and accuracy limitation, acknowledged in A.1 and A.12, not a circularity, because no equation makes the predicted texture equal to the input by construction. The self-citations ([42], [43], [38]) appear only in related-work enumerations and are not load-bearing. The source-reconstruction metric is partly trivial since Iout = IR * Mo + Iin * (1-Mo) copies non-face pixels from the input, but the paper's headline conclusion relies on the target-transfer test, which is genuinely external. Overall, the derivation chain is self-contained against external benchmarks, and no fitted parameter is renamed as a prediction.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

The central claim rests on a set of hand-tuned hyperparameters (n, epsilon, loss weights, iteration schedule, K-means size) and on five domain or ad-hoc assumptions, the most fragile being that external-occlusion illumination can be decomposed into learned per-region SH lights and that the losses prevent shadow absorption into the texture. The AlbedoMM initialization is acknowledged by the authors as a limiting factor.

free parameters (5)
  • n (initial number of light conditions) = 5
    Number of SH light conditions; Table 11 shows n=5 was selected as the best on the evaluation set, making it a tuned hyperparameter.
  • epsilon (ACE area threshold) = 0.17
    Threshold for dropping light masks with small area; set in Table 6 without sensitivity analysis.
  • Loss weights w0..w7 = 2e3, 1e-3, 1.5e2, 0.5, 25, 2e3, 2.0, 1.0
    Balances photometric, landmark, segmentation, area, binary, and prior losses; adjusted using the evaluation data.
  • Iteration schedule iter0..iter3 = 100, 2000, 400, 200
    Stage scheduling parameters; set manually and not varied in the paper.
  • K-means color count for LGP = 16 (4x4 color matrix)
    Number of color clusters for the global hue prior; chosen without justification or sensitivity analysis.
assumptions (5)
  • domain assumption The 3DMM (AlbedoMM) provides a sufficiently accurate initialization of face texture and shape.
    Used to initialize T0 and the statistical coefficients; the paper admits in A.1 and A.12 that AlbedoMM limits the accuracy for high-frequency details.
  • domain assumption NextFace's ray-tracing renderer faithfully renders face appearance given texture, shape, and SH illumination.
    The framework inherits this renderer from [12] and uses it for all photometric losses and light decoupling.
  • ad hoc to paper The effect of external occlusions on illumination can be represented as a composition of multiple per-region SH light conditions selected by learned masks.
    This is the central modeling assumption of Sec 3.3; it is not derived from physical light transport and is justified only by the reported benchmark results.
  • ad hoc to paper The photometric and prior losses are sufficient to prevent the texture from absorbing the shadows (identifiability of texture vs. light).
    Sec 3.3 and 3.4; the decomposition of an image into texture, light, and masks is underdetermined, and no ground-truth textures are used to verify the separation.
  • domain assumption Face parsing and FaceNet models provide reliable face priors for the tested images.
    Used for Lseg and LHP; assumes these pre-trained networks generalize to Voxceleb2, CelebAMask-HQ, and the shadow dataset.
invented entities (1)
  • Multiple imitative light conditions (Light Decoupling)
    purpose: Represent illumination affected by external occlusions as a set of independent SH lights, each active in a face region predicted by an MLP.
    These light conditions are not measured or physically validated; they are latent factors optimized per image. No outside falsifiable prediction (e.g., a light probe measurement) is provided, and evidence is indirect via re-rendering PSNR and LPIPS.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning to Decouple the Lights for 3D Face Texture Modeling." pith.science (2026). https://pith.science/paper/SGAY2E6F

@misc{pith2026241208524,
  author       = {Pith},
  title        = {Pith review of: Learning to Decouple the Lights for 3D Face Texture Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SGAY2E6F}},
  note         = {Machine review of arXiv:2412.08524}
}
read the original abstract

Existing research has made impressive strides in reconstructing human facial shapes and textures from images with well-illuminated faces and minimal external occlusions. Nevertheless, it remains challenging to recover accurate facial textures from scenarios with complicated illumination affected by external occlusions, e.g. a face that is partially obscured by items such as a hat. Existing works based on the assumption of single and uniform illumination cannot correctly process these data. In this work, we introduce a novel approach to model 3D facial textures under such unnatural illumination. Instead of assuming single illumination, our framework learns to imitate the unnatural illumination as a composition of multiple separate light conditions combined with learned neural representations, named Light Decoupling. According to experiments on both single images and video sequences, we demonstrate the effectiveness of our approach in modeling facial textures under challenging illumination affected by occlusions. Please check https://tianxinhuang.github.io/projects/Deface for our videos and codes.

Figures

Figures reproduced from arXiv: 2412.08524 by the authors.

Figure 1
Figure 1. Blue and red rectangles mark regions affected by self and external occlusions, respectively. (a) Texture modeling with diffuse-only texture map. (b) Texture modeling based on diffuse, specular, and roughness albedos from local reflectance model [12], while optimizing with ray-tracing render. (c) Our method learns neural representations to decouple the original illumination into multiple light conditions, where the i… view at source ↗
Figure 2
Figure 2. Illustration of our framework. The pipeline is proposed to recover texture [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Comparison on Voxceleb2 images. The diffuse albedo is visualized as the texture because it [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (16 more)
Figure 4
Figure 4. Figure 4: Comparison results on the CelebAMask-HQ dataset. Ours and Ours+ denote our rendered [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 6
Figure 6. Figure 6: Ablation study for the neural represen￾tations. NA means to remove both f(·) and g(·), while + Light and + Occlusion denote adding f(·) and g(·), respectively [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 4
Figure 4. Figure 4: We observe that our method continues to perform well in the task of recovering clear textures [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 7
Figure 7. Figure 7: Comparison on images with diverse shadows [ [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Detailed design of f(·) and g(·) [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Discussion about the effect of g(·). w2 is the weight to constrain g(·), defined in Alg. 1. The red and black rectangles mark shadow-affected regions and detailed textures, respectively. g(·) will weaken both shadows and details from textures when reducing w2 to loose …
Figure 10
Figure 10. Figure 10: Ablation study for Larea and Lbin in ACE. NA denotes removing both Larea and Lbin. Larea can remove redundant light conditions as shown by the blue rectangle, while Lbin ensures the light condition shown in the red rectangle region is consistent as our observation of …
Figure 11
Figure 11. Figure 11: Qualitative Comparisons against baselines with 2D shadow-removal pre-processing. [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Qualitative Comparisons with the Deocclusion method [25]. [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: Differences between Stage 2 and Stage 3. In Stage 3, the texture is refined with details [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]
Figure 14
Figure 14. Figure 14: Ablation study for the usage of neural representation [PITH_FULL_IMAGE:figures/full_fig_p019_14.png]
Figure 15
Figure 15. Figure 15: Some failure cases. A.13 Effect of Adaptive Condition Estimation As described in Sec. 3.3, the Adaptive Condition Estimation (ACE) is proposed to select effective ML and IRs from the initialized MN and IRn. To remove ACE, we use initialized MN and IRn as ML and IRs di…
Figure 16
Figure 16. Figure 16: Ablation study for ACE. W/O ACE and W/ ACE denote removing ACE by using MN and IRn as ML and IRs, and using ACE to select ML and IRs, respectively. Source CPEM D3DFR NextFace NextFace* FFHQ-UV Ours Ours+ GT [PITH_FULL_IMAGE:figures/full_fig_p020_16.png]
Figure 17
Figure 17. Figure 17: Comparison results on the video sequences from Voxceleb2. Ours and Ours+ denote our [PITH_FULL_IMAGE:figures/full_fig_p020_17.png]
Figure 18
Figure 18. Figure 18: More results on single images from Voxceleb2 [ [PITH_FULL_IMAGE:figures/full_fig_p021_18.png]
Figure 19
Figure 19. Figure 19: More reconstructed images/textures results on CelebAMask-HQ [ [PITH_FULL_IMAGE:figures/full_fig_p022_19.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 34 canonical work pages

  1. [1]

    nteractive computer graphics: a top-down approach with shader-based opengl

    Edward Angel. nteractive computer graphics: a top-down approach with shader-based opengl. 2011. 10

    work page 2011

  2. [2]

    Ffhq-uv: Normalized facial uv-texture dataset for 3d face reconstruction

    Haoran Bai, Di Kang, Haoxian Zhang, Jinshan Pan, and Linchao Bao. Ffhq-uv: Normalized facial uv-texture dataset for 3d face reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 362–371, 2023

    work page 2023

  3. [3]

    High-fidelity 3d digital human head creation from rgb-d selfies

    Linchao Bao, Xiangkai Lin, Yajing Chen, Haoxian Zhang, Sheng Wang, Xuefei Zhe, Di Kang, Haozhi Huang, Xinwei Jiang, Jue Wang, et al. High-fidelity 3d digital human head creation from rgb-d selfies. ACM Transactions on Graphics (TOG), 41(1):1–21, 2021

    work page 2021

  4. [4]

    A morphable model for the synthesis of 3d faces

    V olker Blanz and Thomas Vetter. A morphable model for the synthesis of 3d faces. InSeminal Graphics Papers: Pushing the Boundaries, Volume 2, pages 157–164. 2023

    2023

  5. [5]

    Models of light reflection for computer synthesized pictures

    James F Blinn. Models of light reflection for computer synthesized pictures. In Proceedings of the 4th annual conference on Computer graphics and interactive techniques, pages 192–198, 1977

    work page 1977

  6. [6]

    A 3d morphable model learnt from 10,000 faces

    James Booth, Anastasios Roussos, Stefanos Zafeiriou, Allan Ponniah, and David Dunaway. A 3d morphable model learnt from 10,000 faces. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5543–5552, 2016

    work page 2016

  7. [7]

    Vggface2: A dataset for recognising faces across pose and age

    Qiong Cao, Li Shen, Weidi Xie, Omkar M Parkhi, and Andrew Zisserman. Vggface2: A dataset for recognising faces across pose and age. In 2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018), pages 67–74. IEEE, 2018

    2018

  8. [8]

    V oxceleb2: Deep speaker recognition.arXiv preprint arXiv:1806.05622, 2018

    Joon Son Chung, Arsha Nagrani, and Andrew Zisserman. V oxceleb2: Deep speaker recognition.arXiv preprint arXiv:1806.05622, 2018

    arXiv 2018

Show all 45 references
  1. [9]

    Stylefaceuv: A 3d face uv map generator for view-consistent face image synthesis

    Wei-Chieh Chung, Jian-Kai Zhu, I-Chao Shen, Yu-Ting Wu, and Yung-Yu Chuang. Stylefaceuv: A 3d face uv map generator for view-consistent face image synthesis. 2022

    2022

  2. [10]

    Accurate 3d face reconstruc- tion with weakly-supervised learning: From single image to image set

    Yu Deng, Jiaolong Yang, Sicheng Xu, Dong Chen, Yunde Jia, and Xin Tong. Accurate 3d face reconstruc- tion with weakly-supervised learning: From single image to image set. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 0–0, 2019

    2019

  3. [11]

    S2f2: Self-supervised high fidelity face reconstruction from monocular image

    Abdallah Dib, Junghyun Ahn, Cedric Thebault, Philippe-Henri Gosselin, and Louis Chevallier. S2f2: Self-supervised high fidelity face reconstruction from monocular image. arXiv preprint arXiv:2203.07732, 2022

    2022 arXiv

  4. [12]

    Practical face reconstruction via differentiable ray tracing

    Abdallah Dib, Gaurav Bharaj, Junghyun Ahn, Cédric Thébault, Philippe Gosselin, Marco Romeo, and Louis Chevallier. Practical face reconstruction via differentiable ray tracing. In Computer Graphics Forum, volume 40, pages 153–164. Wiley Online Library, 2021

    2021

  5. [13]

    Towards high fidelity monocular face reconstruction with rich reflectance using self-supervised learning and ray tracing

    Abdallah Dib, Cedric Thebault, Junghyun Ahn, Philippe-Henri Gosselin, Christian Theobalt, and Louis Chevallier. Towards high fidelity monocular face reconstruction with rich reflectance using self-supervised learning and ray tracing. In Proceedings of the IEEE/CVF Internationa...

    2021

  6. [14]

    Occlusion-aware 3d morphable models and an illumination prior for face image analysis

    Bernhard Egger, Sandro Schönborn, Andreas Schneider, Adam Kortylewski, Andreas Morel-Forster, Clemens Blumer, and Thomas Vetter. Occlusion-aware 3d morphable models and an illumination prior for face image analysis. International Journal of Computer Vision, 126:1269–1287, 2018

    2018

  7. [15]

    3d morphable face models—past, present, and future

    Bernhard Egger, William AP Smith, Ayush Tewari, Stefanie Wuhrer, Michael Zollhoefer, Thabo Beeler, Florian Bernard, Timo Bolkart, Adam Kortylewski, Sami Romdhani, et al. 3d morphable face models—past, present, and future. ACM Transactions on Graphics (ToG), 39(5):1–38, 2020

    2020

  8. [16]

    Ganfit: Generative adversarial network fitting for high fidelity 3d face reconstruction

    Baris Gecer, Stylianos Ploumpis, Irene Kotsia, and Stefanos Zafeiriou. Ganfit: Generative adversarial network fitting for high fidelity 3d face reconstruction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1155–1164, 2019

    2019

  9. [17]

    Unsupervised portrait shadow removal via generative priors

    Yingqing He, Yazhou Xing, Tianjia Zhang, and Qifeng Chen. Unsupervised portrait shadow removal via generative priors. In Proceedings of the 29th ACM International Conference on Multimedia, pages 236–244, 2021

    2021

  10. [18]

    Face relighting with geometri- cally consistent shadows

    Andrew Hou, Michel Sarkis, Ning Bi, Yiying Tong, and Xiaoming Liu. Face relighting with geometri- cally consistent shadows. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4217–4226, 2022

    2022

  11. [19]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  12. [20]

    Imagenet classification with deep convolutional neural networks

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012

    2012

  13. [21]

    in-the-wild

    Alexandros Lattas, Stylianos Moschoglou, Baris Gecer, Stylianos Ploumpis, Vasileios Triantafyllou, Abhijeet Ghosh, and Stefanos Zafeiriou. Avatarme: Realistically renderable 3d facial reconstruction" in-the-wild". In Proceedings of the IEEE/CVF conference on computer vision an...

    2020

  14. [22]

    Fitme: Deep photorealistic 3d morphable model avatars

    Alexandros Lattas, Stylianos Moschoglou, Stylianos Ploumpis, Baris Gecer, Jiankang Deng, and Stefanos Zafeiriou. Fitme: Deep photorealistic 3d morphable model avatars. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8629–8640, 2023

    2023

  15. [23]

    Avatarme++: Facial shape and brdf inference with photorealistic rendering-aware gans

    Alexandros Lattas, Stylianos Moschoglou, Stylianos Ploumpis, Baris Gecer, Abhijeet Ghosh, and Stefanos Zafeiriou. Avatarme++: Facial shape and brdf inference with photorealistic rendering-aware gans. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12):9269–9...

    2021

  16. [24]

    Maskgan: Towards diverse and interactive facial image manipulation

    Cheng-Han Lee, Ziwei Liu, Lingyun Wu, and Ping Luo. Maskgan: Towards diverse and interactive facial image manipulation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020

    2020

  17. [25]

    Robust model-based face reconstruction through weakly-supervised outlier segmentation

    Chunlu Li, Andreas Morel-Forster, Thomas Vetter, Bernhard Egger, and Adam Kortylewski. Robust model-based face reconstruction through weakly-supervised outlier segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 372–381, 2023

    2023

  18. [26]

    Differentiable monte carlo ray tracing through edge sampling

    Tzu-Mao Li, Miika Aittala, Frédo Durand, and Jaakko Lehtinen. Differentiable monte carlo ray tracing through edge sampling. ACM Trans. Graph. (Proc. SIGGRAPH Asia), 37(6):222:1–222:11, 2018

    2018

  19. [27]

    Face parsing with roi tanh- warping

    Jinpeng Lin, Hao Yang, Dong Chen, Ming Zeng, Fang Wen, and Lu Yuan. Face parsing with roi tanh- warping. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5654–5663, 2019

    2019

  20. [28]

    Blind removal of facial foreign shadows

    Yaojie Liu, Andrew Z Hou, Xinyu Huang, Liu Ren, and Xiaoming Liu. Blind removal of facial foreign shadows. In BMVC, page 88, 2022

    2022

  21. [29]

    Large-scale celebfaces attributes (celeba) dataset

    Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Large-scale celebfaces attributes (celeba) dataset. Retrieved August, 15(2018):11, 2018

    2018

  22. [30]

    Towards accurate facial motion retargeting with identity-consistent and expression-exclusive constraints

    Langyuan Mo, Haokun Li, Chaoyang Zou, Yubing Zhang, Ming Yang, Yihong Yang, and Mingkui Tan. Towards accurate facial motion retargeting with identity-consistent and expression-exclusive constraints. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, p...

    1981

  23. [31]

    Learning physics- guided face relighting under directional light

    Thomas Nestmeyer, Jean-François Lalonde, Iain Matthews, and Andreas Lehrmann. Learning physics- guided face relighting under directional light. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5124–5133, 2020

    2020

  24. [32]

    A 3d face model for pose and illumination invariant face recognition

    Pascal Paysan, Reinhard Knothe, Brian Amberg, Sami Romdhani, and Thomas Vetter. A 3d face model for pose and illumination invariant face recognition. In 2009 sixth IEEE international conference on advanced video and signal based surveillance, pages 296–301. Ieee, 2009

    2009

  25. [33]

    An efficient representation for irradiance environment maps

    Ravi Ramamoorthi and Pat Hanrahan. An efficient representation for irradiance environment maps. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques , pages 497–500, 2001

    2001

  26. [34]

    A signal-processing framework for inverse rendering

    Ravi Ramamoorthi and Pat Hanrahan. A signal-processing framework for inverse rendering. InProceedings of the 28th annual conference on Computer graphics and interactive techniques, pages 117–128, 2001

    2001

  27. [35]

    Facenet: A unified embedding for face recognition and clustering

    Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 815–823, 2015

    2015

  28. [36]

    Sfsnet: Learning shape, reflectance and illuminance of facesin the wild’

    Soumyadip Sengupta, Angjoo Kanazawa, Carlos D Castillo, and David W Jacobs. Sfsnet: Learning shape, reflectance and illuminance of facesin the wild’. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6296–6305, 2018

    2018

  29. [37]

    A morphable face albedo model

    William AP Smith, Alassane Seck, Hannah Dee, Bernard Tiddeman, Joshua B Tenenbaum, and Bernhard Egger. A morphable face albedo model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5011–5020, 2020

    2020

  30. [38]

    Facechain-imagineid: Freely crafting high-fidelity diverse talking faces from disentangled audio

    Chao Xu, Yang Liu, Jiazheng Xing, Weida Wang, Mingze Sun, Jun Dan, Tianxin Huang, Siyuan Li, Zhi-Qi Cheng, Ying Tai, et al. Facechain-imagineid: Freely crafting high-fidelity diverse talking faces from disentangled audio. In Proceedings of the IEEE/CVF Conference on Computer V...

    2024

  31. [39]

    Learning face representation from scratch

    Dong Yi, Zhen Lei, Shengcai Liao, and Stan Z Li. Learning face representation from scratch. arXiv preprint arXiv:1411.7923, 2014

    2014 arXiv

  32. [40]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595, 2018

    2018

  33. [41]

    Portrait shadow manipulation

    Xuaner Zhang, Jonathan T Barron, Yun-Ta Tsai, Rohit Pandey, Xiuming Zhang, Ren Ng, and David E Jacobs. Portrait shadow manipulation. ACM Transactions on Graphics (TOG), 39(4):78–1, 2020

    2020

  34. [42]

    Learning neural proto-face field for disentangled 3d face modeling in the wild

    Zhenyu Zhang, Renwang Chen, Weijian Cao, Ying Tai, and Chengjie Wang. Learning neural proto-face field for disentangled 3d face modeling in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 382–393, June 2023. 12

    2023

  35. [43]

    Learning to restore 3d face from in-the-wild degraded images

    Zhenyu Zhang, Yanhao Ge, Ying Tai, Xiaoming Huang, Chengjie Wang, Hao Tang, Dongjin Huang, and Zhifeng Xie. Learning to restore 3d face from in-the-wild degraded images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 4237–42...

    2022

  36. [44]

    Imface: A nonlinear 3d morphable face model with implicit neural representations

    Mingwu Zheng, Hongyu Yang, Di Huang, and Liming Chen. Imface: A nonlinear 3d morphable face model with implicit neural representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20343–20352, 2022

    2022

  37. [45]

    Neuface: Realistic 3d neural face rendering from multi-view images

    Mingwu Zheng, Haiyu Zhang, Hongyu Yang, and Di Huang. Neuface: Realistic 3d neural face rendering from multi-view images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16868–16877, 2023. 13 A Appendix / Supplementary Material A.1 L...

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.