REVIEW 4 major objections 7 minor 41 references
Generative Landmarks Guided Eyeglasses Removal 3D Face Reconstruction
T0 review · 4 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read The paper claims that a face-parsing-guided generator can delete eyeglasses from a single face photo, and that regressing a 3D morphable model from the cleaned image gives state-of-the-art 3D face reconstruction under occlusion.
desk verdict A plausible pipeline idea, but this is an undisclosed re-publication with an inconsistent loss and thin evaluation, so it should be desk-rejected. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the two-stage delete-then-synthesize face synthesis module: a face parsing map from a U-Net-based deleter identifies and masks the eyeglasses region, and a landmark-conditioned U-Net generator inpaints that region using the corrupted image and 68 facial landmarks as geometric guidance. This cleaned image is what the 3DMM regression network sees, so the quality of the final 3D face depends on the generator removing the glasses while preserving the identity and topology of the eyes, nose, and mouth.
What would settle it
Feed the trained synthesis module a face image with glasses and compare its output to a paired glasses-free image of the same person; if $\mathcal{L}_{\text{pixe}}$ in Eq. (2) was the only supervision on the masked pixels, the network has no gradient signal to force removal, so this paired test would reveal whether the removal is actually learned or merely a side effect of the shape prior.
Extended reading notes
Core claim
The paper's central claim is that eyeglasses occlusion can be handled by explicitly removing the glasses before 3D reconstruction, rather than by making the reconstructor robust to occlusion directly. A face-parsing network (the deleter) identifies the eyeglasses pixels and produces a corrupted image; a U-Net generator then fills the removed area, guided by 68 facial landmarks, under an adversarial discriminator; the resulting glasses-free image is fed to a fixed ResNet-50 that regresses 3DMM shape, expression, texture, lighting, and pose coefficients. The cleaned image is also used for pixel-wise and FaceNet-based feature losses against the rendered face, and a graph-convolutional network refines the texture. The authors report that this pipeline outperforms prior 3D face reconstruction methods when the input contains eyeglasses, including transparent and sunglasses.
Load-bearing premise
The synthesis network is trained with a per-pixel loss (Eq. 2) that compares the generated image to the input image while the text states the input contains no occlusion, so it is never established how the generator learns to remove eyeglasses rather than to reproduce the input.
Editorial extensions
If this is right
- If the pipeline works as claimed, existing 3DMM-based reconstruction systems can be wrapped with a glasses-removal front end and applied to in-the-wild photos with eyeglasses.
- The face parsing map provides a natural way to extend the method to other occlusions, such as masks or respirators, by adding new parsing classes.
- The cleaned image plus 3DMM rendering enables verification under occlusion, as shown by LFW experiments where removing glasses recovers recognition accuracy close to the no-glasses baseline.
- Because the pipeline yields both a glasses-free texture and a 3DMM shape, it can feed downstream tasks like face recognition, animation, and avatar creation without requiring a clean input image.
Reading between the lines
- A testable extension is to apply the same delete-synthesize-reconstruct recipe to other occlusion types, such as respirators or hands, and measure whether the 3DMM regressor transfers without retraining.
- If the per-pixel loss truly compares the output to an occlusion-free input, the generator may be learning identity-preserving reconstruction rather than explicit glasses removal; an ablation that removes landmark conditioning would clarify what actually drives the removal.
- The claimed robustness on LFW could be decomposed: a control experiment that crops out rather than inpaints the glasses region would separate the benefit of glasses removal from the benefit of the 3DMM shape prior itself.
- The paper's own trick of artificially adding eyeglasses to clean 3D face datasets for evaluation suggests a cheap data-augmentation strategy: use the same pairing to supervise the removal stage, which the paper currently does not do.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a two-stage method for single-image 3D face reconstruction under eyeglass occlusion. In Stage I, a face parsing module identifies and deletes the eyeglasses region, and a generative network synthesizes a glasses-free 2D face guided by face landmarks. In Stage II, a ResNet-50 regresses 3DMM shape, expression, texture, illumination, and pose parameters from the synthesized image, followed by a texture refinement network. The authors claim state-of-the-art qualitative performance and report quantitative results on MICC Florence (with artificially added eyeglasses) and LFW.
Significance. The task is relevant to in-the-wild face reconstruction, and the idea of combining face parsing, landmark conditioning, and a synthesis module is a reasonable design direction. If the method worked as described, it would address a practical limitation of current methods that assume unobstructed faces. However, the central training objective for glasses removal is not established in the manuscript, and the quantitative evidence is thin. The paper does not provide code, trained models, or a detailed training-data protocol, which would be necessary to verify the central claim.
major comments (4)
- [Section 3.2, Eq. (2)] The per-pixel loss L_pixe = (1/S)||Iout - Iin|| compares the synthesized output with the original input Iin. Since Iin is the eyeglasses-occluded photo in the pipeline of Fig. 1, minimizing this loss encourages Iout to reproduce the glasses rather than remove them. The sentence immediately below Eq. (2) states 'our input image will not contain occlusion,' which contradicts the pipeline description, and the paper never explains how a glasses-free reference image would be paired with the occluded input during training. Because the downstream 3D reconstruction consumes Iout (Eq. 8), this unstated supervision assumption is load-bearing.
- [Section 3.2, Eq. (5)] The total synthesis loss L_fsm contains only L_pixe, L_style, and L_var. The prose promises a combination of adversarial, per-pixel, perceptual, style, total variation, and adversarial losses, but the equation omits the adversarial and perceptual terms, and no discriminator objective is given. Without an adversarial or perceptual term, the generator has no incentive to inpaint the masked region with plausible facial structure, so the mechanism for intelligent glasses removal is absent.
- [Section 3.2, Eq. (3)] The style loss is computed on the masked region Im between Iout and Iin. If Iin contains eyeglasses, this term penalizes divergence from the original glasses texture and therefore reinforces the per-pixel loss in Eq. (2). This is inconsistent with the stated goal of removing eyeglasses and further undermines the training supervision.
- [Section 4.3] The quantitative evaluation does not support the state-of-the-art claim. The MICC Florence comparison reports single 90%-error numbers (Fig. 5) with no error bars, standard deviations, or details on how many subjects and which glasses augmentations were used. Table 1 on LFW compares only with Tran et al. and omits the baselines used in the qualitative comparisons (PRNet, 3DDFA, DF2Net, Chen et al.); moreover, the verification protocol is not described beyond a reference to 'the same face test system from Anh et al.'
minor comments (7)
- [References] Reference [24] is cited for the LFW dataset, but the cited paper is 'Video generation from single semantic label map' by Pan et al.; the LFW dataset should be cited with its original source (e.g., Huang et al.).
- [Section 4.3] The text cites 'Sela et al.[17]' but reference [17] is Isola et al.; the intended citation is likely Sela et al. (reference [32]).
- [Section 4.3] The text refers to 'the paper of Deng et al. [29]' but reference [29] is Richardson et al.; please correct the citation.
- [References and Contributions] The relationship to the authors' prior works, especially reference [6] with the same title and venue (MMM 2022), should be clarified. If this arXiv manuscript is an extended version, the novelty relative to that publication should be stated explicitly.
- [Eq. (4)] The denominator PIin in the total variation loss is not defined; it presumably denotes the number of pixels in Iin, but this should be stated.
- [Fig. 5 and Section 4.3] The caption says digits denote '90% error (mm)' while the text says 'average 90% largest error'; please specify the exact metric and how it is computed.
- [Conclusions] The conclusion states that the method 'outperforms previous arts by a large margin,' but the paper only provides qualitative evidence and a single MICC number; this claim should be tempered or supported with statistical tests.
Circularity Check
The eyeglasses-removal output is trained, by Eq. (2) and Eq. (5), to reproduce the original (occluded) input; the central 'removal' prediction reduces to the input itself.
-
self definitional
[Section 3.2, Eqs. (2)-(5)]
"Lpixe = 1/S ||Iout − Iin|| (2) ... It should be pointed out that our input image will not contain occlusion, so we don’t need to consider this. ... Lfsm = λpixeLpixe + λstyleLstyle + λvarLvar (5)"
By Eq. (2), the per-pixel target for the generator's output Iout is the original input Iin; the unique minimizer at each pixel is Iout = Iin. If Iin is eyeglasses-occluded, this loss directly penalizes glasses removal; if Iin is clean, no paired clean/glasses supervision is described. Eq. (5) shows that the total synthesis loss consists only of this pixel term, the masked style loss of Eq. (3) (also comparing Iout with Iin inside the glasses mask Im), and a TV term; no adversarial or perceptual term appears despite the prose promising them. Hence the claimed 'eyeglasses-free' image is, by construction, a fit to the input, and the downstream 3DMM regression of Eq. (8) consumes that unchanged image.
full rationale
The downstream 3DMM regression (ResNet-50, Eqs. (6)-(9)) and the MICC/LFW evaluations are external and not circular. The paper's pervasive self-citations [6,7,8,40] are background padding rather than load-bearing derivations, so they do not by themselves raise the score. The load-bearing synthesis stage, however, is self-definitional: its published losses define success as reproducing Iin and omit the adversarial/perceptual terms that the text says are used, so the central claim that the method 'removes eyeglasses intelligently' is not derived from any independent target. This is a partial but central reduction-by-construction, hence score 6. Some aspects (e.g., inability to obtain clean targets) are correctness risks rather than circularity, but the equation-level identity Iout ≈ Iin is the core circular step.
Assumptions & free parameters
free parameters (5)
- lambda_pixe =
1
- lambda_style =
250
- lambda_var =
0.1
- lambda1_3d =
1.4
- lambda2_3d =
0.25
assumptions (5)
- domain assumption The face parser (deleter) trained on CelebAMask-HQ correctly identifies eyeglasses pixels.
- domain assumption The 3DMM PCA bases Aid, Bexp, and Bt span the face behind the glasses.
- standard math Spherical harmonics with 9 coefficients model the scene illumination.
- domain assumption FaceNet cosine similarity is a valid perceptual and identity loss for driving reconstruction.
- ad hoc to paper The per-pixel loss uses a glasses-free reference image during training.
Cite this review
Pith. "Pith review of Generative Landmarks Guided Eyeglasses Removal 3D Face Reconstruction." pith.science (2026). https://pith.science/paper/RIEICV52
@misc{pith2026241219848,
author = {Pith},
title = {Pith review of: Generative Landmarks Guided Eyeglasses Removal 3D Face Reconstruction},
year = {2026},
howpublished = {\url{https://pith.science/paper/RIEICV52}},
note = {Machine review of arXiv:2412.19848}
}
read the original abstract
Single-view 3D face reconstruction is a fundamental Computer Vision problem of extraordinary difficulty. Current systems often assume the input is unobstructed faces which makes their method not suitable for in-the-wild conditions. We present a method for performing a 3D face that removes eyeglasses from a single image. Existing facial reconstruction methods fail to remove eyeglasses automatically for generating a photo-realistic 3D face "in-the-wild".The innovation of our method lies in a process for identifying the eyeglasses area robustly and remove it intelligently. In this work, we estimate the 2D face structure of the reasonable position of the eyeglasses area, which is used for the construction of 3D texture. An excellent anti-eyeglasses face reconstruction method should ensure the authenticity of the output, including the topological structure between the eyes, nose, and mouth. We achieve this via a deep learning architecture that performs direct regression of a 3DMM representation of the 3D facial geometry from a single 2D image. We also demonstrate how the related face parsing task can be incorporated into the proposed framework and help improve reconstruction quality. We conduct extensive experiments on existing 3D face reconstruction tasks as concrete examples to demonstrate the method's superior regulation ability over existing methods often break down.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[6]
In: International Conference on Multimedia Modeling
Dapeng, Z., Yue, Q.: Generative landmarks guided eyeglasses removal 3d face re- construction. In: International Conference on Multimedia Modeling. pp. 111–122. Springer (2022)
work page 2022
-
[1]
In: Proceedings of the 2011 joint ACM workshop on Human gesture and behavior understanding
Bagdanov, A.D., Del Bimbo, A., Masi, I.: The florence 2d/3d hybrid face dataset. In: Proceedings of the 2011 joint ACM workshop on Human gesture and behavior understanding. pp. 79–80 (2011)
work page 2011
-
[2]
In: Pro- ceedings of the 27th annual conference on Computer graphics and interactive tech- niques
Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Pro- ceedings of the 27th annual conference on Computer graphics and interactive tech- niques. pp. 417–424 (2000)
work page 2000
-
[3]
Blanz, V., Vetter, T.: A morphable model for the synthesis of 3d faces. In: Siggraph. vol. 99, pp. 187–194 (1999)
work page 1999
-
[4]
IEEE Transactions on pattern analysis and machine intelligence 25(9), 1063–1074 (2003)
Blanz, V., Vetter, T.: Face recognition based on fitting a 3d morphable model. IEEE Transactions on pattern analysis and machine intelligence 25(9), 1063–1074 (2003)
work page 2003
-
[5]
In: Proceedings of the IEEE International Conference on Computer Vision
Chen, A., Chen, Z., Zhang, G., Mitchell, K., Yu, J.: Photo-realistic facial details synthesis from single image. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 9429–9439 (2019)
work page 2019
-
[7]
In: 2021 International Conference on Virtual Reality and Visualization (ICVR V)
Dapeng, Z., Yue, Q.: Generative contour guided occlusions removal 3d face recon- struction. In: 2021 International Conference on Virtual Reality and Visualization (ICVR V). pp. 74–79. IEEE (2021)
work page 2021
-
[8]
In: 2021 International Conference on Virtual Reality and Visualization (ICVR V)
Dapeng, Z., Yue, Q.: Learning detailed face reconstruction under occluded scenes. In: 2021 International Conference on Virtual Reality and Visualization (ICVR V). pp. 80–84. IEEE (2021)
work page 2021
Show all 41 references
-
[9]
In: Proc
Dou, P., Wu, Y., Shah, S.K., Kakadiaris, I.A.: Robust 3d face shape reconstruc- tion from single images via two-fold coupled structure learning. In: Proc. British Machine Vision Conference. pp. 1–13 (2014)
2014
-
[10]
In: Proceedings of the European Conference on Computer Vision (ECCV)
Feng, Y., Wu, F., Shao, X., Wang, Y., Zhou, X.: Joint 3d face reconstruction and dense alignment with position map regression network. In: Proceedings of the European Conference on Computer Vision (ECCV). pp. 534–551 (2018)
2018
-
[11]
In: 2018 13th IEEE In- ternational Conference on Automatic Face & Gesture Recognition (FG 2018)
Gerig, T., Morel-Forster, A., Blumer, C., Egger, B., Luthi, M., Sch¨ onborn, S., Vetter, T.: Morphable face models-an open framework. In: 2018 13th IEEE In- ternational Conference on Automatic Face & Gesture Recognition (FG 2018). pp. 75–82. IEEE (2018)
2018
-
[12]
In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
Gilani, S.Z., Mian, A.: Learning from millions of 3d scans for large-scale 3d face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 1896–1905 (2018)
2018
-
[13]
arXiv preprint arXiv:2009.09960 (2020)
Guo, J., Zhu, X., Yang, Y., Yang, F., Lei, Z., Li, S.Z.: Towards fast, accurate and stable 3d dense face alignment. arXiv preprint arXiv:2009.09960 (2020)
2020 arXiv
-
[14]
In: Proceedings of the IEEE conference on computer vision and pattern recognition
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)
2016
-
[15]
In: Proceedings of the IEEE/CVF International Conference on Computer Vision
Howard, A., Sandler, M., Chu, G., Chen, L.C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., Vasudevan, V.: Searching for mobilenetv3. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 1314–1324 (2019)
2019
-
[16]
ACM Transactions on graphics (TOG) 33(4), 1–10 (2014)
Huang, J.B., Kang, S.B., Ahuja, N., Kopf, J.: Image completion using planar struc- ture guidance. ACM Transactions on graphics (TOG) 33(4), 1–10 (2014)
2014
-
[17]
In: Proceedings of the IEEE conference on computer vision and pattern recognition
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with condi- tional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1125–1134 (2017) Generative Landmarks Guided Eyeglasses Removal 3D Face Recon...
2017
-
[18]
arXiv preprint arXiv:1710.10196 (2017)
Karras, T., Aila, T., Laine, S., Lehtinen, J.: Progressive growing of gans for im- proved quality, stability, and variation. arXiv preprint arXiv:1710.10196 (2017)
2017 arXiv
-
[19]
In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
Kumar, A., Chellappa, R.: Disentangling 3d pose in a dendritic cnn for uncon- strained 2d face alignment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 430–439 (2018)
2018
-
[20]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Lee, C.H., Liu, Z., Wu, L., Luo, P.: Maskgan: Towards diverse and interactive facial image manipulation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 5549–5558 (2020)
2020
-
[21]
In: Proceedings of the IEEE conference on computer vision and pattern recognition
Li, Y., Liu, S., Yang, J., Yang, M.H.: Generative face completion. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3911–3919 (2017)
2017
-
[22]
arXiv preprint arXiv:2003.05653 (2020)
Lin, J., Yuan, Y., Shao, T., Zhou, K.: Towards high-fidelity 3d face reconstruc- tion from in-the-wild images using graph convolutional networks. arXiv preprint arXiv:2003.05653 (2020)
2020 arXiv
-
[23]
M¨ uller, C.: Spherical harmonics, vol. 17. Springer (2006)
2006
-
[24]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Pan, J., Wang, C., Jia, X., Shao, J., Sheng, L., Yan, J., Wang, X.: Video generation from single semantic label map. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3733–3742 (2019)
2019
-
[25]
In: 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance
Paysan, P., Knothe, R., Amberg, B., Romdhani, S., Vetter, T.: A 3d face model for pose and illumination invariant face recognition. In: 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance. pp. 296–301. Ieee (2009)
2009
-
[26]
In: Proceedings of the 28th annual conference on Computer graphics and interactive techniques
Ramamoorthi, R., Hanrahan, P.: An efficient representation for irradiance environ- ment maps. In: Proceedings of the 28th annual conference on Computer graphics and interactive techniques. pp. 497–500 (2001)
2001
-
[27]
In: Proceedings of the 28th annual conference on Computer graphics and interactive techniques
Ramamoorthi, R., Hanrahan, P.: A signal-processing framework for inverse ren- dering. In: Proceedings of the 28th annual conference on Computer graphics and interactive techniques. pp. 117–128 (2001)
2001
-
[28]
In: 2011 International Joint Conference on Biometrics (IJCB)
Rara, H.M., Farag, A.A., Davis, T.: Model-based 3d shape recovery from single im- ages of unknown pose and illumination using a small number of feature points. In: 2011 International Joint Conference on Biometrics (IJCB). pp. 1–7. IEEE (2011)
2011
-
[29]
In: 2016 Fourth International Conference on 3D Vision (3DV)
Richardson, E., Sela, M., Kimmel, R.: 3d face reconstruction by learning from synthetic data. In: 2016 Fourth International Conference on 3D Vision (3DV). pp. 460–469. IEEE (2016)
2016
-
[30]
In: International Conference on Medical image computing and computer-assisted intervention
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedi- cal image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. pp. 234–241. Springer (2015)
2015
-
[31]
In: Proceedings of the IEEE conference on computer vision and pattern recognition
Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: A unified embedding for face recognition and clustering. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 815–823 (2015)
2015
-
[32]
In: Proceedings of the IEEE International Conference on Computer Vision
Sela, M., Richardson, E., Kimmel, R.: Unrestricted facial geometry reconstruc- tion using image-to-image translation. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 1576–1585 (2017)
2017
-
[33]
In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
Tuan Tran, A., Hassner, T., Masi, I., Medioni, G.: Regressing robust and discrim- inative 3d morphable models with a very deep neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 5163–5172 (2017)
2017
-
[34]
In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
Tuan Tran, A., Hassner, T., Masi, I., Paz, E., Nirkin, Y., Medioni, G.: Extreme 3d face reconstruction: Seeing through occlusions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 3935–3944 (2018) 12 D Zhao,Y Qi
2018
-
[35]
Wang, S., Cheng, Z., Deng, X., Chang, L., Duan, F., Lu, K.: Leveraging 3d blend- shape for facial expression recognition using cnn. Sci. China Inf. Sci 63(120114), 1–120114 (2020)
2020
-
[36]
In: 2007 International Conference on Computational Intelligence and Security Workshops (CISW 2007)
Wang, Z.M., Tao, J.H.: Reconstruction of partially occluded face by fast recursive pca. In: 2007 International Conference on Computational Intelligence and Security Workshops (CISW 2007). pp. 304–307. IEEE (2007)
2007
-
[37]
In: Proceedings of the IEEE conference on computer vision and pattern recognition
Wu, W., Qian, C., Yang, S., Wang, Q., Cai, Y., Zhou, Q.: Look at boundary: A boundary-aware face alignment algorithm. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2129–2138 (2018)
2018
-
[38]
In: 2021 International Conference on Virtual Reality and Visualization (ICVR V)
Yongkang, Z., Jun, L., Zhiping, S., Na, j., Zhilei, L.: Tssn: Temporal self-attention and self-supervision network for efficient action recognition. In: 2021 International Conference on Virtual Reality and Visualization (ICVR V). pp. 87–92. IEEE (2021)
2021
-
[39]
In: Proceedings of the IEEE International Conference on Computer Vision
Zeng, X., Peng, X., Qiao, Y.: Df2net: A dense-fine-finer network for detailed 3d face reconstruction. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 2315–2324 (2019)
2019
-
[40]
In: Computer Graphics International Conference
Zhao, D., Qi, Y.: Generative face parsing map guided 3d face reconstruction under occluded scenes. In: Computer Graphics International Conference. pp. 252–263. Springer (2021)
2021
-
[41]
In: Proc
Zhou, X., Leonardos, S., Hu, X., Daniilidis, K.: 3d shape reconstruction from 2d landmarks: A convex formulation. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition. pp. 4447–4455. Citeseer (2015)
2015
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.