{"id":"4df4548c-16b5-4daf-9f20-61163ee962b0","arxiv_id":"2412.19848","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A landmark-guided GAN deletes and repaints eyeglasses regions, then a ResNet-50 regresses 3DMM coefficients from the cleaned face to produce a glasses-free 3D reconstruction.","lead":"This paper describes a two-stage pipeline that removes eyeglasses from a single face photo, then fits a 3D morphable model to the cleaned image. The claimed payoff is more robust 3D face reconstruction in the wild, where existing methods often assume unobstructed faces.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central mechanism is not specified: Eq. 2 trains the generator to reproduce the occluded input I_in, so per-pixel optimization preserves eyeglasses, and Eq. 5 omits the adversarial/perceptual losses that could justify inpainting.","rationale":"The reader's weakest assumption is correct and matches my independent reading. Section 3.2 is the only place the paper explains how it removes glasses, and it is internally inconsistent: Eq. 2 compares with the input, the text claims the input has no occlusion, Eq. 5 drops the adversarial/perceptual losses, and Eq. 3's style loss reinforces the glasses region. These are correctness risks, not just missing polish. The quantitative evidence (one MICC number, LFW verification without a non-glasses reconstruction baseline) would be weak even if the mechanism were sound, but the mechanism itself is the load-bearing issue. I also note reference [6] is the authors' own MMM 2022 paper with the same title, and the paper does not disclose this overlap; that strengthens the case that this is an unvalidated resubmission rather than new evidence. However, my recommendation does not change the reader's verdict; I find no additional objection that would alter REJECT.","tokens_in":8016,"tokens_out":3894,"duration_ms":35771,"concrete_test":"Reimplement the face synthesis module exactly as specified in Section 3.2 with L_fsm = λ_pixe L_pixe + λ_style L_style + λ_var L_var (Eq. 5), using I_in (the eyeglasses image) as the reconstruction target as Eq. 2 states, on a paired dataset such as CelebA-HQ with synthetic eyeglasses. On held-out eyeglass images, measure the mean |I_out - I_clean| inside the eyeglass mask and the fraction of eyeglass pixels surviving. If the model does not remove glasses or does so only when a clean target is substituted, the stated loss cannot support the central claim; the authors would need to specify the actual clean target and pairing protocol.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing claim is that the synthesis module removes eyeglasses before 3DMM regression (Section 3.2). This is the weakest link because the published loss does not reward removal. Eq. 2 defines L_pixe = (1/S)||I_out - I_in|| with S the mask size; for any pixel under the glasses, the unique minimizer is I_out = I_in, i.e., keeping the eyeglass value. The sentence directly below Eq. 2 states 'our input image will not contain occlusion,' but Fig. 1 and the whole pipeline show I_in is the eyeglasses-occluded photo; if the intended training target is a clean image, no source or pairing procedure for such targets is provided. Eq. 5, stated as the total synthesis loss, contains only L_pixe, L_style, and L_var; the adversarial and perceptual losses promised in the prose are absent from the equation, so the discriminator cannot be the mechanism that drives removal. Eq. 3's style loss also compares G(I_out ⊙ I_m) with G(I_in ⊙ I_m), i.e., in the glasses mask it penalizes divergence from the original glasses content. Because the downstream reconstruction in Section 3.3 takes I_out as its input (Eq. 8, L1 = ||I_out - I_y||_2^2), any failure to remove glasses propagates directly into shape and texture. The central claim therefore rests on an unstated data-availability and supervision assumption: clean-glasses pairs and a loss that targets the clean face. Without that, the method cannot learn the advertised behavior.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a two-stage method for single-image 3D face reconstruction under eyeglass occlusion. In Stage I, a face parsing module identifies and deletes the eyeglasses region, and a generative network synthesizes a glasses-free 2D face guided by face landmarks. In Stage II, a ResNet-50 regresses 3DMM shape, expression, texture, illumination, and pose parameters from the synthesized image, followed by a texture refinement network. The authors claim state-of-the-art qualitative performance and report quantitative results on MICC Florence (with artificially added eyeglasses) and LFW.","tokens_in":8308,"tokens_out":4454,"duration_ms":37020,"significance":"The task is relevant to in-the-wild face reconstruction, and the idea of combining face parsing, landmark conditioning, and a synthesis module is a reasonable design direction. If the method worked as described, it would address a practical limitation of current methods that assume unobstructed faces. However, the central training objective for glasses removal is not established in the manuscript, and the quantitative evidence is thin. The paper does not provide code, trained models, or a detailed training-data protocol, which would be necessary to verify the central claim.","major_comments":[{"comment":"The per-pixel loss L_pixe = (1/S)||Iout - Iin|| compares the synthesized output with the original input Iin. Since Iin is the eyeglasses-occluded photo in the pipeline of Fig. 1, minimizing this loss encourages Iout to reproduce the glasses rather than remove them. The sentence immediately below Eq. (2) states 'our input image will not contain occlusion,' which contradicts the pipeline description, and the paper never explains how a glasses-free reference image would be paired with the occluded input during training. Because the downstream 3D reconstruction consumes Iout (Eq. 8), this unstated supervision assumption is load-bearing.","section":"Section 3.2, Eq. (2)"},{"comment":"The total synthesis loss L_fsm contains only L_pixe, L_style, and L_var. The prose promises a combination of adversarial, per-pixel, perceptual, style, total variation, and adversarial losses, but the equation omits the adversarial and perceptual terms, and no discriminator objective is given. Without an adversarial or perceptual term, the generator has no incentive to inpaint the masked region with plausible facial structure, so the mechanism for intelligent glasses removal is absent.","section":"Section 3.2, Eq. (5)"},{"comment":"The style loss is computed on the masked region Im between Iout and Iin. If Iin contains eyeglasses, this term penalizes divergence from the original glasses texture and therefore reinforces the per-pixel loss in Eq. (2). This is inconsistent with the stated goal of removing eyeglasses and further undermines the training supervision.","section":"Section 3.2, Eq. (3)"},{"comment":"The quantitative evaluation does not support the state-of-the-art claim. The MICC Florence comparison reports single 90%-error numbers (Fig. 5) with no error bars, standard deviations, or details on how many subjects and which glasses augmentations were used. Table 1 on LFW compares only with Tran et al. and omits the baselines used in the qualitative comparisons (PRNet, 3DDFA, DF2Net, Chen et al.); moreover, the verification protocol is not described beyond a reference to 'the same face test system from Anh et al.'","section":"Section 4.3"}],"minor_comments":[{"comment":"Reference [24] is cited for the LFW dataset, but the cited paper is 'Video generation from single semantic label map' by Pan et al.; the LFW dataset should be cited with its original source (e.g., Huang et al.).","section":"References"},{"comment":"The text cites 'Sela et al.[17]' but reference [17] is Isola et al.; the intended citation is likely Sela et al. (reference [32]).","section":"Section 4.3"},{"comment":"The text refers to 'the paper of Deng et al. [29]' but reference [29] is Richardson et al.; please correct the citation.","section":"Section 4.3"},{"comment":"The relationship to the authors' prior works, especially reference [6] with the same title and venue (MMM 2022), should be clarified. If this arXiv manuscript is an extended version, the novelty relative to that publication should be stated explicitly.","section":"References and Contributions"},{"comment":"The denominator PIin in the total variation loss is not defined; it presumably denotes the number of pixels in Iin, but this should be stated.","section":"Eq. (4)"},{"comment":"The caption says digits denote '90% error (mm)' while the text says 'average 90% largest error'; please specify the exact metric and how it is computed.","section":"Fig. 5 and Section 4.3"},{"comment":"The conclusion states that the method 'outperforms previous arts by a large margin,' but the paper only provides qualitative evidence and a single MICC number; this claim should be tempered or supported with statistical tests.","section":"Conclusions"}],"recommendation":"reject","confidential_remarks":"The manuscript appears to overlap substantially with the authors' own MMM 2022 paper (reference [6]), which has the same title. The overlap in title, abstract, and core methodology raises a dual-submission or self-plagiarism concern that the editor may wish to investigate. Additionally, several incorrect citations (e.g., [24] for LFW, [17] for Sela et al., [29] for Deng et al.) suggest that the reference list was not carefully verified. The load-bearing training-supervision problem in Eqs. (2), (3), and (5) is, in my view, sufficient for rejection on technical grounds."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The idea at the center of this paper is reasonable: use face parsing and landmarks to guide a generative module that removes glasses before regressing a 3DMM. Applied to a real use case, that is a sensible way to attack occluded 3D face reconstruction, and the qualitative figures do show outputs that look cleaned up. So there is something here.\n\nBut the manuscript has problems that go beyond polish. The most serious is that reference [6] is the same paper, same title, same authors, published at MMM 2022. The arXiv text presents the contributions as new and never discloses the republication. That is a self-plagiarism issue that any venue should take seriously.\n\nThe technical core is also not supported as written. Eq. 2 defines the per-pixel loss as the distance between the synthesized output and the original input Iin. For a pixel under the glasses, minimizing that loss preserves the glasses, not removes them. The sentence under the equation says the input will not contain occlusion, but the pipeline and Figure 1 show Iin is the occluded photo. Where do clean/occluded pairs come from? The paper doesn't say. Eq. 5 lists the total synthesis loss as only the per-pixel, style, and total variation terms; the adversarial and perceptual losses promised in the prose are missing from the equation. So the mechanism that is supposed to drive glasses removal is not actually specified.\n\nThe evaluation is thin. The MICC comparison is a single aggregate number with no error bars and no note of how many subjects. The LFW table shows the method roughly matches Tran et al. without glasses but drops with glasses, which is not a strong state-of-the-art claim. There is no code, no data, and no comparison to the authors' own prior related works [7, 8, 40].\n\nI don't see fraud, but I do see an unpolished overlapping manuscript with a load-bearing inconsistency. The pipeline may work in practice, but this paper doesn't give us the evidence to check.\n\nI would not send it to peer review; the republication alone is a desk-reject, and the loss issue needs to be fixed before the claim can be evaluated. If the authors resubmit a properly disclosed, corrected version with real evaluation, that would be worth a look.","headline":"A plausible pipeline idea, but this is an undisclosed re-publication with an inconsistent loss and thin evaluation, so it should be desk-rejected.","tokens_in":8892,"tokens_out":2726,"would_cite":false,"duration_ms":22870,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a face-parsing-guided generator can delete eyeglasses from a single face photo, and that regressing a 3D morphable model from the cleaned image gives state-of-the-art 3D face reconstruction under occlusion.","keywords":["3D face reconstruction","eyeglasses removal","face parsing","3D morphable model","GAN inpainting","facial landmarks","occlusion handling","single-view reconstruction"],"falsifier":"Feed the trained synthesis module a face image with glasses and compare its output to a paired glasses-free image of the same person; if $\\mathcal{L}_{\\text{pixe}}$ in Eq. (2) was the only supervision on the masked pixels, the network has no gradient signal to force removal, so this paired test would reveal whether the removal is actually learned or merely a side effect of the shape prior.","tokens_in":7740,"feed_emoji":"👓","tokens_out":5616,"duration_ms":44457,"temperature":0.7,"pith_summary":"This paper aims to make single-image 3D face reconstruction work when the face wears eyeglasses, a case that current systems handle poorly because they assume unobstructed input. The proposed pipeline first locates and deletes the eyeglasses region using a face parsing map, then synthesizes a glasses-free face image with a generator conditioned on facial landmarks, and finally regresses a 3D morphable model (3DMM) from that cleaned image. The authors claim that this division into delete, synthesize, and reconstruct yields photorealistic glasses-free textures and state-of-the-art qualitative performance on in-the-wild images, with quantitative gains on the MICC Florence and LFW benchmarks. The method matters because eyeglasses are one of the most common real-world occlusions, and a robust pipeline would extend face reconstruction, recognition, and animation to ordinary photos.","feed_headline":"Eyeglasses removed, then 3D face rebuilt from one photo","feed_subtitle":"Face parsing and landmark guidance erase glasses before 3DMM regression, improving in-the-wild reconstruction.","key_machinery":"The load-bearing mechanism is the two-stage delete-then-synthesize face synthesis module: a face parsing map from a U-Net-based deleter identifies and masks the eyeglasses region, and a landmark-conditioned U-Net generator inpaints that region using the corrupted image and 68 facial landmarks as geometric guidance. This cleaned image is what the 3DMM regression network sees, so the quality of the final 3D face depends on the generator removing the glasses while preserving the identity and topology of the eyes, nose, and mouth.","core_discovery":"The paper's central claim is that eyeglasses occlusion can be handled by explicitly removing the glasses before 3D reconstruction, rather than by making the reconstructor robust to occlusion directly. A face-parsing network (the deleter) identifies the eyeglasses pixels and produces a corrupted image; a U-Net generator then fills the removed area, guided by 68 facial landmarks, under an adversarial discriminator; the resulting glasses-free image is fed to a fixed ResNet-50 that regresses 3DMM shape, expression, texture, lighting, and pose coefficients. The cleaned image is also used for pixel-wise and FaceNet-based feature losses against the rendered face, and a graph-convolutional network refines the texture. The authors report that this pipeline outperforms prior 3D face reconstruction methods when the input contains eyeglasses, including transparent and sunglasses.","pith_inferences":["A testable extension is to apply the same delete-synthesize-reconstruct recipe to other occlusion types, such as respirators or hands, and measure whether the 3DMM regressor transfers without retraining.","If the per-pixel loss truly compares the output to an occlusion-free input, the generator may be learning identity-preserving reconstruction rather than explicit glasses removal; an ablation that removes landmark conditioning would clarify what actually drives the removal.","The claimed robustness on LFW could be decomposed: a control experiment that crops out rather than inpaints the glasses region would separate the benefit of glasses removal from the benefit of the 3DMM shape prior itself.","The paper's own trick of artificially adding eyeglasses to clean 3D face datasets for evaluation suggests a cheap data-augmentation strategy: use the same pairing to supervise the removal stage, which the paper currently does not do."],"forward_implications":["If the pipeline works as claimed, existing 3DMM-based reconstruction systems can be wrapped with a glasses-removal front end and applied to in-the-wild photos with eyeglasses.","The face parsing map provides a natural way to extend the method to other occlusions, such as masks or respirators, by adding new parsing classes.","The cleaned image plus 3DMM rendering enables verification under occlusion, as shown by LFW experiments where removing glasses recovers recognition accuracy close to the no-glasses baseline.","Because the pipeline yields both a glasses-free texture and a 3DMM shape, it can feed downstream tasks like face recognition, animation, and avatar creation without requiring a clean input image."],"supporting_citations":[{"why":"Defines the 3DMM parametric face space that the reconstructor regresses.","marker":"[3]"},{"why":"Supplies the face parsing map supervision used by the deleter to identify eyeglasses pixels.","marker":"[20]"},{"why":"Provides the U-Net architecture used for both the deleter and the generator.","marker":"[30]"},{"why":"Provides the MobileNet-V3 backbone for the facial landmark estimation network.","marker":"[15]"},{"why":"Provides the fixed ResNet-50 network that regresses 3DMM coefficients from the cleaned image.","marker":"[14]"},{"why":"Provides FaceNet, the face recognition embedding used in the feature loss L2.","marker":"[31]"},{"why":"Supplies the MICC Florence 3D face dataset used for quantitative 90% error comparison.","marker":"[1]"},{"why":"Supplies the face verification test protocol used for the LFW evaluation.","marker":"[34]"},{"why":"Serves as a baseline 3DMM regression method compared in the LFW table.","marker":"[33]"}],"fun_headline_variants":["3D face from one photo after digital glasses removal","AI deletes eyeglasses, rebuilds 3D face from single shot","Glasses off: single-image 3D face reconstruction","Remove eyeglasses, reconstruct 3D face from one photo","Landmark-guided 3D face rebuild without eyeglasses"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The synthesis network is trained with a per-pixel loss (Eq. 2) that compares the generated image to the input image while the text states the input contains no occlusion, so it is never established how the generator learns to remove eyeglasses rather than to reproduce the input.","fun_headline_variants_meta":{"raw":{"variants":["3D face from one photo after digital glasses removal","AI deletes eyeglasses, rebuilds 3D face from single shot","Glasses off: single-image 3D face reconstruction","Remove eyeglasses, reconstruct 3D face from one photo","Landmark-guided 3D face rebuild without eyeglasses"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000651,"raw_usage":{"total_tokens":2988,"prompt_tokens":953,"completion_tokens":2035,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":569,"completion_tokens_details":{"reasoning_tokens":1947}},"tokens_in":569,"tokens_out":2035,"duration_ms":12382,"temperature":1.0,"reasoning_tokens":1947,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:20:12.119749+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Feed the trained synthesis module a face image with glasses and compare its output to a paired glasses-free image of the same person; if $\\mathcal{L}_{\\text{pixe}}$ in Eq. (2) was the only supervision on the masked pixels, the network has no gradient signal to force removal, so this paired test would reveal whether the removal is actually learned or merely a side effect of the shape prior.","supporting_citations":[{"cited_title":"In: Siggraph","cited_arxiv_id":null,"evidence_quote":"Defines the 3DMM parametric face space that the reconstructor regresses."},{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"Supplies the face parsing map supervision used by the deleter to identify eyeglasses pixels."},{"cited_title":"In: Proceedings of the IEEE/CVF International Conference on Computer Vision","cited_arxiv_id":null,"evidence_quote":"Provides the MobileNet-V3 backbone for the facial landmark estimation network."},{"cited_title":"In: Proceedings of the 2011 joint ACM workshop on Human gesture and behavior understanding","cited_arxiv_id":null,"evidence_quote":"Supplies the MICC Florence 3D face dataset used for quantitative 90% error comparison."},{"cited_title":"In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"Supplies the face verification test protocol used for the LFW evaluation."},{"cited_title":"In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"Serves as a baseline 3DMM regression method compared in the LFW table."}],"review_version":1}