{"id":"21bbc590-5572-4f69-a305-d96d4d51b1e2","arxiv_id":"2508.03241","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"FFHQ-Makeup provides 90K paired bare/makeup images across 18K identities with five styles each, generated by a pair-free 3DMM-guided diffusion transfer method.","lead":"This paper introduces FFHQ-Makeup, a synthetic dataset pairing 18,000 faces with five makeup styles each, for a total of 90,000 bare/makeup image pairs. It uses a 3D-morphable-model-guided makeup transfer pipeline to preserve identity and expression across the pairs.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The disentanglement claim is only as strong as the unmeasured purity of the 3DMM makeup residual; if R leaks identity or geometry, the claimed pair consistency is unsupported.","rationale":"The reader's conditional verdict is appropriate. The strongest claim is that FFHQ-Makeup provides 90K pairs with facial consistency. That claim rests on the disentanglement of identity and makeup, which is achieved through the residual R. If R is contaminated, the bare-makeup pairs are not actually paired in the claimed sense; the makeup style may carry source identity or geometry, and the dataset would not satisfy P_consistency as defined. This is more load-bearing than the missing release or small evaluation, because those affect usability and evidence strength but not the internal validity of the construction. The paper does provide supporting evidence: two ablations show that removing the residual or the augmentation hurts consistency, and manual filtering removes obvious failures. However, ablations are relative improvements; they do not establish the absolute purity of R. A synthetic ground-truth test would settle the issue. If the test shows low leakage and high makeup recovery, the central claim holds. If not, the dataset may still be useful, but the makeup-only residual story and the consistency guarantee must be weakened. This is exactly the condition the reader attached, so the verdict remains unchanged as CONDITIONAL pending the residual-purity check.","tokens_in":11366,"tokens_out":5417,"duration_ms":66651,"concrete_test":"Construct a synthetic ground-truth test set using the same FLAME/3DMM pipeline: render 100 identities from held-out meshes with known identity albedo, expression, and lighting, then add known synthetic makeup textures to produce paired I_bare and I_makeup. Run the paper's fitting and reconstruction to obtain \\hat I_b and R = I_makeup - \\hat I_b. Measure (i) makeup recovery: cosine similarity or L1 error between R and the ground-truth makeup texture in the masked face region; (ii) identity leakage: train a lightweight classifier (or use ArcFace embeddings) on R alone to predict the source identity and compare against chance accuracy, and also measure ArcFace similarity between R and the true bare-face albedo. If identity prediction is far above chance or makeup recovery is low, the core disentanglement premise fails and the consistency claim needs re-evaluation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.1 defines the makeup residual as R = I_S - \\hat I_b, where \\hat I_b is a bare face reconstructed by fitting a 3DMM to the makeup image. The whole pipeline and the central consistency claim presuppose that R contains makeup appearance only, with no identity, geometry, lighting, or fitting error. This is not established. 3DMM fitting of a makeup face is ill-posed: makeup changes the observed albedo, so the optimizer may fold makeup into shape or shading, or leave source-specific pores, wrinkles, and illumination in R. The sampling-and-re-rendering step then transports whatever is in R onto every target geometry, so any non-makeup content becomes a source of cross-identity contamination in all 90K pairs. The ablations show that the residual representation helps, but they never measure how much identity or geometry actually remains in R. The paper's own limitation section admits the results are 'partially affected by the accuracy of 3DMM fitting,' and no diagnostic quantifies that effect. Without such a measurement, the quantitative superiority in Table 3 could be attributed to the ControlNet structure prior rather than to a truly makeup-only residual.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces FFHQ-Makeup, a synthetic paired bare-makeup dataset built from 18K FFHQ identities, each paired with five makeup styles for a total of 90K image pairs. The generation pipeline uses a 3DMM to reconstruct a bare face from a single makeup image, computes a makeup residual R = I_S - \\hat I_b, augments the residual by sampling and re-rendering on diverse target geometries, and feeds the augmented residual through a Stable-Makeup/FreeUV-style diffusion pipeline with ControlNet for structural guidance. The authors claim that this disentangles identity from makeup, and they report quantitative and visual preference comparisons against LADN-Syn and BeautyBank, plus ablations. The paper's main contribution is the dataset itself, with the generation method supporting its construction.","tokens_in":11620,"tokens_out":3507,"duration_ms":42913,"significance":"If the dataset is released as promised, it would fill a real gap: existing paired synthetic makeup datasets are either small, not public, or suffer from identity/expression drift. The scale (18K identities, 5 styles per identity, 90K pairs), the multi-style per-identity design, and the public availability are concrete strengths. The paper also uses external metrics (ArcFace, DINO-I, SSIM) rather than relying only on its own outputs, and the ablation study is a useful check on the design choices. However, the central claim that the 3DMM residual is makeup-only is not directly measured, and the reported quantitative advantages are presented without variance or significance tests. The manual curation steps are also not specified, so the dataset-level comparison may partly reflect the filtering effort rather than the method. These issues are fixable and do not invalidate the resource itself, but they need to be addressed before the empirical claims can be accepted at face value.","major_comments":[{"comment":"The central disentanglement claim rests on the residual R = I_S - \\hat I_b being a makeup-only signal, but the paper never measures how much identity, geometry, lighting, or fitting error remains in R. 3DMM fitting from a makeup image is ill-posed because makeup alters the observed albedo, so the optimizer may fold makeup into shape or shading, and source-specific structure such as nostrils, wrinkles, or illumination can leak into the residual. Since the sampling-and-re-rendering step transports R onto every target geometry, any non-makeup content becomes a source of cross-identity contamination in all 90K pairs. The ablations in Section 4.2 show that the residual representation helps, but they do not quantify residual purity. The paper's own limitation section (Section 5.1) admits that results are \"partially affected by the accuracy of 3DMM fitting,\" yet no diagnostic quantifies this. I recommend adding a direct measurement: for example, train a classifier or linear probe on R to see whether source identity can be recovered, or report ArcFace similarity between residuals rendered on different geometries, or decompose the residual into makeup vs. non-makeup components on a small labeled set. Without such evidence, the superiority of FFHQ-Makeup in Table 3 could be attributed to the ControlNet structure prior rather than to a truly makeup-only residual.","section":"Section 3.1"},{"comment":"The quantitative claims of superiority are based on single point estimates of ArcFace, DINO-I, and SSIM, with no variances, confidence intervals, or significance tests. This matters because the dataset-level comparison samples 90K images from each dataset, so even small differences could be statistically unreliable; the method-level rows in the same table are presumably on a smaller subset and may not be directly comparable. I recommend reporting bootstrap confidence intervals or per-identity/per-style error bars, and a paired significance test where appropriate (for example, when the same target identities are generated by different variants). Without this, \"outperforms\" in the abstract and conclusion is not yet supported.","section":"Section 4.4, Table 3"},{"comment":"The visual preference study uses 50 groups judged by GPT-4o, with no human validation, no inter-annotator agreement measure, and no statistical test. A vision-language model may have systematic preferences (for example, favoring smoother images or penalizing certain artifacts) that do not match human judgment, and 50 groups is a small sample. I recommend either adding a human preference study (even on a subset) or reporting agreement between GPT-4o and human raters, and providing confidence intervals for the preference percentages.","section":"Section 4.1, Table 2"},{"comment":"The dataset construction pipeline relies on several manual filtering steps: removing extreme makeup styles, manually masking failed segmentation regions, filtering out FFHQ samples with inaccurate 3DMM fitting, and group-wise visual inspection of the final outputs. These steps are not described with explicit criteria, and the dataset-level evaluation in Section 4.1 and Table 3 is performed on the final filtered dataset. As a result, the comparison against LADN-Syn and BeautyBank may partly measure the curation effort rather than the generation method. I recommend documenting the filtering criteria, reporting the number of discarded samples at each stage, and, if feasible, running the dataset-level metrics on unfiltered outputs to separate the method's contribution from the manual selection effect.","section":"Section 3.3"}],"minor_comments":[{"comment":"There is a typo in the text: \"Stable-Stable-Makeup\" should be \"Stable-Makeup,\" and the sentence starting \"Stable-Stable-Makeup [51] proposed...\" is grammatically incomplete.","section":"Section 2.1"},{"comment":"The phrase \"existing makeup datasets construction still fail short\" should be revised to \"existing makeup dataset construction still falls short,\" and the abstract contains \"this is the first work that focuses specifically on constructing makeup dataset,\" which should be softened and made grammatical since prior works such as LADN-Syn and BeautyBank also construct synthetic datasets.","section":"Section 1"},{"comment":"Figure 8 contains unlabeled numeric annotations (e.g., 74, 178, 59, 426, 68766) whose meaning is unclear; please explain them in the caption or text.","section":"Figure 8"},{"comment":"References [1] and [2] are the same paper (Alashkar et al., AAAI 2017, \"Examples-rules guided deep neural network for makeup recommendation\"); please merge them and use a single reference.","section":"References"},{"comment":"The sentence \"This self-supervised approach enables scalable and flexible for synthetic makeup dataset generation\" is ungrammatical; consider \"This self-supervised approach enables scalable and flexible synthetic makeup dataset generation.\"","section":"Section 3.1"}],"recommendation":"major_revision","confidential_remarks":"The dataset itself is likely to be a useful community resource if released with clear documentation and license terms. The main technical risk is the unmeasured purity of the 3DMM residual, which is the foundation of the disentanglement claim; I would like to see a direct diagnostic before endorsing the central claim. The paper also overstates its novelty in the abstract and conclusion given that LADN-Syn and BeautyBank already construct synthetic paired datasets, even if not as their primary focus. The self-citations to FreeUV and Stable-Makeup are appropriate given the method builds directly on those works."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"FFHQ-Makeup is a useful resource, provided the dataset actually ships. 18K identities, five styles each, 90K pairs of bare and makeup images that hold identity and expression reasonably well—that fills a real gap. The relative comparison against LADN-Syn and BeautyBank is appropriate, and the numbers favor FFHQ-Makeup on ArcFace and DINO-I. The residual-based feature extraction combined with sampling and re-rendering is a reasonable idea, and the ablations show it helps: dropping the residual causes identity leakage, and skipping re-rendering leaves structural artifacts. The limitation section is honest about 3DMM fitting and segmentation issues.\n\nThe soft spots are real, but they mostly affect the method claims rather than the dataset itself. The stress-test concern about residual purity is fair: R = I_S - \\hat I_b is assumed to be makeup-only, but 3DMM fitting of a makeup face is ill-posed. Makeup can be folded into shape or shading, and pores, wrinkles, or illumination can leak into R. The paper never measures how much identity or geometry is actually in R. The end-to-end consistency results are good, but they could be carried by the ControlNet structure prior rather than by a clean residual. The paper should either quantify residual purity or soften the disentanglement claim. The ablation shows the residual representation matters, but not that it is pure.\n\nEvaluation is thinner than it should be. Metrics are reported without variance or significance tests. The visual preference study is 50 groups judged by GPT-4o with no human validation; that is a weak basis for a dataset-quality claim. Manual filtering lacks explicit criteria. And the 'first dataset-construction-focused' claim is overstated—BeautyBank and LADN-Syn exist, even if not as clean. The biggest practical issue is that no dataset link is provided; 'open-source' is asserted but not substantiated.\n\nNone of this kills the paper. The dataset, if released, will be a useful benchmark for makeup transfer, privacy, and beauty analysis. The method is incremental but sensible. I would send this to peer review, with a strong request to release the data and add human evaluation. I would not cite it until the dataset is actually available. The disentanglement story needs tempering, not surgery.","headline":"A genuinely useful paired makeup dataset if it ships, with an overstated disentanglement story and thin evaluation.","tokens_in":12077,"tokens_out":3541,"would_cite":false,"duration_ms":43151,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper builds a synthetic dataset of 18K identities, each paired with five makeup styles, keeping identity and expression fixed across the bare–makeup pairs.","keywords":["FFHQ-Makeup","synthetic makeup dataset","makeup transfer","3D morphable model","diffusion models","facial consistency","paired bare-makeup images","face editing"],"falsifier":"Take a makeup image, fit the 3DMM, render the bare reconstruction, and form the residual; feed the residual to the trained transfer model and generate makeup on a different identity. Then run ArcFace on the generated face against the original makeup-image identity: if this identity similarity is substantially above chance, the residual still encodes identity and the disentanglement claim is false. A simpler check is to render the residual colors onto a probe 3D face and ask an identity-recognition model whether it matches the source subject.","tokens_in":11221,"feed_emoji":"💄","tokens_out":10477,"duration_ms":111878,"temperature":0.7,"pith_summary":"FFHQ-Makeup is a large synthetic dataset of 90,000 paired bare–makeup face images: 18,000 identities from FFHQ, each paired with five different makeup styles at 512×512 resolution. The paper's goal is to break the trade-off that has made such pairs scarce: real before-and-after photos are costly to collect, warping-based synthesis distorts faces, and text-to-image synthesis drifts identity and expression. The authors' generation pipeline fits a 3D morphable model to a makeup image, reconstructs a bare version of the same face, subtracts it to isolate a makeup residual, and re-renders that residual onto many different target faces before training a diffusion makeup-transfer model. Their central claim is that this residual-based disentanglement, together with structure control from the reconstructed face and landmarks, produces bare–makeup pairs that change makeup while keeping identity and expression fixed, and the resulting dataset outperforms existing synthetic paired datasets on identity similarity and semantic consistency.","feed_headline":"Dataset pairs 18K faces with five makeup styles each","feed_subtitle":"3D-reconstructed bare faces are subtracted and re-rendered so makeup changes but identity stays fixed.","key_machinery":"The load-bearing object is the 3DMM-based makeup residual $R=I_S-\\hat{I}_b$, where $\\hat{I}_b$ is a bare face reconstructed by fitting a 3D morphable model (a statistical model of 3D face shape and texture) to the makeup image and blending it with the background. Subtracting the reconstruction strips away identity, expression, pose, and illumination cues, leaving appearance tied to makeup; sampling the residual's vertex-wise colors and re-rendering them onto many different reconstructed face geometries further enforces invariance to facial structure. The makeup-transfer network then reads these augmented residuals with a frozen CLIP image encoder and a channel-attention residual learner, while ControlNet conditions generation on the reconstructed bare face and detected landmarks. This separation of appearance features from structural guidance is what lets the pipeline generate 90K paired images while holding identity and expression fixed.","core_discovery":"On the paper's own terms, the core discovery is that faithful bare–makeup pairs can be synthesized at scale without any real paired supervision. For a makeup image $I_S$, the method fits a 3D morphable model to capture identity, expression, pose, illumination, and skin tone, renders a reconstructed bare face $\\hat{I}_b$, blends it with the original background, and forms the makeup residual $R=I_S-\\hat{I}_b$; because this residual lives on the reconstructed 3D surface, its color values can be sampled and re-rendered onto other faces. The network therefore never sees the residual attached to the source face's geometry, and the makeup style is decoupled from facial structure. Trained with this representation and guided by ControlNet on the reconstructed bare face and landmarks, the model transfers real makeup styles onto 18,000 FFHQ identities, five styles each, producing 90,000 pairs. The paper reports that these pairs achieve higher ArcFace identity similarity and DINO-I semantic consistency than the existing LADN-Syn and BeautyBank synthetic datasets.","pith_inferences":["A testable consequence of the disentanglement claim is that an identity-recognition model should not be able to match a residual-rendered face back to the original makeup image's identity; if it can, the residual is leaking identity.","The same residual sampling-and-re-rendering trick could in principle be applied to other appearance attributes that lack paired data, such as beard growth, skin texture, or aging, whenever a 3D prior can reconstruct the attribute-free version.","Because the style pool is limited to MT and LADN and the identity pool to FFHQ, the reported consistency gains should be re-tested when either pool is expanded, since the current numbers could partly reflect the curated, well-aligned source material.","Independent human raters, rather than a vision-language model, would strengthen the visual-preference comparison; if humans do not share the reported preference, the perceptual claim would need qualification."],"forward_implications":["A public 90K-pair dataset is available for beauty-related tasks that previously had to train on unpaired or low-quality data.","Models for makeup-invariant face recognition and facial privacy protection can be trained and benchmarked on consistent bare–makeup pairs at scale.","Because each identity appears with five styles, the dataset supports identity-conditioned style transfer and controlled studies of how makeup affects recognition.","The pair-free generation pipeline can be re-run on new face datasets, limited mainly by 3DMM fitting and segmentation quality, so the dataset can grow beyond FFHQ.","Other synthetic makeup datasets can now be compared directly against FFHQ-Makeup on identity and semantic consistency metrics, giving the field a common evaluation set."],"supporting_citations":[{"why":"Supplies the Stable-Makeup diffusion backbone that the generation method builds on and the paired baseline it is compared with.","marker":"[51]"},{"why":"Contributes the FreeUV channel-attention feature extraction architecture used in the makeup residual learner.","marker":"[48]"},{"why":"Provides the FLAME 3D morphable face model used to reconstruct bare faces and define the residual geometry.","marker":"[24]"},{"why":"Gives the 3DMM fitting procedure used to obtain each reconstructed 3D face from a makeup image.","marker":"[46]"},{"why":"Provides ControlNet structural conditioning that keeps pose, expression, and geometry aligned between pair images.","marker":"[50]"},{"why":"Provides the pretrained Stable Diffusion model that underlies the makeup transfer generation.","marker":"[34]"},{"why":"Supplies the FFHQ target identity set whose 18K faces give the dataset its facial diversity.","marker":"[23]"},{"why":"Supplies the MT real-makeup style source used as appearance examples in training.","marker":"[25]"},{"why":"Supplies the LADN real-makeup style source and the LADN-Syn comparison dataset used in evaluation.","marker":"[15]"},{"why":"Supplies the BeautyBank synthetic paired dataset used as the main comparison baseline.","marker":"[29]"}],"fun_headline_variants":["90K synthetic makeup pairs, no real pairs needed","18K faces get five makeup looks, identity preserved","First synthetic bare-makeup dataset, 90K pairs","Paired bare-makeup images at scale, identity fixed","Makeup transfer without real pairs, 90K examples"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole pipeline rests on the assumption that fitting a single 3D morphable model to a makeup image reconstructs the bare face accurately enough that subtracting it leaves a residual containing the makeup but none of the person's identity or facial geometry.","fun_headline_variants_meta":{"raw":{"variants":["90K synthetic makeup pairs, no real pairs needed","18K faces get five makeup looks, identity preserved","First synthetic bare-makeup dataset, 90K pairs","Paired bare-makeup images at scale, identity fixed","Makeup transfer without real pairs, 90K examples"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001506,"raw_usage":{"total_tokens":6076,"prompt_tokens":1019,"completion_tokens":5057,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":635,"completion_tokens_details":{"reasoning_tokens":4976}},"tokens_in":635,"tokens_out":5057,"duration_ms":37591,"temperature":1.0,"reasoning_tokens":4976,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T04:32:55.771627+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a makeup image, fit the 3DMM, render the bare reconstruction, and form the residual; feed the residual to the trained transfer model and generate makeup on a different identity. Then run ArcFace on the generated face against the original makeup-image identity: if this identity similarity is substantially above chance, the residual still encodes identity and the disentanglement claim is false. A simpler check is to render the residual colors onto a probe 3D face and ask an identity-recognition model whether it matches the source subject.","supporting_citations":[{"cited_title":"Freeuv: Ground-Truth-Free realistic facial UV texture recovery via cross-assembly inference strategy","cited_arxiv_id":null,"evidence_quote":"Contributes the FreeUV channel-attention feature extraction architecture used in the makeup residual learner."},{"cited_title":"Black, Hao Li, and Javier Romero","cited_arxiv_id":null,"evidence_quote":"Provides the FLAME 3D morphable face model used to reconstruct bare faces and define the residual geometry."},{"cited_title":"Makeup extraction of 3D representation via illumination-aware image decomposition","cited_arxiv_id":null,"evidence_quote":"Gives the 3DMM fitting procedure used to obtain each reconstructed 3D face from a makeup image."},{"cited_title":"Adding conditional control to text-to-image diffusion models","cited_arxiv_id":null,"evidence_quote":"Provides ControlNet structural conditioning that keeps pose, expression, and geometry aligned between pair images."},{"cited_title":"High-Resolution image synthesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"Provides the pretrained Stable Diffusion model that underlies the makeup transfer generation."},{"cited_title":"A style-based generator architecture for generative adversarial networks","cited_arxiv_id":null,"evidence_quote":"Supplies the FFHQ target identity set whose 18K faces give the dataset its facial diversity."},{"cited_title":"BeautyGAN: Instance-level fa- cial makeup transfer with deep generative adversarial net- work","cited_arxiv_id":null,"evidence_quote":"Supplies the MT real-makeup style source used as appearance examples in training."},{"cited_title":"LADN: Local adversarial disentangling network for facial makeup and de-makeup","cited_arxiv_id":null,"evidence_quote":"Supplies the LADN real-makeup style source and the LADN-Syn comparison dataset used in evaluation."},{"cited_title":"BeautyBank: Encoding facial makeup in latent space","cited_arxiv_id":null,"evidence_quote":"Supplies the BeautyBank synthetic paired dataset used as the main comparison baseline."}],"review_version":1}