Pith. sign in

REVIEW 4 major objections 5 minor 58 references

SHMT: Self-supervised Hierarchical Makeup Transfer via Latent Diffusion Models

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SHMT transfers makeup with a self-supervised latent diffusion model, removing pseudo-paired data and beating seven baselines on realism and makeup fidelity.

desk verdict Self-supervised training without pseudo-pairs is a real departure, but the SOTA claim is undercut by metrics that reward copying reference identity; no identity metric is reported. read the letter →

arxiv 2412.11058 v1 pith:TH3VYT5Q submitted 2024-12-15 cs.CV

Zhaoyang Sun , Shengwu Xiong , Yaxiong Chen , Fei Du , Weihua Chen , Fan Wang , Yi Rong This is my paper · ORCID
classification cs.CV
keywords makeuptransferlatentdiffusionmodelsself-supervisedlearningLaplacianpyramidspatialalignmentfaceimageeditingdecoupling-and-reconstructionIterativeDual
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SHMT is a makeup-transfer method that claims to remove the need for pseudo-paired training data, which previous methods synthesize by histogram matching or geometric warping and which the authors argue misleads the model. The method trains a latent diffusion model to reconstruct a face from separated content and makeup representations, then transfers makeup at inference by swapping in a reference image's makeup representation. To handle the fact that simple and elaborate makeup styles affect the face differently, it decomposes grayscale source texture with a Laplacian pyramid and lets the user choose a fine or coarse detail level. The paper reports that this self-supervised approach surpasses seven baselines, including a diffusion-based rival, on FID, CLS (makeup fidelity), and Key-sim (content preservation), and that the choice of texture level trades content preservation against makeup fidelity.

What carries the argument

The load-bearing mechanism is a 'decoupling-and-reconstruction' pipeline built on a latent diffusion model. A pretrained face parser separates background from foreground; the makeup representation is produced by randomly cropping, rotating, and elastically distorting the foreground, on the premise that these transformations destroy identity and geometry while preserving cosmetic color and texture. The content representation is the combination of a 3D face shape from a face-reconstruction model and one level of a Laplacian pyramid, a multi-scale decomposition of an image into successively coarser high-frequency bands, computed on the grayscale foreground. Fine levels (h0) retain source details such as freckles and eyelashes, while coarse levels (h4) force the model to pull high-frequency makeup from the reference. At each denoising step, the Iterative Dual Alignment module computes a softmax correlation between content and makeup features, deforms the makeup latent with it, then blends in a second alignment computed between the noisy intermediate image and the makeup representation, with the blend weight predicted by a small MLP from the timestep; the mixed condition is injected into the U-Net encoder and the diffusion model is trained with the standard noise-prediction loss.

What would settle it

Compare the makeup representation extracted from an original foreground with the representation from its randomly distorted version on faces with strongly asymmetric makeup: if the distorted representation no longer encodes which eye carried the heavy shadow, or if the reconstruction drops that asymmetry, the self-supervised premise fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, SHMT establishes that high-fidelity makeup transfer can be learned entirely from single-image reconstruction rather than from synthesized source-reference pairs. The authors report that the variant SHMT-h4, which injects the coarsest Laplacian texture detail into the content representation, achieves the best FID and CLS values on the MT, Wild-MT, and LADN test sets, while SHMT-h0 achieves the best Key-sim, quantifying an explicit tradeoff between transferring the reference's makeup and preserving the source's content. An additional Iterative Dual Alignment module dynamically re-aligns the makeup feature against the noisy intermediate image at every denoising step, correcting the semantic alignment errors caused by the domain gap between content and makeup representations. The paper also demonstrates flexible controls: staggering SHMT-h0 and SHMT-h4 across denoising timesteps produces seamless interpolation, and mixing aligned makeup feature maps yields global or local makeup interpolation.

Load-bearing premise

Randomly cropping, rotating, and elastically warping a face erases its identity and geometry while leaving its makeup intact, and the pretrained face-parsing and 3D-reconstruction models used to separate content are accurate enough not to break the reconstruction.

Editorial extensions

If this is right

  • Choosing texture level h0 preserves source details like freckles, eyelashes, and subtle expressions, while h4 reproduces heavy or high-frequency reference makeup; staggering the two models over denoising timesteps gives a seamless interpolation between those behaviors.
  • The aligned makeup feature maps support global makeup interpolation between two reference faces and local interpolation restricted to lip or eye regions, with the source image serving as the neutral endpoint.
  • Because IDA re-aligns against the noisy intermediate image, the model is robust to large differences in pose, expression, age, and gender between source and reference, and it generalizes to sketch and anime images outside its training distribution.
  • The reported results imply that pseudo-paired data, however carefully synthesized, is not a necessary ingredient for makeup transfer; a reconstruction-only training objective can beat methods that invest in better pseudo-pair generation.
  • Skin tone can be preserved by interpolating the makeup feature within the face region toward the source's own feature, with the interpolation weight at zero leaving the source skin unchanged.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If random distortion truly cancels content while keeping makeup, the same decoupling recipe should transfer to other localized appearance edits without paired data, such as hair color changes, tattoos, or temporary age effects.
  • The Laplacian level index acts as a continuous texture-transfer dial; a natural extension would be a single model conditioned on the level rather than one separately trained model per level.
  • The failure mode the paper shows, face parsing labeling forehead makeup as hair, sets a ceiling on worst-case quality, suggesting that a makeup-aware segmentation module or learned foreground mask would improve robustness.
  • IDA's timestep-dependent weighting learns when to trust the noisy intermediate image as an anchor; that principle could generalize to any diffusion conditioning task where the condition and the target come from different domains.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SHMT, a self-supervised makeup transfer method built on latent diffusion models. Following a decoupling-and-reconstruction paradigm, it extracts a content representation (3D face shape plus hierarchical texture details from a Laplacian pyramid) and a makeup representation (a spatially distorted foreground), then reconstructs the original image during training. At inference, the makeup representation comes from the reference image, and an Iterative Dual Alignment (IDA) module refines the injected condition at each denoising step. Experiments on MT, Wild-MT, and LADN report favorable FID, CLS, and Key-sim values against seven baselines, with ablations for the texture level and IDA. The code is publicly available.

Significance. If the central claims hold, the self-supervised paradigm is a relevant alternative to pseudo-paired training data for makeup transfer, and the hierarchical texture control is a useful capability that prior GAN-based methods lack. The paper provides a reasonable amount of ablative evidence (e.g., Table 3 shows IDA improves FID from 32.42 to 27.01 on LADN) and releases code, which supports reproducibility. However, the significance is tempered by the fact that the reported quantitative advantages depend on a non-standard FID usage and the absence of identity-controlled metrics, as detailed below.

major comments (4)
  1. [Sec. 4.2, Table 1] The central claim that SHMT-h4 achieves the highest FID and CLS may be confounded by reference-identity leakage. FID is computed between transferred results and reference images, and CLS is the cosine similarity of DINO CLS tokens between the same pairs. Since SHMT-h4 intentionally transfers high-frequency texture details from the reference foreground (Sec. 3.3 and Appendix A), the improved FID/CLS could reflect copying the reference face's identity, geometry, and skin texture rather than makeup fidelity per se. No identity-similarity metric between source and result (e.g., face-recognition cosine distance) is reported for any method, so the paper does not currently distinguish high-fidelity makeup transfer from a face-appearance-swap effect.
  2. [Sec. 3.3, 'Makeup Representation'] The load-bearing premise that random cropping, rotation, and elastic distortion of the foreground 'destroy content information' while 'retaining makeup information' is asserted but not demonstrated. No quantitative analysis shows that Im loses identity/geometry but preserves makeup attributes. During self-supervised training, Im is a distorted version of the same image the model must reconstruct, so the optimization does not force the network to separate makeup from skin albedo, illumination, pores, wrinkles, or identity-bearing facial texture. Without a direct evaluation of Im's content and makeup retention (e.g., face-recognition scores on Im and makeup-attribute classification), the decoupling claim is not established.
  3. [Sec. 4.2, Evaluation Metrics] The use of FID 'between reference images and transferred results' is non-standard and potentially misleading. FID is defined for two distributions; in this paper the reference set and the generated set are paired, and a low FID rewards any resemblance to the reference distribution, including identity and background transfer. The paper should justify this choice, and ideally supplement it with makeup-specific metrics (e.g., per-component makeup color statistics or a makeup-attribute classifier) to isolate makeup fidelity from global appearance matching.
  4. [Sec. 4.2, Table 1] No error bars, confidence intervals, or significance tests are reported for any metric. The metrics are computed on a random selection of 1000 source-reference pairs, and FID estimates are known to have nontrivial sampling variance. For instance, the CLS advantage of SHMT-h4 over Stable-Makeup on MT is 0.715 vs 0.682, but without variance estimates it is unclear whether such differences are stable across random subsets. Adding repeated evaluation with different seeds or bootstrapped intervals would strengthen the quantitative claims.
minor comments (5)
  1. [Sec. 4.1, Baselines] The baseline list says 'PSAGN [17]' but the cited work is PSGAN; please correct the typo.
  2. [Sec. 3.4, Eq. (2)] The notation fc(i) and fm(j) is not fully defined; please specify that i and j index spatial positions in the content and makeup feature maps, respectively.
  3. [Sec. 4.3, Figure 8(c)] The plot of w versus timestep t lacks axis labels and a legend for the two curves; adding these would improve readability.
  4. [Sec. 3.3, Implementation Details] The statement that the autoencoder with downsampling factor 4 'preserves texture details better' is not backed by a comparison; either provide a supporting ablation or soften the claim.
  5. [Appendix D, Limitations] The limitation regarding face parsing mislabeling forehead makeup as hair is clearly illustrated, but a short sentence on possible mitigations (e.g., retraining the parser on makeup images) would make the limitation more constructive.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the SOTA claim rests on a self-supervised reconstruction loss and external benchmarks, not on fitted predictions or self-citation.

full rationale

The derivation chain is self-contained. Training optimizes the standard LDM loss (Eq. 1) to reconstruct the input from its own content representation (I3d, hi), background Ibg, and makeup representation Im obtained by random spatial distortion of the same foreground. This is a conventional self-supervised reconstruction objective: the network is trained to invert its own conditioning, not to match the test metrics. At inference, Im is simply replaced by the reference image; no test-set values enter training, and hyperparameters (τ=100, pyramid level h4, DDIM steps) are fixed choices, not fitted to FID/CLS. The only author self-citations (SSAT [36], SSAT++ [37]) appear in the related-work discussion and in the baseline list; they are not used to justify any premise of SHMT. The CLS/FID gains of SHMT-h4 are reported as empirical measurements from Table 1, and the trade-off with Key-sim is explicitly acknowledged (Sec. 4.2, Appendix A). While the assumption that spatial distortion retains makeup but suppresses content is an unproved premise (and the paper's own limitations show face-parsing failure modes), an unsupported premise is not circularity: no equation defines the desired FID/CLS result into the loss or the self-citation chain.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method is built on established pretrained models and existing building blocks; it does not introduce new physical or learned entities beyond trained network weights. The main assumptions are the reliability of the pretrained components and the validity of the self-supervised decoupling.

free parameters (3)
  • Softmax temperature tau = 100
    Set by hand in Eq. 3; controls the sharpness of the spatial attention matrix. No sensitivity analysis is reported.
  • Laplacian pyramid level L = 4 (models for h0 to h4)
    Number of decomposition levels selected by hand. The reported variants SHMT-h0 and SHMT-h4 correspond to different texture-detail injections, and the paper uses coarse levels for complex styles.
  • Autoencoder downsampling factor = 4
    Chosen empirically because it 'preserves texture details better' than factor 8; no quantitative comparison is given.
assumptions (4)
  • domain assumption The pretrained LDM autoencoder E,D provides a faithful latent space for 256x256 face reconstruction and transfer.
    Sec. 3.1 relies on E and D from Rombach et al. [31] without fine-tuning them.
  • domain assumption BiSeNet face parsing and 3DDFA-V2 produce sufficiently accurate masks and face shapes for the content representation.
    Sec. 3.3 uses them to extract Ibg and I3d; Appendix D shows a failure when parsing labels forehead makeup as hair.
  • ad hoc to paper Random spatial transformations of the foreground remove identity and geometry information while preserving makeup information.
    Sec. 3.3 'Makeup Representation.' This assumption underpins the self-supervised decoupling, but no independent validation is provided.
  • domain assumption DINO CLS and Key-sim similarities are valid proxies for makeup fidelity and content preservation.
    Sec. 4.1 adopts these metrics from [39] without calibrating them against human perceptual judgments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SHMT: Self-supervised Hierarchical Makeup Transfer via Latent Diffusion Models." pith.science (2026). https://pith.science/paper/TH3VYT5Q

@misc{pith2026241211058,
  author       = {Pith},
  title        = {Pith review of: SHMT: Self-supervised Hierarchical Makeup Transfer via Latent Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TH3VYT5Q}},
  note         = {Machine review of arXiv:2412.11058}
}
read the original abstract

This paper studies the challenging task of makeup transfer, which aims to apply diverse makeup styles precisely and naturally to a given facial image. Due to the absence of paired data, current methods typically synthesize sub-optimal pseudo ground truths to guide the model training, resulting in low makeup fidelity. Additionally, different makeup styles generally have varying effects on the person face, but existing methods struggle to deal with this diversity. To address these issues, we propose a novel Self-supervised Hierarchical Makeup Transfer (SHMT) method via latent diffusion models. Following a "decoupling-and-reconstruction" paradigm, SHMT works in a self-supervised manner, freeing itself from the misguidance of imprecise pseudo-paired data. Furthermore, to accommodate a variety of makeup styles, hierarchical texture details are decomposed via a Laplacian pyramid and selectively introduced to the content representation. Finally, we design a novel Iterative Dual Alignment (IDA) module that dynamically adjusts the injection condition of the diffusion model, allowing the alignment errors caused by the domain gap between content and makeup representations to be corrected. Extensive quantitative and qualitative analyses demonstrate the effectiveness of our method. Our code is available at \url{https://github.com/Snowfallingplum/SHMT}.

Figures

Figures reproduced from arXiv: 2412.11058 by the authors.

Figure 1
Figure 1. Illustration of two main difficulties in the makeup transfer task. (a) Due to the absence [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. In addition to color matching, our approach allows flexible control to preserve or discard [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The framework of SHMT. A facial image I is decomposed into background area Ibg, makeup representation Im, and content representation (I3d, hi). The makeup transfer procedure is simulated by reconstructing the original image from these components. Hierarchica texture details hi are constructed to respond to different makeup styles. In each denoising step t, IDA draws on the noisy intermediate result ˆIt to dynamicall… view at source ↗
Figures from the paper (15 more)
Figure 3
Figure 3. Figure 3: If the resolution of the texture details exceeds that of the latent space, we downsample it [PITH_FULL_IMAGE:figures/full_fig_p005_3.png]
Figure 4
Figure 4. Figure 4: Qualitative comparison with GAN-based baselines on simple makeup styles. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparison with GAN-based baselines on complex makeup styles. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Qualitative comparison with the Stable-Makeup baseline on simple makeup styles. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Qualitative comparison with the Stable-Makeup baseline on complex makeup styles. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Ablation studies of each proposed module to validate its effectiveness. Zoomed-in view for [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: The robustness and generalization ability of the model SHMT- [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: Qualitative results of models equipped with different texture details under complex makeup [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Qualitative comparison of our method SHMT with the style transfer method InstantStyle. [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: The illustration of global makeup interpolation. The first row is the result of a single [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: The illustration of local makeup interpolation. The first row is lipstick control, the second [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]
Figure 14
Figure 14. Figure 14: By default, our method transfers makeup to change the skin tone. Optionally, the local [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]
Figure 15
Figure 15. Figure 15: More qualitative results of different methods in simple makeup styles. [PITH_FULL_IMAGE:figures/full_fig_p019_15.png]
Figure 16
Figure 16. Figure 16: More qualitative results of different methods in complex makeup styles. [PITH_FULL_IMAGE:figures/full_fig_p020_16.png]
Figure 17
Figure 17. Figure 17: Limitations of our approach. The face parsing model often marks high-frequency [PITH_FULL_IMAGE:figures/full_fig_p021_17.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 32 canonical work pages

  1. [1]

    Interpretable-through-prototypes deep- fake detection for diffusion models

    Agil Aghasanli, Dmitry Kangin, and Plamen Angelov. Interpretable-through-prototypes deep- fake detection for diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 467–474, 2023

    work page 2023

  2. [2]

    The laplacian pyramid as a compact image code

    Peter J Burt and Edward H Adelson. The laplacian pyramid as a compact image code. In Readings in computer vision, pages 671–679. Elsevier, 1987

    work page 1987

  3. [3]

    Pairedcyclegan: Asymmetric style transfer for applying and removing makeup

    Huiwen Chang, Jingwan Lu, Fisher Yu, and Adam Finkelstein. Pairedcyclegan: Asymmetric style transfer for applying and removing makeup. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 40–48, 2018

    work page 2018

  4. [4]

    Beautyglow: On-demand makeup transfer framework with reversible generative network

    Hung-Jen Chen, Ka-Ming Hui, Szu-Yu Wang, Li-Wu Tsao, Hong-Han Shuai, and Wen-Huang Cheng. Beautyglow: On-demand makeup transfer framework with reversible generative network. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pages 10042–10050, 2019

    work page 2019

  5. [5]

    Anydoor: Zero-shot object-level image customization

    Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao. Anydoor: Zero-shot object-level image customization. arXiv preprint arXiv:2307.09481, 2023

    arXiv 2023

  6. [6]

    On the detection of synthetic images generated by diffusion models

    Riccardo Corvi, Davide Cozzolino, Giada Zingarini, Giovanni Poggi, Koki Nagano, and Luisa Verdoliva. On the detection of synthetic images generated by diffusion models. InICASSP 2023- 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2023

    2023

  7. [7]

    Spatially-invariant style- codes controlled makeup transfer

    Han Deng, Chu Han, Hongmin Cai, Guoqiang Han, and Shengfeng He. Spatially-invariant style- codes controlled makeup transfer. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pages 6549–6557, 2021

    work page 2021

  8. [8]

    Joint 3d face reconstruction and dense alignment with position map regression network

    Yao Feng, Fan Wu, Xiaohu Shao, Yanfeng Wang, and Xi Zhou. Joint 3d face reconstruction and dense alignment with position map regression network. In Proceedings of the European conference on computer vision (ECCV), pages 534–551, 2018

    work page 2018

Show all 58 references
  1. [9]

    An image is worth one word: Personalizing text-to-image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:2208.01618, 2022

    2022 arXiv

  2. [10]

    Generative adversarial nets

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014

    2014

  3. [11]

    Ladn: Local adversarial disentangling network for facial makeup and de-makeup

    Qiao Gu, Guanzhi Wang, Mang Tik Chiu, Yu-Wing Tai, and Chi-Keung Tang. Ladn: Local adversarial disentangling network for facial makeup and de-makeup. In Proceedings of the IEEE/CVF International conference on computer vision, pages 10481–10490, 2019

    2019

  4. [12]

    Digital face makeup by example

    Dong Guo and Terence Sim. Digital face makeup by example. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 73–79. IEEE, 2009

    2009

  5. [13]

    Towards fast, accurate and stable 3d dense face alignment

    Jianzhu Guo, Xiangyu Zhu, Yang Yang, Fan Yang, Zhen Lei, and Stan Z Li. Towards fast, accurate and stable 3d dense face alignment. In European Conference on Computer Vision, pages 152–168. Springer, 2020

    2020

  6. [14]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017. 11

    2017

  7. [15]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  8. [16]

    Real-world automatic makeup via identity preservation makeup net

    Zhikun Huang, Zhedong Zheng, Chenggang Yan, Hongtao Xie, Yaoqi Sun, Jianzhong Wang, and Jiyong Zhang. Real-world automatic makeup via identity preservation makeup net. In International Joint Conference on Artificial Intelligence . International Joint Conference on Artificial I...

    2021

  9. [17]

    Psgan: Pose and expression robust spatial-aware gan for customizable makeup transfer

    Wentao Jiang, Si Liu, Chen Gao, Jie Cao, Ran He, Jiashi Feng, and Shuicheng Yan. Psgan: Pose and expression robust spatial-aware gan for customizable makeup transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5194–5202, 2020

    2020

  10. [18]

    Glow: Generative flow with invertible 1x1 convolutions

    Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. Advances in neural information processing systems, 31, 2018

    2018

  11. [19]

    Simulating makeup through physics-based manipulation of intrinsic image layers

    Chen Li, Kun Zhou, and Stephen Lin. Simulating makeup through physics-based manipulation of intrinsic image layers. In Proceedings of the IEEE Conference on computer vision and pattern recognition, pages 4621–4629, 2015

    2015

  12. [20]

    Learning a model of facial shape and expression from 4d scans

    Tianye Li, Timo Bolkart, Michael J Black, Hao Li, and Javier Romero. Learning a model of facial shape and expression from 4d scans. ACM Trans. Graph., 36(6):194–1, 2017

    2017

  13. [21]

    Beautygan: Instance-level facial makeup transfer with deep generative adversarial network

    Tingting Li, Ruihe Qian, Chao Dong, Si Liu, Qiong Yan, Wenwu Zhu, and Liang Lin. Beautygan: Instance-level facial makeup transfer with deep generative adversarial network. In Proceedings of the 26th ACM international conference on Multimedia, pages 645–653, 2018

    2018

  14. [22]

    Caphuman: Capture your moments in parallel universes

    Chao Liang, Fan Ma, Linchao Zhu, Yingying Deng, and Yi Yang. Caphuman: Capture your moments in parallel universes. arXiv preprint arXiv:2402.00627, 2024

    2024 arXiv

  15. [23]

    Sdxl-lightning: Progressive adversarial diffusion distillation

    Shanchuan Lin, Anran Wang, and Xiao Yang. Sdxl-lightning: Progressive adversarial diffusion distillation. arXiv preprint arXiv:2402.13929, 2024

    2024 arXiv

  16. [24]

    Psgan++: Robust detail-preserving makeup transfer and removal

    Si Liu, Wentao Jiang, Chen Gao, Ran He, Jiashi Feng, Bo Li, and Shuicheng Yan. Psgan++: Robust detail-preserving makeup transfer and removal. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11):8538–8551, 2021

    2021

  17. [25]

    Dpm- solver++: Fast solver for guided sampling of diffusion probabilistic models

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm- solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint arXiv:2211.01095, 2022

    2022 arXiv

  18. [26]

    Sogan: 3d-aware shadow and occlusion robust gan for makeup transfer

    Yueming Lyu, Jing Dong, Bo Peng, Wei Wang, and Tieniu Tan. Sogan: 3d-aware shadow and occlusion robust gan for makeup transfer. In Proceedings of the 29th ACM International conference on multimedia, pages 3601–3609, 2021

    2021

  19. [27]

    Least squares generative adversarial networks

    Xudong Mao, Qing Li, Haoran Xie, Raymond YK Lau, Zhen Wang, and Stephen Paul Smolley. Least squares generative adversarial networks. In Proceedings of the IEEE international conference on computer vision, pages 2794–2802, 2017

    2017

  20. [28]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 4296–4304, 2024

    2024

  21. [29]

    Lipstick ain’t enough: beyond color matching for in-the-wild makeup transfer

    Thao Nguyen, Anh Tuan Tran, and Minh Hoai. Lipstick ain’t enough: beyond color matching for in-the-wild makeup transfer. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pages 13305–13314, 2021

    2021

  22. [30]

    Hierarchical text-conditional image generation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3, 2022

    2022 arXiv

  23. [31]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  24. [32]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22500...

    2023

  25. [33]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

    2022

  26. [34]

    Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network

    Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew P Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE conference on compute...

    2016

  27. [35]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020

    2010 arXiv

  28. [36]

    Ssat: A symmetric semantic-aware transformer network for makeup transfer and removal

    Zhaoyang Sun, Yaxiong Chen, and Shengwu Xiong. Ssat: A symmetric semantic-aware transformer network for makeup transfer and removal. In Proceedings of the AAAI Conference on artificial intelligence, pages 2325–2334, 2022

    2022

  29. [37]

    Ssat++: A semantic-aware and versatile makeup transfer network with local color consistency constraint

    Zhaoyang Sun, Yaxiong Chen, and Shengwu Xiong. Ssat++: A semantic-aware and versatile makeup transfer network with local color consistency constraint. IEEE Transactions on Neural Networks and Learning Systems, 2023

    2023

  30. [38]

    Example-based cosmetic transfer

    Wai-Shun Tong, Chi-Keung Tang, Michael S Brown, and Ying-Qing Xu. Example-based cosmetic transfer. In 15th Pacific Conference on Computer Graphics and Applications (PG’07), pages 211–218. IEEE, 2007

    2007

  31. [39]

    Splicing vit features for semantic appearance transfer

    Narek Tumanyan, Omer Bar-Tal, Shai Bagon, and Tali Dekel. Splicing vit features for semantic appearance transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10748–10757, 2022

    2022

  32. [40]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  33. [41]

    Facial attribute transformers for precise and robust makeup transfer

    Zhaoyi Wan, Haoran Chen, Jie An, Wentao Jiang, Cong Yao, and Jiebo Luo. Facial attribute transformers for precise and robust makeup transfer. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 1717–1726, 2022

    2022

  34. [42]

    Instantstyle: Free lunch towards style-preserving in text-to-image generation

    Haofan Wang, Qixun Wang, Xu Bai, Zekui Qin, and Anthony Chen. Instantstyle: Free lunch towards style-preserving in text-to-image generation. arXiv preprint arXiv:2404.02733, 2024

    2024 arXiv

  35. [43]

    Stableidentity: Inserting anybody into anywhere at first sight

    Qinghe Wang, Xu Jia, Xiaomin Li, Taiqing Li, Liqian Ma, Yunzhi Zhuge, and Huchuan Lu. Stableidentity: Inserting anybody into anywhere at first sight. arXiv preprint arXiv:2401.15975, 2024

    2024 arXiv

  36. [44]

    Instantid: Zero-shot identity-preserving generation in seconds

    Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, and Anthony Chen. Instantid: Zero-shot identity-preserving generation in seconds. arXiv preprint arXiv:2401.07519, 2024

    2024 arXiv

  37. [45]

    Ramgan: region attentive morphing gan for region-level makeup transfer

    Jianfeng Xiang, Junliang Chen, Wenshuang Liu, Xianxu Hou, and Linlin Shen. Ramgan: region attentive morphing gan for region-level makeup transfer. In European Conference on Computer Vision, pages 719–735. Springer, 2022

    2022

  38. [46]

    Beautyrec: Robust, efficient, and content-preserving makeup transfer

    Qixin Yan, Chunle Guo, Jixin Zhao, Yuekun Dai, Chen Change Loy, and Chongyi Li. Beautyrec: Robust, efficient, and content-preserving makeup transfer. arXiv preprint arXiv:2212.05855, 2022

    2022 arXiv

  39. [47]

    Elegant: Exquisite and locally editable gan for makeup transfer

    Chenyu Yang, Wanrong He, Yingqing Xu, and Yang Gao. Elegant: Exquisite and locally editable gan for makeup transfer. In European Conference on Computer Vision, pages 737–754. Springer, 2022

    2022

  40. [48]

    Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.06721, 2023

    2023 arXiv

  41. [49]

    Bisenet: Bilateral segmentation network for real-time semantic segmentation

    Changqian Yu, Jingbo Wang, Chao Peng, Changxin Gao, Gang Yu, and Nong Sang. Bisenet: Bilateral segmentation network for real-time semantic segmentation. In Proceedings of the European conference on computer vision (ECCV), pages 325–341, 2018

    2018

  42. [50]

    Dino: Detr with improved denoising anchor boxes for end-to-end object detection

    Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung- Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605, 2022. 13

    2022 arXiv

  43. [51]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023

    2023

  44. [52]

    Stable-makeup: When real-world makeup transfer meets diffusion model

    Yuxuan Zhang, Lifu Wei, Qing Zhang, Yiren Song, Jiaming Liu, Huaxia Li, Xu Tang, Yao Hu, and Haibo Zhao. Stable-makeup: When real-world makeup transfer meets diffusion model. arXiv preprint arXiv:2403.07764, 2024

    2024 arXiv

  45. [53]

    Fast ode-based sampling for diffusion models in around 5 steps

    Zhenyu Zhou, Defang Chen, Can Wang, and Chun Chen. Fast ode-based sampling for diffusion models in around 5 steps. arXiv preprint arXiv:2312.00094, 2023

    2023 arXiv

  46. [54]

    a woman, best quality, high quality

    Mingrui Zhu, Yun Yi, Nannan Wang, Xiaoyu Wang, and Xinbo Gao. Semi-parametric makeup transfer via semantic-aware correspondence. arXiv preprint arXiv:2203.02286, 2022. 14 Appendix A The Effectiveness of Hierarchical Texture Details As the texture details go from fine to coarse...

    2022 arXiv

  47. [55]

    We will encrypt or anonymize facial images during transmission and storage, such as using hash values instead of real image data

  48. [56]

    We will use the Stable diffusion safety checker 3 to conduct security checks on our generated images, so that we can identify and handle Not Safe For Work (NSFW) contents in images

  49. [57]

    Since our method is working on human faces, we will also employ some deep-fake detection models [1, 6] to filter the results generated by our model

  50. [58]

    ReferenceSource Figure 15: More qualitative results of different methods in simple makeup styles. 19 PSGANSCGANEleGANtSSATLADNCPMSHMT-!

    We will ask the users to agree to a license or conform a code of ethics before accessing our model, which requires them to use our model in a more standardized manner. E.3 Responsibility to Face Images The face images in this study are taken from publicly accessible datasets, ...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.