Pith. sign in

REVIEW 3 major objections 6 minor 56 references

ControlFace: Harnessing Facial Parametric Control for Face Rigging

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read ControlFace aims to show that face rigging can be zero-shot: one reference photo plus 3DMM target renderings yields a rigged face with no per-identity fine-tuning.

desk verdict A credible zero-shot face-rigging system with real component-level ablations, but the headline superiority claim rests on fragile per-category metrics and an unverified paired-frame training assumption. read the letter →

arxiv 2412.01160 v4 pith:E6FESTOT submitted 2024-12-02 cs.CV

Wooseok Jang , Youngjun Hong , Geonho Cha , Seungryong Kim This is my paper · ORCID
classification cs.CV
keywords facerigging3Dmorphablemodelslatentdiffusionidentitypreservationparametriccontrolzero-shotgeneralizationreferenceguidancevideo-pairtraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ControlFace tries to make face rigging a single-pass operation: given one photo of a person, a user specifies a new pose, expression, shape, or lighting through 3D morphable model (3DMM) renderings, and the model generates the edited face without any per-identity fine-tuning. The paper argues that prior diffusion riggers fail because they are trained to reconstruct the reference image itself, giving the identity encoder so much information that it can ignore the control. ControlFace instead trains on pairs of frames from the same video clip, so the target frame's renderings are the only instruction the model needs. The reported numbers are an average DECA re-inference error of 4.85, the best in its comparison table, and an LPIPS of 0.1429, indicating that identity and semantic details such as hairstyle survive the edit. If the method works as claimed, animators and casual users could rig faces from a single photo with no extra training data.

What carries the argument

The machinery is three additions to a latent diffusion U-Net. FaceNet is a second U-Net with the same architecture that reads the reference image and contributes keys and values to the denoising U-Net's self-attention, preserving fine detail without compressing identity into a single vector. The control mixer module (CMM) runs the target control $D_T$ and the reference control $D_R$ through shared-weight encoders with cross-attention and fuses the result into the augmented self-attention, telling the model where to attend under the requested change. Reference control guidance (RCG) extrapolates the noise estimate as $\hat{\epsilon}_\theta(\cdot, D_T) = \epsilon_\theta(\cdot, D_R) + w(\epsilon_\theta(\cdot, D_T) - \epsilon_\theta(\cdot, D_R))$, using $D_R$ as a grounded null condition instead of an empty label. The load-bearing training choice is random two-frame pairing from CelebV-HQ videos, giving quadruplets $\{X_R, X_T, D_R, D_T\}$ that avoid the trivial reconstruction objective.

What would settle it

Set the target control equal to the reference control ($D_T = D_R$) for held-out FFHQ faces and measure the LPIPS between the generated image and the reference. A model that truly follows the control should return the reference almost unchanged; large drift would show that the video-pair training has coupled incidental frame differences to output changes. A second check is to build test pairs where $D_T = D_R$ but the target frame comes from a different moment of the same clip with different hair or blink state and see whether the model rewrites appearance despite identical controls.

Watch

Extended reading notes

Core claim

The central claim is that reconstruction-based training is the reason prior rigging models can ignore the control: when the reference image is also the training target, copying the reference is a perfect solution. ControlFace replaces this with paired-video training, randomly selecting two frames from the same CelebV-HQ clip so that the target's DECA renderings define the only meaningful change. A dual-branch U-Net injects reference detail through augmented self-attention, a control mixer module correlates target and reference renderings, and reference control guidance extrapolates toward the target condition at inference. On FFHQ evaluation images the paper reports an average DECA re-inference error of 4.85, better than the 5.06 of the DiffusionRig baseline, and an LPIPS of 0.1429 against DiffusionRig's 0.3758, while preserving an identity cosine similarity of 0.7586. ControlFace is evaluated zero-shot on FFHQ identities the model never trained on and also on out-of-domain animation faces.

Load-bearing premise

The load-bearing premise is that two randomly chosen frames from the same video clip form a valid training pair in which the target control renderings are the only meaningful change; if hair motion, blinks, occlusions, or exposure drift are common in those pairs, the model can learn to reproduce spurious frame differences instead of the intended control.

Editorial extensions

If this is right

  • A single reference photo is enough: pose, expression, lighting, and shape edits work at inference time with no per-identity fine-tuning, and the paper reports better average control adherence than the fine-tuning-based DiffusionRig baseline.
  • Fine details survive large edits: the reported LPIPS of 0.1429 versus 0.3758 for DiffusionRig means hairstyle, background, and facial detail are preserved rather than regenerated from a compact identity vector.
  • Lighting becomes a strongly controllable attribute: the DECA re-inference error for light is 3.75 against 6.31 for DiffusionRig, so the control signal is not being averaged away.
  • Reference control guidance costs the same as classifier-free guidance at inference (two forward passes) and does not require dropping out conditions during training, making it a drop-in boost for control adherence.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own limitation section concedes dependence on DECA's 3DMM accuracy and training on a single video dataset, so the zero-shot guarantee is conditional on the reference and target being faces DECA reconstructs well.
  • Because the paired-frames recipe is what makes training work, the strongest test would deliberately insert nuisance differences such as hair motion, blinks, or exposure shifts into training pairs and measure whether control adherence or identity preservation moves.
  • The RCG recipe, anchoring guidance to the source condition rather than an empty condition, should transfer to other conditional diffusion tasks with paired source and target controls, such as video re-animation or relighting, though the paper does not test this.
  • A natural next experiment is temporal rigging: run the model on every frame of a video with per-frame target renderings and measure flicker, since the paper demonstrates single-frame rigging only.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. ControlFace proposes a zero-shot face-rigging method that manipulates pose, expression, shape, and lighting of a single reference photo using DECA 3DMM renderings as controls. The architecture couples a FaceNet branch that encodes the reference image with a denoising U-Net, injects correlated reference/target control embeddings via a Control Mixer Module, and uses Reference Control Guidance at inference. Training uses paired frames randomly sampled from the same CelebV-HQ video clip, avoiding reconstruction-style training. The paper reports better average DECA re-inference error than prior methods, strong LPIPS/FID, and favorable user-study scores on FFHQ, plus ablations supporting each component.

Significance. If the claims hold, ControlFace is practically significant: it offers rigging without per-identity fine-tuning, and the dual-branch design with CMM and RCG is a plausible and useful contribution. The paper is also commendable for evaluating on unseen FFHQ identities, providing out-of-domain tests, and including ablations of the conditioning modules and guidance mechanisms. However, the central quantitative claim of superior control precision is currently supported by a partly circular metric, an unverified training-pair assumption, and a per-attribute comparison that does not uniformly favor the method. These issues are load-bearing because the paper's headline conclusions depend on the DECA re-inference average and on the implicit paired-frame assumption.

major comments (3)
  1. [Section 5.2, Table 1] The primary control-adherence metric is circular: the controls DT and DR are DECA renderings, and the metric is the DECA re-inference error between the output and the target. A model that learns to invert DECA's image-to-parameter mapping can score well without visibly following the control. This concern is partially mitigated by the user study, but the quantitative headline remains dependent on this closed loop. Please add an independent re-inference with a different 3DMM (e.g., EMOCA) or a non-parametric alignment metric, and report the correlation between the DECA re-inference error and human semantic-consistency scores.
  2. [Section 4.2 and Eq. (5)] Training quadruplets are built by randomly selecting two frames from the same CelebV-HQ clip, implicitly assuming that the DECA controls are the only semantically meaningful change between XR and XT. In real video, frame pairs also differ in blink state, hair motion, occlusions, motion blur, and exposure; because Eq. (5) supervises denoising of the target latent given the reference and the two renderings, the model can be rewarded for reproducing target-frame nuisances that are unavailable at inference, or for learning spurious correlations between control values and incidental appearance changes. This is the core assumption behind the zero-shot rigging claim, yet the paper reports no measurement of nuisance disagreement over the sampled pairs and no ablation that filters or controls for it. Please quantify how often large nuisance differences occur, add a training-pair filtering or reweighting strategy, and evaluate on held-out video identities with ground-truth target frames.
  3. [Table 1] The headline 'superior control precision' is not uniformly supported by the per-attribute results: ControlFace is worse than DiffusionRig on shape (2.56 vs 2.11) and pose (7.67 vs 6.26), and the lower average (4.85 vs 5.06) is driven mainly by the lighting column. Moreover, the lighting error is computed directly on spherical-harmonics coefficients while the other columns are mesh RMSE values, so averaging heterogeneous units is questionable. Please report per-attribute statistical significance, use a normalized or unit-consistent average, and temper the claims accordingly.
minor comments (6)
  1. [Section 4.5, Eq. (5)] The notation z_{T,t} appears to be a typo: the surrounding text says the noise is used to perturb z_R, so the argument should presumably be z_{R,t}; please correct this.
  2. [Section 5.4, last paragraph] In the sentence 'We additionally provide ablation study on different inputs and architecture for CCM', 'CCM' should read 'CMM'.
  3. [Supplementary Appendix C.2, Table 7] The text says 'Although FaceNet achieves lower FID', but Table 7 reports FaceNet FID of 32.45 versus CLIP's 29.77, i.e., FaceNet has higher FID; please fix the wording or explain the discrepancy.
  4. [Tables 1 and 3] The Arc2Face citations in the table headers are listed as [5] rather than [29]; please correct the reference keys.
  5. [Supplementary Appendix B] The user study uses only eight participants split into two groups, with 52 generated images per model, and no confidence intervals or inter-annotator agreement are reported; please add these details or temper the strength of the user-study conclusions.
  6. [Table 4 caption] The caption says 'We plot parameter counts', but the table reports numbers rather than a plot; please change 'plot' to 'report'.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; the derivation is self-contained, with only a minor self-referential DECA evaluation metric.

full rationale

ControlFace's pipeline is an empirical method rather than a derivation chain whose output is equivalent to its input. The training objective (Eq. 5) is a standard denoising loss on paired quadruplets sampled from CelebV-HQ, and the architectural components (FaceNet injection via Aug-Attn, CMM, face controller, RCG in Eq. 7) are introduced as design choices and ablated rather than fitted quantities later renamed as predictions. The control-adherence metric is a DECA re-inference error, and because the conditioning renderings DR and DT are themselves produced by DECA, the metric is partly self-referential: it rewards models that can invert DECA's own rendering-to-parameters mapping. However, this metric is applied identically to all baselines, so the relative ranking is meaningful, and the paper supplements it with independent evidence: a user study (Table 3), LPIPS, FID, identity similarity, and qualitative comparisons. Thus the central claim does not reduce to the DECA loop by construction. The paired-frame assumption in Sec. 4.2 (randomly selecting two frames from the same clip as XR and XT) is an external-validity concern about nuisance differences such as blinks, hair motion, or exposure, not a circularity. No load-bearing self-citation was found: the augmented self-attention technique is attributed to multiple prior works, including ones with no author overlap with this paper, and no uniqueness theorem or ansatz is imported from the authors' own prior work. The stated limitation about relying on DECA is acknowledged by the authors and does not constitute circular reasoning. Score 1 reflects the minor self-referential evaluation metric while recognizing that the method's predictions are otherwise independently supported.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

No new physical or theoretical entities are introduced. FaceNet, CMM, and RCG are engineered network components whose behavior is evidenced only through experiments, not independent falsifiable handles outside the paper.

free parameters (1)
  • Reference Control Guidance scale w = 4
    Selected based on ablation over w in 1.0 to 4.0 (Sec. 5.4, Fig. 8) to minimize DECA re-inference error. The central quantitative results use w=4, so the reported gains depend on this hand-tuned inference hyperparameter.
assumptions (4)
  • domain assumption DECA-estimated 3DMM renderings (normals, albedo, Lambertian) are a sufficient pixel-aligned control signal for pose, shape, expression, and lighting rigging.
    Invoked in Sec. 4.1 and 4.2; if DECA renderings omit hair or background or are inaccurate, the control signal cannot fully specify the edit. The paper lists DECA's limited fidelity as a limitation.
  • domain assumption Random frame pairs from the same CelebV-HQ video are valid training targets, with the target control DT as the only meaningful difference from the reference.
    Sec. 4.2 relies on this for paired training; no analysis of nuisance variations such as blink, hair motion, or occlusions is provided.
  • domain assumption DECA re-inference error is a valid measure of control adherence.
    Used in Tables 1, 4, 5 and Fig. 8 following prior work. Since the control and the metric share the same DECA estimator, this assumption creates evaluation circularity.
  • domain assumption Pretrained Stable Diffusion v1.5 weights and CLIP image embeddings provide a good initialization for face rigging.
    Sec. 5.1 initializes FaceNet and the denoising U-Net from an SD1.5 variant; the paper provides no systematic study of this choice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ControlFace: Harnessing Facial Parametric Control for Face Rigging." pith.science (2026). https://pith.science/paper/E6FESTOT

@misc{pith2026241201160,
  author       = {Pith},
  title        = {Pith review of: ControlFace: Harnessing Facial Parametric Control for Face Rigging},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E6FESTOT}},
  note         = {Machine review of arXiv:2412.01160}
}
read the original abstract

Manipulation of facial images to meet specific controls such as pose, expression, and lighting, also known as face rigging, is a complex task in computer vision. Existing methods are limited by their reliance on image datasets, which necessitates individual-specific fine-tuning and limits their ability to retain fine-grained identity and semantic details, reducing practical usability. To overcome these limitations, we introduce ControlFace, a novel face rigging method conditioned on 3DMM renderings that enables flexible, high-fidelity control. We employ a dual-branch U-Nets: one, referred to as FaceNet, captures identity and fine details, while the other focuses on generation. To enhance control precision, the control mixer module encodes the correlated features between the target-aligned control and reference-aligned control, and a novel guidance method, reference control guidance, steers the generation process for better control adherence. By training on a facial video dataset, we fully utilize FaceNet's rich representations while ensuring control adherence. Extensive experiments demonstrate ControlFace's superior performance in identity preservation and control precision, highlighting its practicality. Please see the project website: https://cvlab-kaist.github.io/ControlFace/.

Figures

Figures reproduced from arXiv: 2412.01160 by the authors.

Figure 1
Figure 1. Our ControlFace can edit the input face image using explicit facial parametric controls, generating realistic images without compromising the identity and other semantic details such as hairstyle. Abstract Manipulation of facial images to meet specific controls such as pose, expression, and lighting, also known as face rigging, is a complex task in computer vision. Existing methods are limited by their reliance on i… view at source ↗
Figure 2
Figure 2. Limitations of reconstruction-based training. We compare the results of our model trained on an image dataset [21] in a reconstruction setup and on a video dataset [54] with paired samples created by randomly selecting two frames from each video. The results by reconstruction-based training often ignores the target control at inference. mance in identity preservation, semantic consistency, and control precision comp… view at source ↗
Figure 3
Figure 3. Overall Architecture. ControlFace encodes the reference image XR into the FaceNet and CLIP image encoder for identity and semantic preservation. For face control, the target control DT is incorporated into the denoising U-Net through face controller. To enhance the control adherence, the correlated feature between reference control DR and target control DT is acquired from the proposed control mixer module. is encod… view at source ↗
Figures from the paper (14 more)
Figure 4
Figure 4. Figure 4: Visualization of Reference Control Guidance. We vi￾sualize the deltas, ϵθ(·, DT )−ϵθ(·, ∅) and ϵθ(·, DT )−ϵθ(·, DR), which corresponds to CFG [17] applied to face controller input and RCG, respectively, across different timesteps t. The first and third row display RCG …
Figure 5
Figure 5. Figure 5: Qualitative Results. We compare the results of rigging pose, expression, and light with four different baselines [8, 11, 26, 29]. The reference images are from the FFHQ [21] dataset. Compared to the baselines ControlFace aligns with the target control, maintaining the …
Figure 6
Figure 6. Figure 6: Comparison with Fine-tuning Method. We compare the results with the fine-tuned DiffusionRig [8] on two reference images. We show the generated results along with the identity similarity (ID) [5] and DECA [10] re-inference error. a reference image. Results in Tab. 2 sho…
Figure 7
Figure 7. Figure 7: Results on Out-of-domain Data. We test ControlFace on animation-like faces to illustrate the robustness and generaliz￾ability [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Ablation on Guidance Scale. We plot the DECA [10] re-inference error for guidance scales w from 1.0 to 4.0. Increasing w improves errors for RCG, CFG∗ , and CFG† to some extents, but greatly worsens for DiffusionRig [8]. input to the CLIP encoder, following the approac…
Figure 9
Figure 9. Figure 9: Example of user study. For each generated result, the participants were asked three questions. We provide detailed information about the user study. Following the methodology of ImagenHub [23], participants evaluated each model based on two criteria: (1) how well the g…
Figure 10
Figure 10. Figure 10: Qualitative results of model with only CLIP [32] image encoder. We train the denoising U￾Net with only CLIP image encoder attached and com￾pare the results with the FaceNet. We trained two additional models using different inputs to the CMM to identify the most suitab…
Figure 11
Figure 11. Figure 11: Qualitative results of training DiffuionRig [8] on video dataset. We illustrate the results of two variants of DiffusionRig that is trained on CelebV-HQ [54]. Reference Aligned Target Aligned Landmarks Three Rend. Lambertian Reference Control Results Reference Aligned…
Figure 12
Figure 12. Figure 12: Qualitative results of CMM input ablation. We provide generated images on three different models which takes different inputs for CMM. Interestingly, RCG exhibits over-saturated predicted X0 whereas CFG generates over-smoothed predictions. This happens because RCG is …
Figure 13
Figure 13. Figure 13: Visualization of RCG. We visualize each difference, ϵθ(·, DT ) − ϵθ(·, ∅) and ϵθ(·, DT ) − ϵθ(·, DR), along with the predicted X0 on each timesteps t. 4 [PITH_FULL_IMAGE:figures/full_fig_p015_13.png]
Figure 14
Figure 14. Figure 14: Additional results on FFHQ [21]. We randomly select faces from FFHQ [21] and rig the pose parameters. 5 [PITH_FULL_IMAGE:figures/full_fig_p016_14.png]
Figure 15
Figure 15. Figure 15: Additional results on FFHQ [21]. We randomly select faces from FFHQ [21] and rig the shape parameters for the results on the top and expression parameters for the bottom. 6 [PITH_FULL_IMAGE:figures/full_fig_p017_15.png]
Figure 16
Figure 16. Figure 16: Additional results on FFHQ [21]. We randomly select faces from FFHQ [21] and rig the light parameters. Reference Lambertian Normal Albedo Results [PITH_FULL_IMAGE:figures/full_fig_p018_16.png]
Figure 17
Figure 17. Figure 17: Additional results on out-of-distribution faces. We apply ControlFace on portraits obtained from the internet and generated by a text-to-2d diffusion model. 7 [PITH_FULL_IMAGE:figures/full_fig_p018_17.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 25 canonical work pages

  1. [1]

    Improving image generation with better captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8, 2023. 3

    2023

  2. [2]

    A morphable model for the synthesis of 3d faces

    V olker Blanz and Thomas Vetter. A morphable model for the synthesis of 3d faces. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2, pages 157–164. 2023. 1, 2, 3

    work page 2023

  3. [3]

    Large scale 3d morphable models

    James Booth, Anastasios Roussos, Allan Ponniah, David Dunaway, and Stefanos Zafeiriou. Large scale 3d morphable models. International Journal of Computer Vision , 126(2): 233–254, 2018. 2, 3

    work page 2018

  4. [4]

    Emoca: Emotion driven monocular face capture and animation

    Radek Dan ˇeˇcek, Michael J Black, and Timo Bolkart. Emoca: Emotion driven monocular face capture and animation. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 20311–20322, 2022. 1, 8

    work page 2022

  5. [5]

    Arcface: Additive angular margin loss for deep face recognition

    Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 4690–4699, 2019. 2, 5, 6, 7, 8

    work page 2019

  6. [6]

    Disentangled and controllable face image genera- tion via 3d imitative-contrastive learning

    Yu Deng, Jiaolong Yang, Dong Chen, Fang Wen, and Xin Tong. Disentangled and controllable face image genera- tion via 3d imitative-contrastive learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5154–5163, 2020. 2

    2020

  7. [7]

    8-bit optimizers via block-wise quantization

    Tim Dettmers, Mike Lewis, Sam Shleifer, and Luke Zettle- moyer. 8-bit optimizers via block-wise quantization. arXiv preprint arXiv:2110.02861, 2021. 1

    arXiv 2021

  8. [8]

    Diffusionrig: Learning personalized priors for facial appearance editing

    Zheng Ding, Xuaner Zhang, Zhihao Xia, Lars Jebe, Zhuowen Tu, and Xiuming Zhang. Diffusionrig: Learning personalized priors for facial appearance editing. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12736–12746, 2023. 1, 2, 3, 5, 6, 7, 8

    work page 2023

Show all 56 references
  1. [9]

    Scaling recti- fied flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. In Forty-first International Conference on Mach...

    2024

  2. [10]

    Learning an animatable detailed 3d face model from in-the- wild images

    Yao Feng, Haiwen Feng, Michael J Black, and Timo Bolkart. Learning an animatable detailed 3d face model from in-the- wild images. ACM Transactions on Graphics (ToG), 40(4): 1–13, 2021. 1, 3, 5, 7, 8, 2

    2021

  3. [11]

    Gif: Generative interpretable faces

    Partha Ghosh, Pravir Singh Gupta, Roy Uziel, Anurag Ran- jan, Michael J Black, and Timo Bolkart. Gif: Generative interpretable faces. In 2020 International Conference on 3D Vision (3DV), pages 868–878. IEEE, 2020. 1, 2, 5, 6, 7

    2020

  4. [12]

    Generative adversarial networks

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Commu- nications of the ACM, 63(11):139–144, 2020. 2

    2020

  5. [13]

    A generalist facex via learning unified facial representation

    Yue Han, Jiangning Zhang, Junwei Zhu, Xiangtai Li, Yan- hao Ge, Wei Li, Chengjie Wang, Yong Liu, Xiaoming Liu, and Ying Tai. A generalist facex via learning unified facial representation. arXiv preprint arXiv:2401.00551, 2023. 2, 3

    2023 arXiv

  6. [14]

    Ganspace: Discovering interpretable gan con- trols

    Erik H ¨ark¨onen, Aaron Hertzmann, Jaakko Lehtinen, and Sylvain Paris. Ganspace: Discovering interpretable gan con- trols. Advances in neural information processing systems , 33:9841–9850, 2020. 2

    2020

  7. [15]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 5

    2016

  8. [16]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 6, 7, 8, 2

    2017

  9. [17]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 2, 3, 5, 8, 1

    2022 arXiv

  10. [18]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 1

    2020

  11. [19]

    Animate anyone: Consistent and controllable image- to-video synthesis for character animation

    Li Hu. Animate anyone: Consistent and controllable image- to-video synthesis for character animation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8153–8163, 2024. 2, 4

    2024

  12. [20]

    Discontrolface: Disentangled control for personalized facial image editing

    Haozhe Jia, Yan Li, Hengfei Cui, Di Xu, Changpeng Yang, Yuwang Wang, and Tao Yu. Discontrolface: Disentangled control for personalized facial image editing. arXiv preprint arXiv:2312.06193, 2023. 1, 2, 3, 6

    2023 arXiv

  13. [21]

    A style-based generator architecture for generative adversarial networks

    Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 4401–4410, 2019. 2, 5, 6, 3, 7

    2019

  14. [22]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 3, 4, 1

    2013 arXiv

  15. [23]

    Imagenhub: Standardizing the evaluation of conditional image generation models

    Max Ku, Tianle Li, Kai Zhang, Yujie Lu, Xingyu Fu, Wen- wen Zhuang, and Wenhu Chen. Imagenhub: Standardizing the evaluation of conditional image generation models. In The Twelfth International Conference on Learning Represen- tations, 2024. 7, 1

    2024

  16. [24]

    Controlnet++: Improving conditional controls with efficient consistency feedback, 2024

    Ming Li, Taojiannan Yang, Huafeng Kuang, Jie Wu, Zhaon- ing Wang, Xuefeng Xiao, and Chen Chen. Controlnet++: Improving conditional controls with efficient consistency feedback, 2024. URL https://arxiv. org/abs/2404.07987. 2

    2024 arXiv

  17. [25]

    Learning a model of facial shape and expression from 4d scans

    Tianye Li, Timo Bolkart, Michael J Black, Hao Li, and Javier Romero. Learning a model of facial shape and expression from 4d scans. ACM Trans. Graph., 36(6):194–1, 2017. 1, 3

    2017

  18. [26]

    Caphuman: Capture your moments in parallel universes

    Chao Liang, Fan Ma, Linchao Zhu, Yingying Deng, and Yi Yang. Caphuman: Capture your moments in parallel universes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6400– 6409, 2024. 1, 2, 3, 5, 6, 7

    2024

  19. [27]

    3d-fm gan: Towards 3d-controllable face manipulation

    Yuchen Liu, Zhixin Shu, Yijun Li, Zhe Lin, Richard Zhang, and Sun-Yuan Kung. 3d-fm gan: Towards 3d-controllable face manipulation. In European Conference on Computer Vision, pages 107–125. Springer, 2022. 1, 2 9

    2022

  20. [28]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 4296–4304, 2024. 2

    2024

  21. [29]

    Arc2face: A foundation model of human faces

    Foivos Paraperas Papantoniou, Alexandros Lattas, Stylianos Moschoglou, Jiankang Deng, Bernhard Kainz, and Stefanos Zafeiriou. Arc2face: A foundation model of human faces. arXiv preprint arXiv:2403.11641, 2024. 1, 2, 5, 6, 7

    2024 arXiv

  22. [30]

    Controlnext: Powerful and effi- cient control for image and video generation

    Bohao Peng, Jian Wang, Yuechen Zhang, Wenbo Li, Ming- Chang Yang, and Jiaya Jia. Controlnext: Powerful and effi- cient control for image and video generation. arXiv preprint arXiv:2408.06070, 2024. 2, 4, 8

    2024 arXiv

  23. [31]

    Diffusion autoen- coders: Toward a meaningful and decodable representation

    Konpat Preechakul, Nattanat Chatthee, Suttisak Wizad- wongsa, and Supasorn Suwajanakorn. Diffusion autoen- coders: Toward a meaningful and decodable representation. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 10619–10629, 2022. 2

    2022

  24. [32]

    Learning transferable visual models from natural language supervision, 2021

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. 2, 3, 4, 5, 1

    2021

  25. [33]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67, 2020. 2

    2020

  26. [34]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 1, 3, 8

    2022 arXiv

  27. [35]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 2, 3, 5, 8

    2022

  28. [36]

    Palette: Image-to-image diffusion models

    Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 conference proceedings, pages 1–10, 2022

    2022

  29. [37]

    Image super- resolution via iterative refinement

    Chitwan Saharia, Jonathan Ho, William Chan, Tim Sali- mans, David J Fleet, and Mohammad Norouzi. Image super- resolution via iterative refinement. IEEE transactions on pattern analysis and machine intelligence, 45(4):4713–4726,

  30. [38]

    Genwarp: Single image to novel views with semantic-preserving generative warping

    Junyoung Seo, Kazumi Fukuda, Takashi Shibuya, Takuya Narihira, Naoki Murata, Shoukang Hu, Chieh-Hsin Lai, Se- ungryong Kim, and Yuki Mitsufuji. Genwarp: Single image to novel views with semantic-preserving generative warping. arXiv preprint arXiv:2405.17251, 2024. 4

    2024 arXiv

  31. [39]

    Faceid-gan: Learning a symmetry three-player gan for identity-preserving face synthesis

    Yujun Shen, Ping Luo, Junjie Yan, Xiaogang Wang, and Xi- aoou Tang. Faceid-gan: Learning a symmetry three-player gan for identity-preserving face synthesis. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, pages 821–830, 2018. 2

    2018

  32. [40]

    Interfacegan: Interpreting the disentangled face representa- tion learned by gans

    Yujun Shen, Ceyuan Yang, Xiaoou Tang, and Bolei Zhou. Interfacegan: Interpreting the disentangled face representa- tion learned by gans. IEEE transactions on pattern analysis and machine intelligence, 44(4):2004–2018, 2020. 2

    2004

  33. [41]

    Instadrag: Lightning fast and accurate drag- based image editing emerging from videos

    Yujun Shi, Jun Hao Liew, Hanshu Yan, Vincent YF Tan, and Jiashi Feng. Instadrag: Lightning fast and accurate drag- based image editing emerging from videos. arXiv preprint arXiv:2405.13722, 2024. 4

    2024 arXiv

  34. [42]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 1

    2010 arXiv

  35. [43]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. arXiv preprint arXiv:2011.13456, 2020. 1

    2011 arXiv

  36. [44]

    Stylerig: Rigging style- gan for 3d control over portrait images

    Ayush Tewari, Mohamed Elgharib, Gaurav Bharaj, Florian Bernard, Hans-Peter Seidel, Patrick P ´erez, Michael Zoll- hofer, and Christian Theobalt. Stylerig: Rigging style- gan for 3d control over portrait images. In Proceedings of the IEEE/CVF conference on computer vision and p...

    2020

  37. [45]

    Real- time expression transfer for facial reenactment

    Justus Thies, Michael Zollh ¨ofer, Matthias Nießner, Levi Val- gaerts, Marc Stamminger, and Christian Theobalt. Real- time expression transfer for facial reenactment. ACM Trans. Graph., 34(6):183–1, 2015. 1

    2015

  38. [46]

    Designing an encoder for stylegan image manipulation

    Omer Tov, Yuval Alaluf, Yotam Nitzan, Or Patashnik, and Daniel Cohen-Or. Designing an encoder for stylegan image manipulation. ACM Transactions on Graphics (TOG), 40(4): 1–14, 2021. 2

    2021

  39. [47]

    Face0: Instantaneously conditioning a text-to- image model on a face

    Dani Valevski, Danny Lumen, Yossi Matias, and Yaniv Leviathan. Face0: Instantaneously conditioning a text-to- image model on a face. InSIGGRAPH Asia 2023 Conference Papers, pages 1–10, 2023. 2

    2023

  40. [48]

    Instantid: Zero-shot identity-preserving generation in seconds

    Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, Anthony Chen, Huaxia Li, Xu Tang, and Yao Hu. Instantid: Zero-shot identity-preserving generation in seconds. arXiv preprint arXiv:2401.07519, 2024. 2

    2024 arXiv

  41. [49]

    Magicanimate: Temporally consistent human image animation using diffusion model

    Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. Magicanimate: Temporally consistent human image animation using diffusion model. 2024. 2, 4

    2024

  42. [50]

    Facestudio: Put your face everywhere in seconds

    Yuxuan Yan, Chi Zhang, Rui Wang, Yichao Zhou, Gege Zhang, Pei Cheng, Gang Yu, and Bin Fu. Facestudio: Put your face everywhere in seconds. arXiv preprint arXiv:2312.02663, 2023. 2

    2023 arXiv

  43. [51]

    Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models. arXiv preprint arXiv:2308.06721,

    arXiv

  44. [52]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 3, 4, 8

    2023

  45. [53]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 6, 7, 8 10

    2018

  46. [54]

    CelebV- HQ: A large-scale video facial attributes dataset

    Hao Zhu, Wayne Wu, Wentao Zhu, Liming Jiang, Siwei Tang, Li Zhang, Ziwei Liu, and Chen Change Loy. CelebV- HQ: A large-scale video facial attributes dataset. In ECCV,

  47. [55]

    Champ: Controllable and consistent human image animation with 3d parametric guidance

    Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, and Siyu Zhu. Champ: Controllable and consistent human image animation with 3d parametric guidance. arXiv preprint arXiv:2403.14781 ,

    arXiv

  48. [56]

    Towards metrical reconstruction of human faces

    Wojciech Zielonka, Timo Bolkart, and Justus Thies. Towards metrical reconstruction of human faces. In European confer- ence on computer vision, pages 250–269. Springer, 2022. 1, 8 11 ControlFace: Harnessing Facial Parametric Control for Face Rigging - Supplementary Material - ...

    2022

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.