Pith. sign in

REVIEW 4 major objections 5 minor 61 references

Disentangling 3D from Large Vision-Language Models for Controlled Portrait Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A pretrained CLIP model can drive text-controlled appearance in a deformable 3D portrait GAN, provided every training image is canonicalized to a fixed frontal, neutral-geometry frame and the conditioning network is regularized against…

desk verdict Canonicalization is a clever trick and the Jacobian regularizer is solid math; the paper deserves peer review, but the central invariance assumption needs a quantitative test. read the letter →

arxiv 2506.14015 v1 pith:BQ47AUFP submitted 2025-06-16 cs.CV

Nick Yiwen Huang , Akin Caliskan , Berkay Kicanaoglu , James Tompkin , Hyeongwoo Kim This is my paper
classification cs.CV
keywords 3D-awareGANtext-guidedportraitgenerationCLIPdisentanglementFLAMEmorphablemodelcanonicalizationJacobianregularizationdistributioncollapseunlabeled2Dfacedata
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes CLIPortrait, a way to make a pretrained large vision-language model such as CLIP control the appearance of generated 3D portraits while a parametric face model controls expression and camera pose. The central claim is that 3D information can be disentangled from CLIP's embeddings by canonicalization: every training image is first re-rendered in a fixed frontal pose with neutral face geometry, so the CLIP embedding can only describe appearance, not pose or shape. A second, subtler entanglement, namely per-image noise in CLIP embeddings that would collapse the generator to a single deterministic output, is removed with a Jacobian regularization that keeps the conditioning network insensitive to that noise. If this works, creators can add free-form text control and explicit 3D control to their own unlabeled 2D face data, without paired text labels or the compute to train a large model.

What carries the argument

The load-bearing device is two-stage canonicalization. Stage 1 trains an unconditional deformable 3D generator; then each image is projected back into the latent space and re-rendered with neutral FLAME parameters and a frontal camera, producing a canonical image whose CLIP embedding contains no pose or shape signal. The conditioning mechanism is an alignment network, zero-initialized, that adds a learned direction to the generator's style latent vector, with a scalar $\alpha$ controlling alignment strength. To stop CLIP's per-image noise from making the generator deterministic, the paper regularizes the squared Frobenius norm of $\partial w_{\hat{x}}/\partial r_{\hat{x}}$ through the identity $\mathbb{E}[\|\partial w/\partial r\|_F^2]=\lim_{\sigma\to 0}\mathbb{E}[\|w(r+\epsilon)-w(r)\|^2/\sigma^2]$, which is cheap to evaluate with one Gaussian perturbation; this bounds the sensitivity of the full generator to the condition and, with a norm penalty and alternating unconditional training, preserves diversity.

What would settle it

Take unlabeled face images, compute their canonicalized CLIP embeddings $r_{\hat{x}}$, and test whether a linear classifier can predict the original camera yaw or FLAME expression coefficients from $r_{\hat{x}}$ above chance; if it can, the canonicalization has not removed 3D information and the central disentanglement claim fails.

Watch

Extended reading notes

Core claim

The paper's discovery is that the alignment objective of large vision-language models entangles geometry, camera, and appearance, and that both entanglements can be removed by structuring the generator rather than by adding labels. The method uses a deformable neural 3D tri-plane generator with FLAME-based deformation, so camera and face geometry are explicit controls. Every dataset image is fitted, inverted, and re-rendered in a canonical frontal, neutral-geometry frame; CLIP encodes only this canonical image, making the text-conditioning vector a pure appearance signal. The remaining collapse caused by CLIP's noise dimensions is prevented by penalizing the sensitivity of the predicted style direction to the CLIP embedding, computed by a stochastic finite-difference approximator, together with a norm-growth penalty and randomized unconditional training. The result is claimed to be the only text-guided 3D portrait generator that produces high-quality, photorealistic outputs with independent geometry and camera control from an unlabelled 2D dataset.

Load-bearing premise

The canonical frontal re-render is assumed to contain no 3D information: DECA/FLAME fitting is assumed to remove all pose, shape, and expression variation before CLIP sees the image, so any residual geometric or lighting cue left in the canonical image would keep geometry entangled with the text condition.

Editorial extensions

If this is right

  • Changing a text prompt such as 'he has a beard' alters appearance only, while FLAME parameters control expression and camera, so editing one does not change the other.
  • A creator with an unlabeled 2D face dataset, such as proprietary studio data, can obtain free-form text control and explicit 3D control without manual attribute labeling.
  • The Jacobian regularization prevents CLIP's noise factors from collapsing the conditional distribution, so a generic prompt yields diverse portraits rather than one fixed face.
  • At inference, new prompts are applied in about 0.10 seconds, in contrast to optimization-based methods that retrain for each prompt and take minutes.
  • The recipe transfers to other vision-language models and other deformable 3D generators, since canonicalization and regularization are not tied to CLIP or to the tri-plane backbone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same canonicalize-then-condition pattern should generalize to any object class with a parametric deformation model, such as bodies, hands, or animals, provided the canonical render removes all pose and shape variation before the vision-language embedding.
  • A boundary the paper leaves implicit is that because geometry is deliberately stripped from the CLIP condition, prompts that name geometric attributes such as 'side profile' or 'open mouth' may fail or be ignored, since those factors are routed through the FLAME controls instead.
  • The Jacobian penalty targets an upper bound via the chain rule and drops the $\partial G/\partial w$ factor, so residual sensitivity through the style mapping could remain; a direct test would measure per-prompt diversity with $\alpha$ held fixed.
  • The canonical image still contains lighting, background, and incidental texture, so those cues can continue to act as noise; segmenting the canonical render to the face region before CLIP encoding is a natural testable extension.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes CLIPortrait, a two-stage method for training a deformable 3D-aware GAN on unlabeled 2D face datasets, using CLIP as a text-conditioning source and FLAME as the geometric prior. Stage 1 trains an unconditional deformable tri-plane generator; each training image is then canonicalized by inverting the fitted FLAME deformation and re-rendering from a frontal neutral camera, and the CLIP embedding of this canonicalized image serves as the appearance condition. Stage 2 trains alignment networks for the generator and discriminator, adding a Jacobian regularization on the style-vector change with respect to the CLIP condition and a norm-preservation regularizer to prevent distribution collapse. The authors report FID, KID, and CLIP-score results on FFHQ and MMCelebA, show qualitative text-to-3D portrait synthesis and editing, and claim to be the only method combining text guidance, explicit geometry and camera control, and unlabeled 2D-only training.

Significance. If validated, the core ideas are valuable: identifying that CLIP embeddings of uncanonicalized images entangle pose and geometry, and that per-instance noise in the CLIP embedding causes conditional GAN collapse, is a useful diagnosis, and the proposed canonicalization plus the stochastic Jacobian regularizer is a practical remedy. The Appendix A proof of the finite-difference estimator for the squared Frobenius Jacobian norm is mathematically sound, and the qualitative evidence in Figs. 4, 5, 7, 8, and 10 is compelling. The method is also potentially generic, since the LVLM and the 3D backbone are treated as swappable components. However, the central disentanglement claim currently rests on an unquantified invariance assumption, and the quantitative comparisons are reported without error bars and with a partially circular text-alignment metric. These issues are addressable with additional experiments, and the contribution remains potentially significant for controllable 3D portrait generation from small unlabeled datasets.

major comments (4)
  1. [§3.3, Eq. (10)] The central claim that r_hatx is free of 3D information is not quantitatively tested. The canonicalization pipeline assumes that DECA-based FLAME fitting, after landmark refinement, exactly recovers c_cam and c_geo for every training image, and that the Stage-1 canonical volume is genuinely pose- and expression-invariant under deformation. Both assumptions are plausible but unverified, and residual shape or expression errors, or shading and lighting cues in the frontal render, could leak geometry into r_hatx and couple text to geometry. Please add a direct invariance test: for fixed w_x, measure the CLIP embedding distance between x_hat rendered at different c_cam and c_geo values, or train a linear probe for pose and expression from r_hatx, and report quantitative disentanglement metrics such as how much text edits change the rendered geometry. The qualitative evidence in Fig. 4 does not by itself establish invariance.
  2. [Table 1] The quantitative comparison as reported is not conclusive. FID and KID values are given without confidence intervals, bootstrap estimates, or the number of random samples used, and the FID column mixes volume-rendered methods with rasterization-based baselines whose achievable fidelity differs. On MMCelebA, the CLIP score is computed against the provided text annotations, but the generator is conditioned on CLIP embeddings, making the metric partially circular as a measure of text alignment. Please report means and variances over multiple seeds or bootstrap resamples, specify the sampling protocol, and include a non-CLIP evaluation, such as human ratings or a different vision-language model, for at least one configuration.
  3. [§3.5, Eqs. (18)–(20)] The Jacobian regularizer is motivated by bounding the norm of ∂G/∂r_hatx, but Eq. (18) only gives an upper bound up to the omitted factor ||∂G/∂w_rhatx||_2; the paper does not show that this factor is bounded or that penalizing ||∂w_rhatx/∂r_hatx||_F suffices to control generator sensitivity. The qualitative diversity improvement in Fig. 7 supports the heuristic, but the collapse claim should be backed by quantitative diversity measurements, such as pairwise LPIPS variance or conditional FID, and by an ablation of the regularizer weights. In addition, the finite-difference estimator uses one epsilon sample with sigma = 0.1, and a small sensitivity study of sigma and the number of samples would clarify the bias-variance trade-off.
  4. [Table 3 vs. §5] There is a direct contradiction affecting the claim that CLIPortrait is the only method with all listed properties: Table 3 marks TG-3DFace as supporting 'Image only training' with a checkmark, while §5 states that TG-3DFace has a 'requirement of text annotated training dataset.' Please correct the table or the text, and define the comparison criteria consistently across all rows so that the 'only method' claim is unambiguous.
minor comments (5)
  1. [Fig. 4 caption] The caption says '[FFHQ 5mil. images.]' but FFHQ contains 70,000 images; this appears to be a typo and should be corrected.
  2. [Eq. (1)] The InfoNCE denominator is written over x' sampled from X and t' sampled from T; please clarify whether this denotes the batch used for the contrastive loss, since a sum over the entire dataset would not match the standard CLIP training objective.
  3. [§2] The symbol c_imp is introduced for the implicit appearance condition, but Stage-2 mostly uses r_hatx; please state explicitly that c_imp equals r_hatx and define the dimensionality and normalization of r_hatx in the implementation.
  4. [Appendix B] The choice of sigma = 0.1 and one epsilon vector for the finite-difference estimator is reported, but no sensitivity analysis is provided; a short experiment varying sigma and the number of samples would make the approximation claim more robust.
  5. [§4] The evaluation protocol for 'random samples for noise and CLIP embedding' should specify the number of samples, the random seed, and whether alpha = 0 unconditional samples are included, so that the FID and KID numbers are reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No load-bearing circularity: canonicalization and the Jacobian regularization are constructed independently of the text-alignment target, and the only same-author citation is an implementation detail rather than a justification of the central claim.

full rationale

The derivation chain is self-contained. Stage-1 fits an unconditional deformable generator to unlabeled images; the canonicalized condition r_hat_x = E_img(hat_x) in Eq. (10) is an empirical CLIP embedding of a re-rendered neutral image obtained by LPIPS inversion of w_x, not by fitting the text-alignment target. Stage-2 trains an alignment network to move a random style vector toward r_hat_x and at inference substitutes the text embedding r_t; whether this substitution generalizes is an empirical claim evaluated by FID/KID and CLIP score, not a quantity forced by construction. The Jacobian regularization in Eqs. (19)-(20) follows from a chain-rule upper bound and a finite-difference identity, both derived in the appendix, and it addresses a separately documented distribution-collapse phenomenon; it does not presuppose the disentanglement result. The only same-author citation ([17], Huang et al.) is used for a discriminator-conditioning implementation detail in Stage-1, not as justification for the central disentanglement or uniqueness claim. The CLIP-score metric does share the CLIP model used for conditioning, which is a metric-validity caveat, but the paper explicitly acknowledges this ('CLIP score is not a reliable measure on its own without FID') and reports FID alongside, so this does not reduce the derivation to its inputs. No load-bearing step equates a fitted parameter with a prediction, and no uniqueness theorem is imported from the authors' prior work.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claims rest on the canonicalization hypothesis and the Jacobian-regularizer proxy; both are method-level assumptions rather than proven statements. Four hand-set hyperparameters are required. No new physical or conceptual entities are introduced.

free parameters (4)
  • JacReg_weight = 0.01
    Weight of the Jacobian regularizer Rrhat in the Stage-2 objective (Sec. 3.5, Appendix B). Chosen empirically; directly controls the strength of the collapse fix.
  • NormReg_weight = 10
    Weight of the norm-growth regularizer Rnorm (Eq. 21). Prevents the condition vector from drifting out of the trained manifold.
  • sigma_finite_diff = 0.1
    Noise scale for the stochastic finite-difference approximator of the Jacobian (Eq. 20, Appendix A). Affects the bias-variance tradeoff of the regularizer.
  • alpha_uncond_prob = 0.5
    Probability of setting alpha=0 (purely unconditional generation) during Stage-2 training (Sec. 3.5). This forces the model to retain diversity.
assumptions (5)
  • standard math InfoNCE maximizes a lower bound on the mutual information between CLIP image and text embeddings (Eqs. 1-2).
    Invoked in Sec. 2 to justify why CLIP embeddings carry entangled, spurious factors.
  • domain assumption DECA and FLAME provide accurate per-image face geometry (shape, expression, pose) for the FFHQ/MMCelebA datasets.
    Used to build the deformation field D (Eq. 5) and to render canonicalized images (Eq. 10). Inaccurate fits would leave residual geometry in the canonical images.
  • ad hoc to paper A frontal, neutral-geometry canonical render removes all 3D information from the CLIP embedding of that render.
    Core hypothesis of the disentanglement strategy (Sec. 3.3, Fig. 4). Provided as empirical observation, not a theorem. If lighting or residual 3D cues remain, geometry leaks back into rhat.
  • ad hoc to paper Penalizing the alignment network Jacobian ||d(w_hat)/d(rhat)|| is a sufficient proxy for controlling the generator's sensitivity to rhat.
    Used in Eq. (18) where the ||dG/dw|| term is dropped. The inequality is valid, but the upper bound may be loose; the paper relies on this for diversity preservation.
  • domain assumption The EG3D checkpoint trained on FFHQ is an acceptable backbone whose distribution overlap with the target datasets does not materially inflate the reported FID.
    All experiments warm-start from the official FFHQ EG3D checkpoint (Appendix B). The paper does not ablate from-scratch training, so part of the FID improvement may stem from the initialization rather than the conditioning method.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Disentangling 3D from Large Vision-Language Models for Controlled Portrait Generation." pith.science (2026). https://pith.science/paper/BQ47AUFP

@misc{pith2026250614015,
  author       = {Pith},
  title        = {Pith review of: Disentangling 3D from Large Vision-Language Models for Controlled Portrait Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BQ47AUFP}},
  note         = {Machine review of arXiv:2506.14015}
}
read the original abstract

We consider the problem of disentangling 3D from large vision-language models, which we show on generative 3D portraits. This allows free-form text control of appearance attributes like age, hair style, and glasses, and 3D geometry control of face expression and camera pose. In this setting, we assume we use a pre-trained large vision-language model (LVLM; CLIP) to generate from a smaller 2D dataset with no additional paired labels and with a pre-defined 3D morphable model (FLAME). First, we disentangle using canonicalization to a 2D reference frame from a deformable neural 3D triplane representation. But another form of entanglement arises from the significant noise in the LVLM's embedding space that describes irrelevant features. This damages output quality and diversity, but we overcome this with a Jacobian regularization that can be computed efficiently with a stochastic approximator. Compared to existing methods, our approach produces portraits with added text and 3D control, where portraits remain consistent when either control is changed. Broadly, this approach lets creators control 3D generators on their own 2D face data without needing resources to label large data or train large models.

Figures

Figures reproduced from arXiv: 2506.14015 by the authors.

Figure 1
Figure 1. CLIPortrait allows text-guided 3D portrait generation [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of Stage-1. We use a deformable 3D generative model for Stage-1. We adapt GNARF [2] to improve the geometry conditioning of the discriminator using mesh renders. 2) The contrastive objective forces rx to be as discrimi￾native as possible in Y, but the most discriminative factors in rx could be irrelevant to portraits—non-portrait noise fac￾tors rxnoise can outweigh any useful factors when using rx as cimp. … view at source ↗
Figure 3
Figure 3. Overview of Stage-2. (a) The conditioning networks remap CLIP embeddings to a space wr in which 3D information is ignored to be considered in stage (b), while noise vector z maintains sample diversity. (b) We synthesize a neural radiance volume via a tri-plane in a 3D canonical space, and with a particular appearance defined by wr. Then, we deform this volume by FLAME parameters after fitting to the dataset. A discr… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Without canonicalization before CLIP, faces look flat. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Increasing α increases prompt alignment. Text Prompt: ”Bearded man with long blond hair wearing glasses” CLIP corre￾lates regular glasses and sunglasses as increasing “glasses” intensity, and we observe a similar phenomenon. it aligns with rxˆ. Note that our Stage 1 mo…
Figure 6
Figure 6. Figure 6: The compari￾son of cosine similarity be￾tween CLIP image embed￾dings of training images and CLIP text embeddings for prompts related to faces and those not related to faces. Noise prompts can have higher cosine similarity than face prompts. Further, the average cosine …
Figure 7
Figure 7. Figure 7: Showing distribution collapse. Our noise regularization improves diversity and quality. [FFHQ 5mil. images.] ∂G/∂wrxˆ is the computation bottleneck and not directly relevant to rxˆ, we thus omit this Jacobian term and pe￾nalize ∥∂wrxˆ /∂rxˆ∥F , which penalizes the uppe…
Figure 8
Figure 8. Figure 8: Qualitative Evaluation On text-to-3D portrait generation with explicit geometry control. Method FFHQ Method MM-Celeb-A-HQ FID ↓ KID ↓ FID ↓ CLIP Score ↑ Text2Mesh* [30] 219.59 0.185 SEA-T2F [44] 93.8 20.8 ClipMatrix* [19] 198.34 0.180 ControlGAN [25] 74.5 21.3 FlameTex…
Figure 9
Figure 9. Figure 9: Text-guided 3D face appearance manipulation. Increasing [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: Text and 3D control (face shape, expression, and camera). Our method shows improved quality against CLIPFace [1], and improved diversity and control against TG3DFace [52], which cannot vary face shape or expression. These examples have α = 1, showing the most strong o…
Figure 12
Figure 12. Figure 12: Text-guided appearance editing results [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

61 extracted references · 47 canonical work pages

  1. [1]

    Clipface: Text-guided editing of textured 3d mor- phable models

    Shivangi Aneja, Justus Thies, Angela Dai, and Matthias Nießner. Clipface: Text-guided editing of textured 3d mor- phable models. InACM SIGGRAPH 2023 Conference Pro- ceedings, pages 1–11, 2023. 7, 8, 13

    work page 2023

  2. [2]

    Bergman, Petr Kellnhofer, Yifan Wang, Eric R

    Alexander W. Bergman, Petr Kellnhofer, Yifan Wang, Eric R. Chan, David B. Lindell, and Gordon Wetzstein. Generative neural articulated radiance fields. InNeurIPS, 2022. 1, 2, 3, 4, 7, 8, 12

    work page 2022

  3. [3]

    Demystifying mmd gans.arXiv preprint arXiv:1801.01401, 2018

    Mikołaj Bi´nkowski, Danica J Sutherland, Michael Arbel, and Arthur Gretton. Demystifying mmd gans.arXiv preprint arXiv:1801.01401, 2018. 7

    arXiv 2018

  4. [4]

    Text and image guided 3d avatar generation and ma- nipulation

    Zehranaz Canfes, M Furkan Atasoy, Alara Dirik, and Pinar Yanardag. Text and image guided 3d avatar generation and ma- nipulation. InProceedings of the IEEE/CVF Winter Confer- ence on Applications of Computer Vision, pages 4421–4431,

    work page

  5. [5]

    Chan, Connor Z

    Eric R. Chan, Connor Z. Lin, Matthew A. Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas Guibas, Jonathan Tremblay, Sameh Khamis, Tero Karras, and Gordon Wetzstein. Efficient geometry-aware 3D generative adversarial networks. InCVPR, 2022. 3, 4, 7, 8

    work page 2022

  6. [6]

    Efficient Text-Guided 3D-Aware Portrait Generation with Score Distillation Sampling on Distribution

    Yiji Cheng, Fei Yin, Xiaoke Huang, Xintong Yu, Jiaxiang Liu, Shikun Feng, Yujiu Yang, and Yansong Tang. Efficient text- guided 3d-aware portrait generation with score distillation sampling on distribution.arXiv preprint arXiv:2306.02083,

    work page Pith review arXiv

  7. [7]

    Generalizable and animatable gaussian head avatar.arXiv preprint arXiv:2410.07971, 2024

    Xuangeng Chu and Tatsuya Harada. Generalizable and animatable gaussian head avatar.arXiv preprint arXiv:2410.07971, 2024. 2

    arXiv 2024

  8. [8]

    Gen- erative adversarial networks: An overview.IEEE signal processing magazine, 35(1):53–65, 2018

    Antonia Creswell, Tom White, Vincent Dumoulin, Kai Arulkumaran, Biswa Sengupta, and Anil A Bharath. Gen- erative adversarial networks: An overview.IEEE signal processing magazine, 35(1):53–65, 2018. 7

    work page 2018

Show all 61 references
  1. [9]

    Cogview: Mastering text-to-image generation via transformers.Advances in Neural Information Processing Systems, 34:19822–19835, 2021

    Ming Ding, Zhuoyi Yang, Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, et al. Cogview: Mastering text-to-image generation via transformers.Advances in Neural Information Processing Systems, 34:19822–19835, 2021. 8

    2021

  2. [10]

    Cogview2: Faster and better text-to-image generation via hierarchical transformers.Advances in Neural Information Processing Systems, 35:16890–16902, 2022

    Ming Ding, Wendi Zheng, Wenyi Hong, and Jie Tang. Cogview2: Faster and better text-to-image generation via hierarchical transformers.Advances in Neural Information Processing Systems, 35:16890–16902, 2022. 8

    2022

  3. [11]

    Semantic image synthesis via adversarial learning

    Hao Dong, Simiao Yu, Chao Wu, and Yike Guo. Semantic image synthesis via adversarial learning. InProceedings of the IEEE international conference on computer vision, pages 5706–5714, 2017. 8

    2017

  4. [12]

    Imagebart: Bidirectional context with multinomial diffusion for autoregressive image synthesis.Advances in neural information processing systems, 34:3518–3532, 2021

    Patrick Esser, Robin Rombach, Andreas Blattmann, and Bjorn Ommer. Imagebart: Bidirectional context with multinomial diffusion for autoregressive image synthesis.Advances in neural information processing systems, 34:3518–3532, 2021. 8

    2021

  5. [13]

    Black, and Timo Bolkart

    Yao Feng, Haiwen Feng, Michael J. Black, and Timo Bolkart. Learning an animatable detailed 3D face model from in-the- wild images. 2021. 3

    2021

  6. [14]

    Generative adversarial nets.Advances in neural information processing systems, 27, 2014

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets.Advances in neural information processing systems, 27, 2014. 13

    2014

  7. [15]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bern- hard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017. 7

    2017

  8. [16]

    Denoising diffu- sion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020. 8

    2020

  9. [17]

    Removing the quality tax in controllable face gener- ation

    Yiwen Huang, Zhiqiu Yu, Xinjie Yi, Yue Wang, and James Tompkin. Removing the quality tax in controllable face gener- ation. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 5364–5373,

  10. [18]

    Adversarial generation of hierarchical gaussians for 3d generative model.arXiv preprint arXiv:2406.02968, 2024

    Sangeek Hyun and Jae-Pil Heo. Adversarial generation of hierarchical gaussians for 3d generative model.arXiv preprint arXiv:2406.02968, 2024. 2

    2024 arXiv

  11. [19]

    Clipmatrix: Text-controlled creation of 3d textured meshes.arXiv preprint arXiv:2109.12922, 2021

    Nikolay Jetchev. Clipmatrix: Text-controlled creation of 3d textured meshes.arXiv preprint arXiv:2109.12922, 2021. 7

    2021 arXiv

  12. [20]

    A style-based generator architecture for generative adversarial networks

    Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4401–4410, 2019. 2, 6, 7

    2019

  13. [21]

    Analyzing and improv- ing the image quality of stylegan

    Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improv- ing the image quality of stylegan. InProceedings of the IEEE/CVF conference on computer vision and pattern recog- nition, pages 8110–8119, 2020. 7

    2020

  14. [22]

    Gghead: Fast and gener- alizable 3d gaussian heads.arXiv preprint arXiv:2406.09377,

    Tobias Kirschstein, Simon Giebenhain, Jiapeng Tang, Markos Georgopoulos, and Matthias Nießner. Gghead: Fast and gener- alizable 3d gaussian heads.arXiv preprint arXiv:2406.09377,

    arXiv

  15. [23]

    Gaus- sian3diff: 3d gaussian diffusion for 3d full head synthesis and editing

    Yushi Lan, Feitong Tan, Di Qiu, Qiangeng Xu, Kyle Gen- ova, Zeng Huang, Sean Fanello, Rohit Pandey, Thomas Funkhouser, Chen Change Loy, and Yinda Zhang. Gaus- sian3diff: 3d gaussian diffusion for 3d full head synthesis and editing. InECCV, 2024. 2

    2024

  16. [24]

    Autoregressive image generation using resid- ual quantization

    Doyup Lee, Chiheon Kim, Saehoon Kim, Minsu Cho, and Wook-Shin Han. Autoregressive image generation using resid- ual quantization. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11523– 11532, 2022. 8

    2022

  17. [25]

    Controllable text-to-image generation.Advances in Neural Information Processing Systems, 32, 2019

    Bowen Li, Xiaojuan Qi, Thomas Lukasiewicz, and Philip Torr. Controllable text-to-image generation.Advances in Neural Information Processing Systems, 32, 2019. 7, 8

    2019

  18. [26]

    Tianye Li, Timo Bolkart, Michael. J. Black, Hao Li, and Javier Romero. Learning a model of facial shape and expres- sion from 4D scans.ACM Transactions on Graphics, (Proc. SIGGRAPH Asia), 36(6):194:1–194:17, 2017. 1, 3, 7

    2017

  19. [27]

    Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning.Advances in Neural Information Processing Systems, 35:17612–17625, 2022

    Victor Weixin Liang, Yuhui Zhang, Yongchan Kwon, Serena Yeung, and James Y Zou. Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning.Advances in Neural Information Processing Systems, 35:17612–17625, 2022. 2

    2022

  20. [28]

    Visual instruction tuning.Advances in neural information processing systems, 36, 2024

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36, 2024. 2

    2024

  21. [29]

    Which training methods for gans do actually converge? In International conference on machine learning, pages 3481–

    Lars Mescheder, Andreas Geiger, and Sebastian Nowozin. Which training methods for gans do actually converge? In International conference on machine learning, pages 3481–

  22. [30]

    Text2mesh: Text-driven neural stylization for meshes

    Oscar Michel, Roi Bar-On, Richard Liu, Sagie Benaim, and Rana Hanocka. Text2mesh: Text-driven neural stylization for meshes. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13492– 13502, 2022. 7

    2022

  23. [31]

    Text2facegan: Face generation from fine grained textual de- scriptions

    Osaid Rehman Nasir, Shailesh Kumar Jha, Manraj Singh Grover, Yi Yu, Ajit Kumar, and Rajiv Ratn Shah. Text2facegan: Face generation from fine grained textual de- scriptions. In2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM), pages 58–67. IEEE, 2019. 8

    2019

  24. [32]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021. 8

    2021 arXiv

  25. [33]

    Repre- sentation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748, 2018

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Repre- sentation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748, 2018. 2

    2018 arXiv

  26. [34]

    Paysan, R

    P. Paysan, R. Knothe, B. Amberg, S. Romdhani, and T. Vetter. A 3d face model for pose and illumination invariant face recognition. Genova, Italy, 2009. IEEE. 1

    2009

  27. [35]

    Towards open-ended text-to-face generation, combination and manipulation

    Jun Peng, Han Pan, Yiyi Zhou, Jing He, Xiaoshuai Sun, Yan Wang, Yongjian Wu, and Rongrong Ji. Towards open-ended text-to-face generation, combination and manipulation. In Proceedings of the 30th ACM International Conference on Multimedia, pages 5045–5054, 2022. 8

    2022

  28. [36]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, p...

    2021

  29. [37]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. InInternational Confer- ence on Machine Learning, pages 8821–8831. PMLR, 2021. 8

    2021

  30. [38]

    Hierarchical text-conditional image genera- tion with clip latents.arXiv preprint arXiv:2204.06125, 1(2): 3, 2022

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image genera- tion with clip latents.arXiv preprint arXiv:2204.06125, 1(2): 3, 2022. 8

    2022 arXiv

  31. [39]

    Generative adver- sarial text to image synthesis

    Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Lo- geswaran, Bernt Schiele, and Honglak Lee. Generative adver- sarial text to image synthesis. InInternational conference on machine learning, pages 1060–1069. PMLR, 2016. 8

    2016

  32. [40]

    Higher order contractive auto-encoder

    Salah Rifai, Gr´egoire Mesnil, Pascal Vincent, Xavier Muller, Yoshua Bengio, Yann Dauphin, and Xavier Glorot. Higher order contractive auto-encoder. InMachine Learning and Knowledge Discovery in Databases, pages 645–660, Berlin, Heidelberg, 2011. Springer Berlin Heidelberg. 12

    2011

  33. [41]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. 2022 ieee. InCVF Con- ference on Computer Vision and Pattern Recognition (CVPR), pages 10674–10685, 2021. 8

    2022

  34. [42]

    Pho- torealistic text-to-image diffusion models with deep language understanding.Advances in Neural Information Processing Systems, 35:36479–36494, 2022

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Pho- torealistic text-to-image diffusion models with deep language understanding.Advances in Neural Informatio...

    2022

  35. [43]

    Conditional image generation and manipulation for user-specified content.arXiv preprint arXiv:2005.04909,

    David Stap, Maurits Bleeker, Sarah Ibrahimi, and Maartje Ter Hoeve. Conditional image generation and manipulation for user-specified content.arXiv preprint arXiv:2005.04909,

    2005 arXiv

  36. [44]

    Multi-caption text-to-face synthesis: Dataset and algo- rithm

    Jianxin Sun, Qi Li, Weining Wang, Jian Zhao, and Zhenan Sun. Multi-caption text-to-face synthesis: Dataset and algo- rithm. InProceedings of the 29th ACM International Confer- ence on Multimedia, pages 2290–2298, 2021. 7, 8

    2021

  37. [45]

    Df-gan: Deep fusion generative adver- sarial networks for text-to-image synthesis.arXiv preprint arXiv:2008.05865, 2(6), 2020

    Ming Tao, Hao Tang, Songsong Wu, Nicu Sebe, Fei Wu, Xiao-Yuan Jing, et al. Df-gan: Deep fusion generative adver- sarial networks for text-to-image synthesis.arXiv preprint arXiv:2008.05865, 2(6), 2020. 8

    2008 arXiv

  38. [46]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017. 8

    2017

  39. [47]

    Faces a la carte: Text-to-face generation via attribute disentanglement

    Tianren Wang, Teng Zhang, and Brian Lovell. Faces a la carte: Text-to-face generation via attribute disentanglement. InPro- ceedings of the IEEE/CVF winter conference on applications of computer vision, pages 3380–3388, 2021. 8

    2021

  40. [48]

    High-fidelity 3d face genera- tion from natural language descriptions

    Menghua Wu, Hao Zhu, Linjia Huang, Yiyu Zhuang, Yuanxun Lu, and Xun Cao. High-fidelity 3d face genera- tion from natural language descriptions. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4521–4530, 2023. 7

    2023

  41. [49]

    Tedigan: Text-guided diverse face image generation and ma- nipulation

    Weihao Xia, Yujiu Yang, Jing-Hao Xue, and Baoyuan Wu. Tedigan: Text-guided diverse face image generation and ma- nipulation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2256–2265,

  42. [50]

    Omniavatar: Geometry-guided controllable 3d head syn- thesis

    Hongyi Xu, Guoxian Song, Zihang Jiang, Jianfeng Zhang, Yichun Shi, Jing Liu, Wanchun Ma, Jiashi Feng, and Linjie Luo. Omniavatar: Geometry-guided controllable 3d head syn- thesis. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 12814–12824,

  43. [51]

    Attngan: Fine- grained text to image generation with attentional generative adversarial networks

    Tao Xu, Pengchuan Zhang, Qiuyuan Huang, Han Zhang, Zhe Gan, Xiaolei Huang, and Xiaodong He. Attngan: Fine- grained text to image generation with attentional generative adversarial networks. InProceedings of the IEEE confer- ence on computer vision and pattern recognition, page...

    2018

  44. [52]

    Towards high-fidelity text-guided 3d face genera- tion and manipulation using only images

    Cuican Yu, Guansong Lu, Yihan Zeng, Jian Sun, Xiaodan Liang, Huibin Li, Zongben Xu, Songcen Xu, Wei Zhang, and Hang Xu. Towards high-fidelity text-guided 3d face genera- tion and manipulation using only images. InProceedings of the IEEE/CVF International Conference on Computer...

    2023

  45. [53]

    Scaling autoregressive mod- els for content-rich text-to-image generation.arXiv preprint arXiv:2206.10789, 2(3):5, 2022

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gun- jan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al. Scaling autoregressive mod- els for content-rich text-to-image generation.arXiv preprint arXiv:2206.10789, 2(3):5, 2022. 8

    2022 arXiv

  46. [54]

    Stack- gan: Text to photo-realistic image synthesis with stacked generative adversarial networks

    Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiao- gang Wang, Xiaolei Huang, and Dimitris N Metaxas. Stack- gan: Text to photo-realistic image synthesis with stacked generative adversarial networks. InProceedings of the IEEE international conference on computer vision, pag...

    2017

  47. [55]

    Stack- gan++: Realistic image synthesis with stacked generative adversarial networks.IEEE transactions on pattern analysis and machine intelligence, 41(8):1947–1962, 2018

    Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiao- gang Wang, Xiaolei Huang, and Dimitris N Metaxas. Stack- gan++: Realistic image synthesis with stacked generative adversarial networks.IEEE transactions on pattern analysis and machine intelligence, 41(8):1947–1962, 2018. 8

    1947

  48. [56]

    Dreamface: Progressive generation of animatable 3d faces under text guidance.arXiv preprint arXiv:2304.03117,

    Longwen Zhang, Qiwei Qiu, Hongyang Lin, Qixuan Zhang, Cheng Shi, Wei Yang, Ye Shi, Sibei Yang, Lan Xu, and Jingyi Yu. Dreamface: Progressive generation of animatable 3d faces under text guidance.arXiv preprint arXiv:2304.03117,

    arXiv

  49. [57]

    M6-ufc: Unifying multi-modal controls for conditional im- age synthesis via non-autoregressive generative transformers

    Zhu Zhang, Jianxin Ma, Chang Zhou, Rui Men, Zhikang Li, Ming Ding, Jie Tang, Jingren Zhou, and Hongxia Yang. M6-ufc: Unifying multi-modal controls for conditional im- age synthesis via non-autoregressive generative transformers. arXiv preprint arXiv:2105.14211, 2021. 8

    2021 arXiv

  50. [58]

    Diffgs: Functional gaussian splatting diffusion.arXiv preprint arXiv:2410.19657, 2024

    Junsheng Zhou, Weiqi Zhang, and Yu-Shen Liu. Diffgs: Functional gaussian splatting diffusion.arXiv preprint arXiv:2410.19657, 2024. 2

    2024 arXiv

  51. [59]

    Generative adversarial network for text-to-face synthesis and manipulation

    Yutong Zhou. Generative adversarial network for text-to-face synthesis and manipulation. InProceedings of the 29th ACM International Conference on Multimedia, pages 2940–2944,

  52. [60]

    Generative adversar- ial network for text-to-face synthesis and manipulation with pretrained bert model

    Yutong Zhou and Nobutaka Shimada. Generative adversar- ial network for text-to-face synthesis and manipulation with pretrained bert model. In2021 16th IEEE International Con- ference on Automatic Face and Gesture Recognition (FG 2021), pages 01–08, 2021. 7

    2021

  53. [61]

    blonde”, “blue eyes

    Minfeng Zhu, Pingbo Pan, Wei Chen, and Yi Yang. Dm- gan: Dynamic memory generative adversarial networks for text-to-image synthesis. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5802–5810, 2019. 8 Appendices A. Proof of the stochas...

    2019

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.