Pith. sign in

REVIEW 4 major objections 6 minor 73 references

Face-MoGLE claims that decoupling semantic masks into binary regions and routing them through global and local experts inside a diffusion transformer yields state-of-the-art controllable face generation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Face-MoGLE improves controllable face generation by feeding decoupled binary masks through global and local experts with time- and space-dependent gating in a diffusion transformer.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A novel mask-conditioned DiT face generator with strong results, but the central architecture claim lacks a same-backbone control; the gains over prior art may be mostly the FLUX backbone. the 4 major comments →

arxiv 2509.00428 v1 pith:CIFKALGH submitted 2025-08-30 cs.CV

Mixture of Global and Local Experts with Diffusion Transformer for Controllable Face Generation

Xuechao Zou , Shun Zhang , Xing Fu , Yue Li , Kai Li , Yushe Cao , Congyan Lang , Pin Tao
show 1 more author
Junliang Xing
This is my paper
classification cs.CV
keywords controllable face generationdiffusion transformermixture of expertssemantic mask decouplingdynamic gatingmultimodal face generationzero-shot generalizationlatent diffusion
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Face-MoGLE sets out to show that a diffusion transformer can be made controllable for faces by treating a semantic mask as a stack of binary region masks rather than one raw condition. The paper argues that global experts (whole-face layout) plus local experts (region details), blended by a gating network whose weights vary across both diffusion timestep and spatial position, can beat U-Net and GAN baselines on fidelity, text alignment, and mask alignment. It reports the best FID, KID, CMMD, mask alignment, and top text alignment for multimodal generation on MM-CelebA-HQ, plus state-of-the-art zero-shot results on an extended FFHQ-Text set. If true, the result matters because it points to a reusable recipe for fine-grained semantic control in DiT-based generators without retraining the base model.

Core claim

The paper claims that controllable face generation is best served by decoupling semantic masks into binary region masks, encoding each through a shared frozen VAE, and recombining the region tokens with a diffusion-aware mixture of global and local experts inside a DiT backbone. The global expert preserves whole-face layout coherence, local experts refine region-specific detail, and a dynamic gating network assigns spatially varying, time-dependent weights so the influence of each region shifts as denoising progresses. Trained with LoRA on a frozen FLUX.1-dev backbone, the resulting Face-MoGLE model reports the best FID/KID/CMMD and mask alignment on MM-CelebA-HQ multimodal generation while

What carries the argument

The central object is the MoGLE mask-encoding module. Each semantic region is converted to a binary mask, passed through one shared frozen VAE encoder to produce region tokens; a global expert sees the full mask while local experts see individual regions; a gating network consumes the noisy image tokens, the timestep embedding, and the global mask token to output spatial weight maps that sum to one; the final mask condition is the weighted sum of expert outputs (Eq. 7-8). This module carries the argument because it replaces raw mask concatenation with a learned, time- and position-adaptive recombination of decoupled region semantics.

Load-bearing premise

The load-bearing premise is that a frozen VAE encoder applied separately to each binary mask preserves the mask's region boundaries and class semantics well enough for learned gating weights to recombine them into a faithful control signal; if those latent tokens lose spatial structure, the condition path degrades.

What would settle it

Erase one semantic region, say the nose, from the input mask at inference and compare the output with the full-mask output while monitoring that region's local-expert gating weight; if the weight does not drop locally and the generated nose is not visibly altered, the claim of region-specific local-expert control is falsified.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • The same trained model can switch between multimodal, mask-only, and text-only generation because text and mask conditions are independently dropped during training; no retraining or architectural change is needed.
  • Because the gating weights are time- and position-dependent, the denoising process can emphasize global layout early and local detail later, so control and fidelity do not have to trade off as sharply as under static fusion.
  • The reported zero-shot gains on MM-FFHQ-Female imply the mask-decoupling and expert-routing scheme generalizes to unseen data and unseen semantic combinations without task-specific fine-tuning.
  • Ablations indicate that neither global nor local experts alone reach the combined model's quality; the shared VAE encoding of binary masks plus dynamic gating is what the reported improvements rest on.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: The binary-mask/expert pattern is not face-specific; decoupling a structured condition into atomic channels, encoding each with a shared frozen encoder, and routing through global/local experts could apply to layout, pose, or part-segmentation control in other DiT generators, but the paper demonstrates this only for faces.
  • Editorial extension: The time-dependent gating weights invite a diagnostic use; probing when each region's expert is most active could reveal the denoising stage at which mask errors are baked in, which could guide better masks or schedule-aware regularization.
  • Editorial extension: The report of near-random deepfake-detection AUC, together with the appendix's suggestion to train forgery detectors on generated faces, points to a concrete follow-up: whether detector generalization actually improves with Face-MoGLE images is promising but not established in this paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Face-MoGLE, a controllable face generation framework built on the FLUX.1-dev diffusion transformer. Semantic masks are decomposed into binary per-region masks, each encoded by a shared frozen VAE encoder, then processed by global and local experts whose outputs are fused by a spatiotemporal gating network conditioned on timestep and noisy latent. The model is trained with LoRA on MM-CelebA-HQ and evaluated for multimodal, mask-to-face, and text-to-face generation, plus zero-shot generalization on MM-FFHQ-Female. The paper reports state-of-the-art FID, KID, CMMD, mask alignment, and text alignment on MM-CelebA-HQ and strong zero-shot results on MM-FFHQ-Female.

Significance. If the architectural claims hold, Face-MoGLE would be a meaningful step toward using modern DiT backbones for fine-grained, multimodal face generation. The design is modular, uses efficient LoRA fine-tuning, and addresses an underexplored problem: injecting structured semantic masks into a large pretrained DiT. The paper also provides extensive ablations of the expert and gating components and releases a dataset extension. However, the main empirical claim is currently weakened by the absence of same-backbone control baselines: all SOTA comparisons use older GAN/U-Net/small-transformer systems, so the reported gains cannot be attributed to MoGLE rather than to the FLUX pretrained prior.

major comments (4)
  1. [§4.4, Tables 1–2] The SOTA comparisons are confounded by backbone choice. Face-MoGLE LoRA-fine-tunes FLUX.1-dev, a 12B-parameter DiT pretrained on internet-scale data, while TediGAN, Collaborative, UaC, PixelFace+, and DDGI use older GANs, U-Nets, or smaller transformers. The 20+ point FID improvement on MM-CelebA-HQ and 27% FID reduction on MM-FFHQ-Female may reflect the base model’s prior rather than the proposed gating/expert architecture. Please add same-backbone controls: FLUX+LoRA with the mask injected through a standard mechanism (e.g., cross-attention adapter, ControlNet-style conditioning, or simple concatenation) under identical training data, steps, and seed. The '× ×' row of Table 5 could serve this role, but its mask-conditioning mechanism is not described. Without such a control, the central claim of architectural superiority is not established.
  2. [§3.2, Eq. (5)–(8); Table 5] The benefit of mask decoupling into binary components is not isolated. The ablations compare 'Only Global', 'Only Local', and 'Global & Local', but not a full-mask conditioning baseline that feeds the entire semantic mask through the same VAE encoder and the same expert/gating machinery without binary decomposition. The observed gain of the full model over 'Only Global' and 'Only Local' could come from increased condition capacity or token multiplicity rather than from semantic decoupling per se. The assumption that VAE-encoded binary masks are linearly combinable via learned gating weights is also untested. Please include a baseline that encodes the full mask as a single token sequence and applies the same gating/expert fusion, and visualize or quantitatively verify that the VAE latent preserves the binary mask’s class/spatial structure.
  3. [§4.3, Table 1, Table 2] The mask-consistency metric is not sufficient to support the controllability claim. 'Mask↓' is DINO Structure Distance, a self-similarity measure that does not directly assess whether the generated image respects the semantic class of each input mask region. The reported improvements (e.g., 2.44 vs. 3.22) may not reflect true semantic alignment. Please report a pixel-level mask-consistency metric such as mIoU or pixel accuracy between the input mask and the segmentation of the generated image. In addition, all FID/KID numbers come from a single fixed seed; on the 760-image MM-FFHQ-Female set FID is noisy. Report multiple seeds or bootstrap confidence intervals.
  4. [§3.1–3.2, Eq. (3), (7), (8)] The architecture is underspecified in ways that block reproduction. Eq. (3) lists C_m as an input to the denoiser, but the text never states how C_m enters each DiT block: cross-attention, concat with image tokens, or an additive token stream? Eq. (7) says the gating network outputs 'spatial weight maps', but their tensor shape, how softmax is applied over the n+1 experts at each spatial location, and how the maps align with the L×d token grid are not defined. The expert modules Expert_i are also not specified (MLP? attention? shared or separate parameters?). Please provide a precise forward-pass description with tensor dimensions.
minor comments (6)
  1. [§3.1, Eq. (1)] M is declared as R^{H×W×3}, which is unusual for a semantic mask. If it is a color-coded RGB rendering of labels, state that; if it is a class-index map, the dimensions should be H×W×C or H×W×1.
  2. [§3.1, text] The text encoder is said to jointly use 'CLIP and H5'; H5 is likely a typo for T5. Please correct and cite the exact T5 variant used (e.g., FLUX.1-dev’s T5-XXL).
  3. [§4.5.2, Table 4] The row labeled 'w/o Diffusion' is described in the text as 'static weights'; the table label is confusing. Rename to 'Static Gating' or similar.
  4. [Table 1, DDGI row] DDGI has missing entries for KID, CMMD, Mask, and Text. If results are copied from the original paper, state explicitly which metrics are unavailable and avoid citing an incomplete row as a comparison.
  5. [References] Reference [26] cites a GitHub repository without a version/commit identifier. Since FLUX is central to the method, please cite the relevant arXiv/technical report and specify the exact checkpoint and revision.
  6. [Figure 2] Figure 2 is captioned 'Diffusion Transformer Block' but does not show how the mask tokens, text tokens, or timestep interact with the block. A more detailed diagram would improve clarity.

Circularity Check

0 steps flagged

No significant circularity: Face-MoGLE is an empirical architecture paper whose claims rest on held-out evaluation, not on derivation-from-inputs.

full rationale

Face-MoGLE does not derive a prediction from first principles; it trains a diffusion transformer with a standard noise-prediction loss (Eq. 4) and reports metrics on held-out data. The gating network and experts are learned modules; their weights are optimized by the MSE objective, not fitted to the reported FID/KID/CMMD/mask/text metrics. The zero-shot evaluation on MM-FFHQ-Female uses a dataset not seen in training, and the ablation tables compare independently trained variants. No equation in the paper defines a predicted quantity in terms of the quantity it is supposed to validate, and no parameter is renamed as a prediction. Citations to FLUX, LoRA, and prior baselines are external works and are not used as a self-citation chain to justify the central architectural claim. Potential concerns about backbone mismatch with baselines are experimental confounds, not circularity. Therefore the paper is self-contained with respect to circularity.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

The paper's claims rest on empirical network learning, not derived invariants. There are no new physical entities; the only introduced components are learned neural modules. The listed axioms are load-bearing premises that are standard in deep learning but not formally guaranteed.

free parameters (2)
  • Mask annotation threshold (OA=0.8) = 0.8
    Masks below this accuracy are manually annotated; the threshold is chosen by hand and affects the quality of the zero-shot test set.
  • FaRL/SegFace sampling ratio = 90% FaRL / 10% SegFace
    Randomized sampling strategy for masks with OA>=0.8; this arbitrary mixture could bias the zero-shot evaluation.
axioms (4)
  • domain assumption Pretrained FLUX.1-dev and its VAE are reliable base models for high-fidelity face synthesis.
    The entire pipeline inherits the generative quality of FLUX; if the base model were weak, the reported FID numbers would change. This is an input assumption.
  • domain assumption Semantic masks provide a sufficient and accurate spatial control signal for face structure.
    The method relies on pixel-level masks (from CelebAMask-HQ or FaRL/SegFace parsing) as the spatial condition; errors in masks propagate to generation. Section 4.1.
  • ad hoc to paper Binary masks, when passed through the shared frozen VAE encoder, yield tokens that are linearly combinable with each other and with the image latent tokens.
    There is no theoretical guarantee that the VAE latent of a binary mask aligns with the image latent; the paper only demonstrates this empirically. Section 3.2, Eq. 5-8.
  • domain assumption The gating network can learn to produce appropriate spatial-temporal weights from noisy latents and the global mask token.
    Standard learnability assumption for neural networks; no theoretical convergence guarantee. Section 3.2, Eq. 7.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Mixture of Global and Local Experts with Diffusion Transformer for Controllable Face Generation." pith.science (2026). https://pith.science/paper/CIFKALGH

@misc{pith2026250900428,
  author       = {Pith},
  title        = {Pith review of: Mixture of Global and Local Experts with Diffusion Transformer for Controllable Face Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CIFKALGH}},
  note         = {Machine review of arXiv:2509.00428}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Controllable face generation poses critical challenges in generative modeling due to the intricate balance required between semantic controllability and photorealism. While existing approaches struggle with disentangling semantic controls from generation pipelines, we revisit the architectural potential of Diffusion Transformers (DiTs) through the lens of expert specialization. This paper introduces Face-MoGLE, a novel framework featuring: (1) Semantic-decoupled latent modeling through mask-conditioned space factorization, enabling precise attribute manipulation; (2) A mixture of global and local experts that captures holistic structure and region-level semantics for fine-grained controllability; (3) A dynamic gating network producing time-dependent coefficients that evolve with diffusion steps and spatial locations. Face-MoGLE provides a powerful and flexible solution for high-quality, controllable face generation, with strong potential in generative modeling and security applications. Extensive experiments demonstrate its effectiveness in multimodal and monomodal face generation settings and its robust zero-shot generalization capability. Project page is available at https://github.com/XavierJiezou/Face-MoGLE.

Figures

Figures reproduced from arXiv: 2509.00428 by Congyan Lang, Junliang Xing, Kai Li, Pin Tao, Shun Zhang, Xing Fu, Xuechao Zou, Yue Li, Yushe Cao.

Figure 1
Figure 1. Figure 1: Mixture of Global and Local Experts with Diffusion Transformer (Face-MoGLE) is a unified and flexible framework for high-quality and controllable face generation. It supports text-to-face synthesis (left), mask-to-face synthesis (right), and multimodal face generation guided jointly by text and masks (middle). By harmonizing global context modeling with local detail refinement, Face-MoGLE produces highly p… view at source ↗
Figure 2
Figure 2. Figure 2: Training pipeline of the diffusion transformer. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Architecture of the Mixture of Global and Local Experts (MoGLE) designed for semantic mask embedding. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Structure of our dynamic gating network. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Visualization of global and partial local weight map. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Visualization results from different methods. The figure compares three generation paradigms: multimodal face [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Comparative results of multimodal face generation on the MM-CelebA-HQ dataset. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Comparative results of mask-to-face generation on the MM-CelebA-HQ dataset. [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Comparative results of text-to-face generation on the MM-CelebA-HQ dataset. [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Comparative results of zero-shot generalization on the MM-FFHQ-Female dataset. [PITH_FULL_IMAGE:figures/full_fig_p014_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Comparative results of ablation studies on the MM-CelebA-HQ dataset. [PITH_FULL_IMAGE:figures/full_fig_p014_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

73 extracted references · 63 canonical work pages

  1. [1]

    Lalith Bharadwaj Baru, Rohit Boddeda, Shilhora Akshay Patel, and Sai Mohan Gajapaka. 2025. Wavelet-Driven Generalizable Framework for Deepfake Face Forgery Detection. In W ACV. 1661–1669

    work page 2025

  2. [2]

    Sutherland, Michael Arbel, and Arthur Gretton

    Mikołaj Bińkowski, Dougal J. Sutherland, Michael Arbel, and Arthur Gretton

  3. [3]

    Bocheng, YuhangMa, wuliebucha, Shanyuan Liu, Ao Ma, Xiaoyu Wu, Dawei Leng, and Yuhui Yin. 2024. HiCo: Hierarchical Controllable Diffusion Model for Layout-to-image Generation. In NeurIPS

    work page 2024

  4. [4]

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. 2023. Instructpix2pix: Learning to follow image editing instructions. In CVPR. 18392–18402

    work page 2023

  5. [5]

    Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. 2021. Emerging Properties in Self-Supervised Vision Transformers. In ICCV. 9650–9660

    work page 2021

  6. [6]

    Prafulla Dhariwal and Alexander Nichol. 2021. Diffusion models beat gans on image synthesis. In NeurIPS, Vol. 34. 8780–8794

    work page 2021

  7. [7]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. InICLR. 1–21

    work page 2021

  8. [8]

    Xiaoxiong Du, Jun Peng, Yiyi Zhou, Jinlu Zhang, Siting Chen, Guannan Jiang, Xiaoshuai Sun, and Rongrong Ji. 2023. PixelFace+: Towards Controllable Face Generation and Manipulation with Text Descriptions and Segmentation Masks. In ACM MM. 4666–4677

    work page 2023

  9. [9]

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, and Robin Rombach. 2024. Scaling rectified flow transformers for high-resolution image synthesis. In ICML. 28 pages

    work page 2024

  10. [10]

    William Fedus, Barret Zoph, and Noam Shazeer. 2022. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. JMLR 23, 120 (2022), 1–39

    work page 2022

  11. [11]

    Erik Härkönen, Aaron Hertzmann, Jaakko Lehtinen, and Sylvain Paris. 2020. Ganspace: Discovering interpretable gan controls. InNeurIPS, Vol. 33. 9841–9850

    work page 2020

  12. [12]

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. GANs trained by a two time-scale update rule converge to a local nash equilibrium. In NeurIPS. 6629–6640

    work page 2017

  13. [13]

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. In NeurIPS, Vol. 33. 6840–6851

    work page 2020

  14. [14]

    Jonathan Ho and Tim Salimans. 2021. Classifier-Free Diffusion Guidance. In NeurIPS. 1–8

    work page 2021

  15. [15]

    Fangzhou Hong, Mingyuan Zhang, Liang Pan, Zhongang Cai, Lei Yang, and Ziwei Liu. 2022. AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars. TOG 41, 4, Article 161 (2022), 19 pages

    work page 2022

  16. [16]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In ICLR. 1–13

    work page 2022

  17. [17]

    Ziqi Huang, Kelvin CK Chan, Yuming Jiang, and Ziwei Liu. 2023. Collaborative diffusion for multi-modal face generation and editing. In CVPR. 6080–6090

    work page 2023

  18. [18]

    Robert A Jacobs, Michael I Jordan, Steven J Nowlan, and Geoffrey E Hinton. 1991. Adaptive mixtures of local experts. Neural computation 3, 1 (1991), 79–87

    1991

  19. [19]

    Sadeep Jayasumana, Srikumar Ramalingam, Andreas Veit, Daniel Glasner, Ayan Chakrabarti, and Sanjiv Kumar. 2024. Rethinking FID: Towards a Better Evalua- tion Metric for Image Generation . In CVPR. 9307–9315

    work page 2024

  20. [20]

    Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2018. Progressive Growing of GANs for Improved Quality, Stability, and Variation. InICLR

    work page 2018

  21. [21]

    Tero Karras, Samuli Laine, and Timo Aila. 2019. A style-based generator archi- tecture for generative adversarial networks. In CVPR. 4401–4410

    work page 2019

  22. [22]

    Tero Karras, Samuli Laine, and Timo Aila. 2019. A Style-Based Generator Archi- tecture for Generative Adversarial Networks. In CVPR. 4401–4410

    work page 2019

  23. [23]

    Jihyun Kim, Changjae Oh, Hoseok Do, Soohyun Kim, and Kwanghoon Sohn

    work page

  24. [24]

    Durk P Kingma and Prafulla Dhariwal. 2018. Glow: Generative flow with invert- ible 1x1 convolutions. In NeurIPS, Vol. 31. 1–10

    work page 2018

  25. [25]

    Diederik P Kingma and Max Welling. 2014. Auto-encoding variational{Bayes}. In ICLR. 1–14

    work page 2014

  26. [26]

    Black Forest Labs. 2024. FLUX. https://github.com/black-forest-labs/flux

    2024

  27. [27]

    Cheng-Han Lee, Ziwei Liu, Lingyun Wu, and Ping Luo. 2020. Maskgan: Towards diverse and interactive facial image manipulation. In CVPR. 5549–5558

    work page 2020

  28. [28]

    Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. 2021. {GS}hard: Scaling Giant Models with Conditional Computation and Automatic Sharding. In ICLR. 1–23

    work page 2021

  29. [29]

    Fang Liu, Xiaoming Deng, Yu-Kun Lai, Yong-Jin Liu, Cuixia Ma, and Hongan Wang. 2019. SketchGAN: Joint Sketch Completion and Recognition With Gener- ative Adversarial Network. In CVPR

    work page 2019

  30. [30]

    Debin Meng, Christos Tzelepis, Ioannis Patras, and Georgios Tzimiropoulos

    work page

  31. [31]

    Konstantin Mishchenko and Aaron Defazio. 2024. Prodigy: An Expeditiously Adaptive Parameter-Free Learner. In ICML. 35779–35804

    work page 2024

  32. [32]

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. 2024. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In AAAI, Vol. 38. 4296–4304

    work page 2024

  33. [33]

    MM2Latent: Text-to-facial image generation and editing in GANs with multimodal assistance. In ECCV. 1–20

    work page

  34. [34]

    Kartik Narayan, Vibashan VS, and Vishal M Patel. 2024. Segface: Face segmenta- tion of long-tail classes. arXiv preprint arXiv:2412.08647 (2024)

    Pith/arXiv arXiv 2024

  35. [35]

    Alexander Quinn Nichol and Prafulla Dhariwal. 2021. Improved denoising diffu- sion probabilistic models. In ICML. 8162–8171

    work page 2021

  36. [36]

    Nithin Gopalakrishnan Nair, Wele Gedara Chaminda Bandara, and Vishal M Patel

    work page

  37. [37]

    Or Patashnik, Zongze Wu, Eli Shechtman, Daniel Cohen-Or, and Dani Lischinski

    work page

  38. [38]

    William Peebles and Saining Xie. 2023. Scalable diffusion models with transform- ers. In ICCV. 4195–4205

    work page 2023

  39. [39]

    Justin N. M. Pinkney and Chuan Li. 2022. clip2latent: Text driven sampling of a pre-trained StyleGAN using denoising diffusion and CLIP. In BMVC. 1–12

    work page 2022

  40. [40]

    Minho Park, Jooyeol Yun, Seunghwan Choi, and Jaegul Choo. 2023. Learning to generate semantic layouts for higher text-image correspondence in text-to-image synthesis. In ICCV. 7591–7600

    work page 2023

  41. [41]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In ICML, Vol. 139. 8748–8763

    work page 2021

  42. [42]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.JMLR 21, 140 (2020), 1–67

    work page 2020

  43. [43]

    Carlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, André Susano Pinto, Daniel Keysers, and Neil Houlsby. 2021. Scaling vision with sparse mixture of experts. In NeurIPS, Vol. 34. 8583–8595

    work page 2021

  44. [44]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-Resolution Image Synthesis With Latent Diffusion Models. In CVPR. 10684–10695

    work page 2022

  45. [45]

    Namboodiri, and C.V

    K R Prajwal, Rudrabha Mukhopadhyay, Vinay P. Namboodiri, and C.V. Jawahar

  46. [46]

    Noam Shazeer, *Azalia Mirhoseini, *Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. 2017. Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer. In ICLR. 1–19

    work page 2017

  47. [47]

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli

  48. [48]

    Jiaming Song, Chenlin Meng, and Stefano Ermon. 2021. Denoising Diffusion Implicit Models. In ICLR. 1–20

    work page 2021

  49. [49]

    Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, Ping Liu, and Yunchao Wei. 2024. Rethinking the up-sampling operations in cnn-based generative network for generalizable deepfake detection. In CVPR. 28130–28139

    work page 2024

  50. [50]

    Zhentao Tan, Menglei Chai, Dongdong Chen, Jing Liao, Qi Chu, Bin Liu, Gang Hua, and Nenghai Yu. 2021. Diverse semantic image synthesis via probability distribution modeling. In CVPR. 7962–7971

    work page 2021

  51. [51]

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In MICCAI. 234–241

    work page 2015

  52. [52]

    Torr, and Nicu Sebe

    Hao Tang, Dan Xu, Yan Yan, Philip H.S. Torr, and Nicu Sebe. 2020. Local Class- Specific and Global Image-Level Generative Adversarial Networks for Semantic- Guided Scene Generation. In CVPR. 7870–7879

    work page 2020

  53. [53]

    Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. 2017. Neural discrete representation learning. In NeurIPS. 6309–6318

    work page 2017

  54. [54]

    Chaoyang Wang, Xiangtai Li, Lu Qi, Henghui Ding, Yunhai Tong, and Ming- Hsuan Yang. 2024. Semflow: Binding semantic segmentation and image synthesis via rectified flow. In NeurIPS, Vol. 37. 138981–139001

    work page 2024

  55. [55]

    Tianyi Wei, Dongdong Chen, Wenbo Zhou, Jing Liao, Weiming Zhang, Lu Yuan, Gang Hua, and Nenghai Yu. 2022. E2Style: Improve the efficiency and effective- ness of StyleGAN inversion. TIP 31 (2022), 3267–3280

    work page 2022

  56. [56]

    Weihao Xia, Yujiu Yang, Jing-Hao Xue, and Baoyuan Wu. 2021. Tedigan: Text- guided diverse face image generation and manipulation. In CVPR. 2256–2265

    work page 2021

  57. [57]

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. 2023. ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation. In NeurIPS, Vol. 36. 15903–15935. arXiv Preprint, arXiv, 2025 Zou et al

    work page 2023

  58. [58]

    Zhenxiong Tan, Songhua Liu, Xingyi Yang, Qiaochu Xue, and Xinchao Wang

  59. [59]

    arXiv preprint arXiv:2411.15098 3 (2024)

    Ominicontrol: Minimal and universal control for diffusion transformer. arXiv preprint arXiv:2411.15098 3 (2024)

    Pith/arXiv arXiv 2024

  60. [60]

    Jinlu Zhang, Yiyi Zhou, Qiancheng Zheng, Xiaoxiong Du, Gen Luo, Jun Peng, Xiaoshuai Sun, and Rongrong Ji. 2024. Fast text-to-3D-aware face generation and manipulation via direct cross-modal mapping and geometric regularization. In ICML. 60605–60625

    work page 2024

  61. [61]

    Kai Zhang, Yawei Li, Wangmeng Zuo, Lei Zhang, Luc Van Gool, and Radu Timofte

    work page

  62. [62]

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. 2023. Adding conditional control to text-to-image diffusion models. In ICCV. 3836–3847

    2023

  63. [63]

    Yinglin Zheng, Hao Yang, Ting Zhang, Jianmin Bao, Dongdong Chen, Yangyu Huang, Lu Yuan, Dong Chen, Ming Zeng, and Fang Wen. 2022. General facial representation learning in a visual-linguistic manner. In CVPR. 18697–18709

    work page 2022

  64. [64]

    Yutong Zhou. 2021. Generative adversarial network for text-to-face synthesis and manipulation. In ACM MM. 2940–2944. A Societal Impacts and Responsible AI Our research focuses on controllable face generation, based on a diffusion transformer architecture combined with a mixture of global and local experts, aiming to support a variety of optimistic applica...

    work page 2021

  65. [66]

    Sicheng Xu, Guojun Chen, Yu-Xiao Guo, Jiaolong Yang, Chong Li, Zhenyu Zang, Yizhong Zhang, Xin Tong, and Baining Guo. 2024. Vasa-1: Lifelike audio-driven talking faces generated in real time. In NeurIPS, Vol. 37. 660–684

    work page 2024

  66. [67]

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. 2023. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.06721 (2023)

    Pith/arXiv arXiv 2023

  67. [2015]

    Deep unsupervised learning using nonequilibrium thermodynamics. In ICML. 2256–2265

    work page

  68. [2018]

    Demystifying MMD GANs. In ICLR. 1–36

    work page

  69. [2020]

    In ACM MM

    A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild. In ACM MM. 484–492

    work page

  70. [2021]

    Styleclip: Text-driven manipulation of stylegan imagery. In ICCV. 2085– 2094

    work page 2085

  71. [2022]

    TPAMI 44, 10 (2022), 6360–6376

    Plug-and-Play Image Restoration With Deep Denoiser Prior. TPAMI 44, 10 (2022), 6360–6376

    work page 2022

  72. [2023]

    Unite and conquer: Plug & play multi-modal synthesis using diffusion models. In CVPR. 6070–6079

    work page

  73. [2024]

    Diffusion-driven gan inversion for multi-modal face image generation. In CVPR. 10403–10412

    work page

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.