Pith. sign in

REVIEW 5 major objections 6 minor 55 references

Omni-ID: Holistic Identity Representation Designed for Generative Tasks

T0 review · 5 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Omni-ID claims that a generative, multi-photo face representation preserves identity in synthesized images far better than ArcFace or CLIP.

desk verdict A useful, well-ablated identity representation for face generation, with a quantitative case that is weaker than the abstract suggests due to single-matcher metrics and no uncertainty estimates. read the letter →

arxiv 2412.09694 v2 pith:OUH46UYE submitted 2024-12-12 cs.CV

Guocheng Qian , Kuan-Chieh Wang , Or Patashnik , Negin Heravi , Daniil Ostashev , Sergey Tulyakov , Daniel Cohen-Or , Kfir Aberman This is my paper · ORCID
classification cs.CV
keywords identityrepresentationfacegenerationfew-to-manyreconstructionmaskedtransformerdecoderflowmatchingmulti-viewdatasetpersonalizedtext-to-imagecontrollablesynthesis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Omni-ID asks how a face should be encoded when the goal is to generate images, not to recognize people. The paper claims that a fixed-size, set-based representation trained by reconstructing many images of one identity from a few inputs captures far more of what makes someone look like themselves across poses and expressions than the standard discriminative embeddings ArcFace and CLIP, when both are injected into the same frozen generator. If true, identity preservation in controllable face synthesis and personalized text-to-image generation would scale with the number of input photos, and generative objectives would be the right way to learn identity features for generative applications.

What carries the argument

The Omni-ID encoder is a transformer with learnable identity queries: input images are tokenized by a finetuned CLIP-H backbone, concatenated along the token axis, and passed as keys and values through cross-attention layers followed by self-attention, producing a fixed-size representation $\ell \in \mathbb{R}^{256 \times 1280}$ from any number of photos. During training, this representation is decoded by two complementary objectives: a Masked Transformer Decoder that reconstructs heavily masked target face images (95 percent mask) from $\ell$ plus a few visible patches, and a conditional Flow-Matching decoder built on FLUX with IP-Adapter injection. The few-to-many regime, which always reconstructs more target images than input images from the same identity's multi-view set, is what forces the representation to be holistic rather than a single-image embedding, with each learned token attending to particular global or local identity features.

What would settle it

A concrete falsifier: rerun the controllable-face benchmark and the ten-identity, twenty-prompt personalization evaluation with a blind two-alternative human likeness test in addition to FaceNet similarity, on a sample stratified by age, skin tone, and pose; if human preference does not track the reported FaceNet margins, or if the margins shrink when a different face-recognition model is used as the metric, the central claim is not established.

Watch

Extended reading notes

Core claim

The paper's central claim is that a face representation learned by generative reconstruction, rather than by recognition or contrastive objectives, carries more of the information that image generators need to preserve a specific person's likeness. In the paper's controlled comparison, with the same IP-Adapter/ControlNet pipeline on frozen FLUX and ArcFace, CLIP, and ArcFace+CLIP as alternatives, Omni-ID reports the highest identity similarity at every input count (for example, 0.742 versus 0.535, 0.682, and 0.664 on MFHQ with seven inputs) and improves as more photos are supplied. In personalized text-to-image generation with a single adapter trained on one million images, the Omni-ID adapter scores 0.8026 average identity similarity against 0.7289 for PuLID and 0.6649 for IP-Adapter-Full, without LoRA or identity losses. The paper attributes this to a few-to-many identity reconstruction objective that forces the encoder to keep attributes visible across multiple views, and to combining a masked autoencoding decoder with a flow-matching decoder to preserve both breadth and fine detail.

Load-bearing premise

The argument's load-bearing premise is that FaceNet similarity between generated and ground-truth images is a valid measure of identity preservation; the paper reports point estimates without confidence intervals or human evaluation, so a biased or noisy metric could change the outcome.

Editorial extensions

If this is right

  • Identity preservation in controllable face generation improves with each additional input photo, so multi-photo pipelines gain more than single-photo pipelines.
  • The same frozen Omni-ID representation can be plugged into different base generators, including FLUX dev, FLUX Schnell, and Stable Diffusion, and improves over single-image embeddings in all of them.
  • Training identity representations with a generative reconstruction objective yields features that keep fine-grained attributes such as beards, nose shape, and skin detail that discriminative models tend to discard.
  • The multi-decoder design is needed for both breadth and detail: removing the masked transformer decoder drops identity similarity from 0.708/0.728 to 0.468/0.473, and removing the flow-matching decoder loses fine detail, so neither objective alone carries the result.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the few-to-many reconstruction objective is not face-specific, so the same set-based encoder and dual-decoder training could be tested on other multi-view instance categories such as pets or branded products.
  • Editorial extension: because the encoder's queries are semantic and attend to different facial regions, the representation may support token-level identity editing or partial attribute transfer between identities, a capability the paper visualizes but does not evaluate.
  • Editorial extension: the paper compares against discriminative embeddings rather than another generative-pretrained representation at the same token budget; such a comparison would isolate whether the gain comes from the generative objective or primarily from aggregating multiple input views.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes Omni-ID, a fixed-size, set-based facial identity representation learned with a few-to-many identity reconstruction objective. The encoder aggregates a variable number of input face images into a structured token representation via cross-attention with learnable queries. Training uses two decoders: a Masked Transformer Decoder (MTD) and a conditional Flow-Matching decoder based on FLUX. The authors introduce a new dataset MFHQ with 134k identities, each with 8 images sampled for pose diversity. The representation is evaluated in controllable face generation and personalized text-to-image generation, compared against ArcFace, CLIP, and their combination, and against recent personalization methods (PuLID, PhotoMaker, InstantID). The main quantitative claims are based on FaceNet cosine similarity (ID similarity) and pose error. The paper reports consistent improvements, e.g., 0.708 vs 0.648 for CLIP on MFHQ and 0.774 vs 0.695 for CLIP on Webface in the controllable generation task, and 0.8026 vs 0.7289 for PuLID in the T2I task. Ablations validate the few-to-many objective, the two decoders, and the MFHQ dataset.

Significance. If the reported results hold, Omni-ID represents a valuable step toward identity representations specifically designed for generative models. The paper's strengths include a coherent and well-ablated training framework, a new large-scale high-quality multi-view face dataset (MFHQ), and external validation on Webface and against state-of-the-art personalization methods. The observation that identity preservation improves with the number of input images is notable and well supported by the ablations. However, the central evaluation is limited by the absence of statistical uncertainty measures and by the use of a private test set, and the causal claim about generative versus discriminative objectives is not fully isolated from the effect of fine-tuning the backbone on the target distribution. These issues should be addressed before the results can be taken as conclusively supporting the abstract's strong claims.

major comments (5)
  1. [Table 1 and Section 4.2] The headline quantitative claim rests entirely on point estimates of FaceNet ID similarity without any error bars, confidence intervals, or significance tests. Generative models are stochastic, and the differences between methods (e.g., 0.708 vs 0.648 for CLIP on MFHQ) may be within run-to-run variance. The authors should report per-identity mean and standard deviation (or bootstrap confidence intervals) for both test sets, and ideally perform a paired statistical test across identities. This is needed to support the claim of 'substantial improvements.'
  2. [Sections 3.4 and 4.2] The primary test column in Table 1 is from MFHQ, the same data distribution used to train Omni-ID (albeit held-out videos). Because the representation and the downstream IP-Adapter are optimized to reconstruct MFHQ-style targets with the same pose/expression sampling, the MFHQ test may favor Omni-ID over off-the-shelf representations. The Webface column is a useful external check, but it is reported only as point estimates. To rule out distribution bias, the authors should provide per-identity results on Webface with uncertainty, and ideally evaluate on an additional independent high-quality multi-view benchmark not used in training.
  3. [Sections 3.1 and 4.1] The comparisons are between Omni-ID (whose CLIP-H backbone is fine-tuned on MFHQ) and frozen off-the-shelf CLIP and ArcFace encoders. Since Omni-ID has benefited from additional training on the target task distribution, the observed gains cannot be attributed solely to the generative objective. To substantiate the abstract's claim that a generative objective outperforms discriminative/contrastive objectives, the paper should include a baseline in which the same transformer-encoder architecture is trained with a discriminative objective (e.g., ArcFace-style classification or a contrastive loss) on the same MFHQ data. Without such a baseline, the central mechanism remains confounded.
  4. [Section 4.3 and Table I] The quantitative evaluation of personalized T2I generation uses only 10 identities and 20 prompts. The standard deviations reported in Table I are across identities, but with 10 identities this does not provide reliable significance evidence. The main text presents only qualitative figures; the quantitative table in the appendix should be moved to the main text with a larger evaluation set (at least 30–50 identities) and a description of how prompts and identities were selected, or the authors should clearly state this as a pilot study and temper the claim of 'significant' improvement.
  5. [Section 3.4] The MFHQ dataset and its test split are not released, and the paper does not state whether they will be made available. Since the method is trained and evaluated on this private dataset, external researchers cannot reproduce the results or compare on the same benchmark. The authors should release the dataset (or a substantial public subset) and the exact evaluation protocol, or provide an alternative public benchmark with the same evaluation. This is important for the paper's impact and for verifying the robustness of the reported gains.
minor comments (6)
  1. [Table 1, Webface columns] The ArcFace row lists five values (0.379/0.373/0.370/0.371/0.373) for four input counts (3/5/8/16); either a value is extraneous or an input-count column is missing.
  2. [Section 4.1 and Table I] The metric description says ID similarity is computed with FaceNet features [12], but Table I in the appendix defines it as 'cosine distance between the generated samples and the five images of each identity' without specifying the model; please reconcile and state the matcher.
  3. [Section 4.3] The statement 'we evaluate them on 10 identities and 20 diverse prompts' lacks detail on how the identities and prompts were chosen; a list or a reference to an appendix with the prompts would improve reproducibility.
  4. [Equation (4)] The face segmentation mask M^r is not defined; please state how it is obtained (e.g., precomputed segmentation model) and whether it is applied during both training and evaluation.
  5. [Section 4.4] The phrase 'as aforementioned' is used to refer to 'additional inputs yield only marginal performance improvements' but this was not explicitly quantified in the main text; consider adding a curve or table showing ID similarity vs number of inputs.
  6. [Section 5] The conclusion acknowledges limitations such as hair hallucination and skin-tone predominance; these are significant for generative quality and should be discussed earlier, ideally in Sections 4.2 and 4.3 where qualitative results are shown.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: Omni-ID is trained by reconstruction losses and evaluated by an external FaceNet similarity metric, with all baselines sharing the same downstream adapter training.

full rationale

The paper's derivation chain is self-contained rather than circular. The representation ℓ=E(X) is trained with the MTD loss (Eq. 4) and flow-matching loss (Eq. 6), both of which reconstruct target images from the Omni-ID representation and a corrupted target. The reported identity-preservation numbers are cosine similarities between FaceNet embeddings of generated and ground-truth images (Sec. 4.1). No equation or construction identifies the training loss with the FaceNet metric, and no evaluation result is fed back as a training target. The downstream controllable-face benchmark trains a separate IP-Adapter+ControlNet for every representation, including ArcFace and CLIP, under the same frozen FLUX backbone, and it is evaluated on both a held-out MFHQ split and an external Webface test set, so the comparison has independent content. Self-citations in the related-work section (e.g., Refs. [26], [35], [36], [40], [42]) are used as background and are not load-bearing for the central claim. The resemblance between the few-to-many reconstruction objective and the downstream controllable-face task is a task-alignment concern about benchmark validity, not a definitional reduction of the kind required for circularity; accordingly no circular step is exhibited.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

No formal derivation is present; the method is empirical. The free parameters are hyperparameters selected by ablations, and the axioms are domain assumptions about feature extractors, dataset consistency, and the evaluation metric. The paper's own conclusion, Sec. 5, admits known limitations such as lighting injection, skin-tone predominance, and the absence of non-intrinsic attributes like hair, which partially offsets the strength of the assumptions.

free parameters (5)
  • L=256 learned query tokens, C=1280 channels = 256 x 1280
    Representation size chosen by hand; C=1280 is fixed by CLIP-H features, L=256 is chosen to balance capacity and fixed-size structure. No ablation on L is reported.
  • MTD masking ratio = 0.95
    Selected by ablation of 0.85/0.95/0.99 in Table 2; this affects how much identity information the encoder must supply.
  • Few-to-many input and output counts = 1-3 inputs, 8 targets
    Ablation table compares 3-1, 3-5, 3-8, and 8-8; the 3-8 configuration is chosen as best and used in training.
  • Encoder and decoder block counts = 2 cross-attention + 2 self-attention in encoder; 6 cross-attention + 2 self-attention in MTD decoder
    Architecture hyperparameter; the appendix says the decoder block count was chosen to avoid compensating for encoder quality, but no sweep is shown.
  • MFHQ pose sampling schedule = 16 pose clusters, 8 frames, at least 15 degrees of pose difference
    Dataset construction rule chosen to ensure pose diversity; it is not varied or ablated in the paper.
assumptions (5)
  • domain assumption CLIP-H features, after full fine-tuning, are a sufficient per-image token source for identity encoding.
    Sec. 3.1: the Omni-ID encoder builds image features as keys and values from CLIP-H; if CLIP-H discards local facial details, the encoder cannot recover them.
  • domain assumption FLUX dev is a strong enough frozen prior that conditional flow matching can reconstruct identity-specific faces and serve as a representation teacher.
    Sec. 3.3 and Appendix A.2: the flow-matching decoder is a pretrained FLUX model with IP-Adapter projections; this assumes the base model's prior supports face reconstruction.
  • domain assumption FaceNet cosine similarity is a valid proxy for identity preservation in generated images.
    Sec. 4.1: ID similarity is the headline metric; no human rating or alternative metric is used for the main comparisons.
  • domain assumption MFHQ video clusters correspond to one consistent identity with enough pose and expression variation.
    Sec. 3.4 and Fig. I: identities are clustered by face embedding cosine distance; clustering errors would inject identity noise and hurt training.
  • domain assumption A 95% masking ratio plus face-segmentation loss prevents autoencoding shortcuts, so reconstruction forces identity information into the representation.
    Sec. 3.3: the paper claims MTD ensures 'identity information is solely derived from l'; if visible pixels leak identity, the few-to-many reconstruction provides less incentive to learn a holistic identity code.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Omni-ID: Holistic Identity Representation Designed for Generative Tasks." pith.science (2026). https://pith.science/paper/OUH46UYE

@misc{pith2026241209694,
  author       = {Pith},
  title        = {Pith review of: Omni-ID: Holistic Identity Representation Designed for Generative Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OUH46UYE}},
  note         = {Machine review of arXiv:2412.09694}
}
read the original abstract

We introduce Omni-ID, a novel facial representation designed specifically for generative tasks. Omni-ID encodes holistic information about an individual's appearance across diverse expressions and poses within a fixed-size representation. It consolidates information from a varied number of unstructured input images into a structured representation, where each entry represents certain global or local identity features. Our approach uses a few-to-many identity reconstruction training paradigm, where a limited set of input images is used to reconstruct multiple target images of the same individual in various poses and expressions. A multi-decoder framework is further employed to leverage the complementary strengths of diverse decoders during training. Unlike conventional representations, such as CLIP and ArcFace, which are typically learned through discriminative or contrastive objectives, Omni-ID is optimized with a generative objective, resulting in a more comprehensive and nuanced identity capture for generative tasks. Trained on our MFHQ dataset -- a multi-view facial image collection, Omni-ID demonstrates substantial improvements over conventional representations across various generative tasks.

Figures

Figures reproduced from arXiv: 2412.09694 by the authors.

Figure 1
Figure 1. Omni-ID is a facial representation that consolidates in￾formation from a varied number of images of an individual into a fixed-size, structured encoding. Each element of this encod￾ing captures specific global or local identity features, enabling high-fidelity generation in new poses, expressions, and capturing identity-consistent variations. like CLIP [29], are not well suited for generative applica￾tions. Intuitiv… view at source ↗
Figure 2
Figure 2. Face generation comparison of different facial rep￾resentations with single input (top row) and two inputs (bottom row). We evaluate different facial representations by training an IP-adapter [47] on FLUX [4] with each representation. It can be seen that single-instance representations such as ArcFace and CLIP struggle to combine unique features appear in each observa￾tion (e.g., eye color and nose shape), whereas o… view at source ↗
Figure 3
Figure 3. Omni-ID employs a multi-decoder few-to-many identity reconstruction training strategy, incorporating three key design features: (1) An encoder that learns a unified, fixed-size identity representation from a varied number of inputs; (2) A few-to-many identity reconstruction task, designed to generate multiple faces of an individual in various poses and expressions from a limited set of samples of the same individual… view at source ↗
Figures from the paper (6 more)
Figure 5
Figure 5. Figure 5: Multi-decoder training. (left) Masked Transformer De￾coder (MTD) is designed to reconstruct unseen facial pixels from the Omni-ID representation and a minimal subset of visible pixels which do not leak identity. (right) Flow Matching Decoder en￾hances the encoder by a …
Figure 6
Figure 6. Figure 6: Qualitative comparisons in controllable face generation. We train the same IP-Adapter+ControlNet for each representation. Omni-ID achieves superior identity preservation and captures nuanced changes with varying poses and expressions more faithfully. tion features [12]…
Figure 7
Figure 7. Figure 7: Qualitative comparisons with different representa￾tions in personalized T2I generation. We show results of the same IP-Adapter trained with different representations. Omni￾ID achieves better ID preservation for both single and multiple input images. See more examples a…
Figure 9
Figure 9. Figure 9 [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 8
Figure 8. Figure 8: Gallery of Omni-ID in personalized T2I generation. Omni-ID enables high identity preservation. Results achieved by injecting Omni-ID representation through IP-Adapter [47] into the frozen FLUX dev model [4] without LoRA [21] or postprocessing. not accurately represente…
Figure 10
Figure 10. Figure 10 [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 30 canonical work pages

  1. [1]

    Im- age2stylegan: How to embed images into the stylegan latent space? In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4432–4441, 2019

    Rameen Abdal, Yipeng Qin, and Peter Wonka. Im- age2stylegan: How to embed images into the stylegan latent space? In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4432–4441, 2019. 2

    work page 2019

  2. [2]

    Styleflow: Attribute-conditioned exploration of stylegan- generated images using conditional continuous normalizing flows

    Rameen Abdal, Peihao Zhu, Niloy J Mitra, and Peter Wonka. Styleflow: Attribute-conditioned exploration of stylegan- generated images using conditional continuous normalizing flows. In ACM Transactions on Graphics (TOG) , pages 1–

    work page

  3. [3]

    On the emergence of invariance and disentangling in deep representations

    Alessandro Achille and Stefano Soatto. On the emergence of invariance and disentangling in deep representations. CoRR,

    work page

  4. [4]

    Black Forest Labs. Flux. https://github.com/ black-forest-labs/flux , 2024. 2, 5, 7, 11, 12, 13, 15

    work page 2024

  5. [5]

    A morphable model for the synthesis of 3d faces

    V olker Blanz and Thomas Vetter. A morphable model for the synthesis of 3d faces. In ACM Transactions on Graphics (SIGGRAPH), pages 187–194. ACM, 1999. 2

    work page 1999

  6. [6]

    Facewarehouse: A 3d facial expression database for visual computing

    Chen Cao, Yanlin Weng, Shun Zhou, Yiying Tong, and Kun Zhou. Facewarehouse: A 3d facial expression database for visual computing. IEEE Transactions on Visualization and Computer Graphics, 20(3):413–425, 2014. 2

    work page 2014

  7. [7]

    Topiq: A top-down approach from semantics to distortions for image quality assessment

    Chaofeng Chen, Jiadi Mo, Jingwen Hou, Haoning Wu, Liang Liao, Wenxiu Sun, Qiong Yan, and Weisi Lin. Topiq: A top-down approach from semantics to distortions for image quality assessment. IEEE Transactions on Image Processing, 33:2404–2418, 2024. 5, 12

    work page 2024

  8. [8]

    Insightface: 2d and 3d face anal- ysis project

    InsightFace Contributors. Insightface: 2d and 3d face anal- ysis project. https://github.com/deepinsight/ insightface, 2024. Accessed: 2024-11-15. 11

    work page 2024

Show all 55 references
  1. [9]

    Emoca: Emotion driven monocular face capture and animation

    Radek Dan ˇeˇcek, Michael J Black, and Timo Bolkart. Emoca: Emotion driven monocular face capture and animation. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 20311–20322, 2022. 2

    2022

  2. [10]

    ArcFace: Additive angular mar- gin loss for deep face recognition.IEEE Transactions on Pat- tern Analysis and Machine Intelligence , 44(10):5962–5979,

    Jiankang Deng, Jia Guo, Jing Yang, Niannan Xue, Irene Kot- sia, and Stefanos Zafeiriou. ArcFace: Additive angular mar- gin loss for deep face recognition.IEEE Transactions on Pat- tern Analysis and Machine Intelligence , 44(10):5962–5979,

  3. [11]

    Diffusion mod- els beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion mod- els beat gans on image synthesis. NeurIPS, 34:8780–8794,

  4. [12]

    facenet-pytorch: Pretrained pytorch face detec- tion and recognition models

    Tim Esler. facenet-pytorch: Pretrained pytorch face detec- tion and recognition models. https://github.com/ timesler/facenet-pytorch, 2020. 6

    2020

  5. [13]

    Learning an animatable detailed 3d face model from in-the- wild images

    Yao Feng, Haiwen Feng, Michael J Black, and Timo Bolkart. Learning an animatable detailed 3d face model from in-the- wild images. ACM Transactions on Graphics (ToG), 40(4): 1–13, 2021. 2

    2021

  6. [14]

    An image is worth one word: Personalizing text-to- image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patash- nik, Amit H Bermano, Gal Chechik, and Daniel Cohen- Or. An image is worth one word: Personalizing text-to- image generation using textual inversion. arXiv preprint arXiv:2208.01618, 2022. 2

    2022 arXiv

  7. [15]

    Encoder-based domain tuning for fast personalization of text-to-image models.ACM Transactions on Graphics (TOG), 42(4):1–13, 2023

    Rinon Gal, Moab Arar, Yuval Atzmon, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. Encoder-based domain tuning for fast personalization of text-to-image models.ACM Transactions on Graphics (TOG), 42(4):1–13, 2023. 3

    2023

  8. [16]

    Lcm- lookahead for encoder-based text-to-image personalization

    Rinon Gal, Or Lichter, Elad Richardson, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. Lcm- lookahead for encoder-based text-to-image personalization. arXiv preprint arXiv:2404.03620, 2024. 3

    2024 arXiv

  9. [17]

    Pulid: Pure and lightning ID customization via con- trastive alignment

    Zinan Guo, Yanze Wu, Zhuowei Chen, Lang Chen, and Qian He. Pulid: Pure and lightning ID customization via con- trastive alignment. CoRR, abs/2404.16022, 2024. 3, 12, 13, 15

    2024 arXiv

  10. [18]

    Ganspace: Discovering interpretable gan con- trols

    Erik H ¨ark¨onen, Aaron Hertzmann, Jaakko Lehtinen, and Sylvain Paris. Ganspace: Discovering interpretable gan con- trols. In Advances in Neural Information Processing Sys- tems, pages 9841–9850, 2020. 2

    2020

  11. [19]

    Classifier-free diffusion guidance

    Jonathan Ho. Classifier-free diffusion guidance. ArXiv, abs/2207.12598, 2022. 2

    2022 arXiv

  12. [20]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. NeurIPS, 33:6840–6851, 2020. 2

    2020

  13. [21]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In ICLR, 2022. 2, 7, 12, 13, 15

    2022

  14. [22]

    A style-based generator architecture for generative adversarial networks

    Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In CVPR, pages 4401–4410, 2019. 2

    2019

  15. [23]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597, 2023. 11

    2023 arXiv

  16. [24]

    Tianye Li, Timo Bolkart, Michael. J. Black, Hao Li, and Javier Romero. Learning a model of facial shape and ex- pression from 4D scans. ACM Transactions on Graphics, (Proc. SIGGRAPH Asia), 36(6):194:1–194:17, 2017. 2, 12

    2017

  17. [25]

    Photomaker: Customizing realistic human photos via stacked id embedding

    Zhen Li, Mingdeng Cao, Xintao Wang, Zhongang Qi, Ming- Ming Cheng, and Ying Shan. Photomaker: Customizing realistic human photos via stacked id embedding. arXiv preprint arXiv:2312.04461, 2023. 3, 6, 12, 13, 16

    2023 arXiv

  18. [26]

    Mystyle: A personalized generative prior

    Yotam Nitzan, Kfir Aberman, Qiurui He, Orly Liba, Michal Yarom, Yossi Gandelsman, Inbar Mosseri, Yael Pritch, and Daniel Cohen-Or. Mystyle: A personalized generative prior. ACM Transactions on Graphics (TOG), 41(6):1–10, 2022. 2

    2022

  19. [27]

    Arc2face: A foundation model of human faces

    Foivos Paraperas Papantoniou, Alexandros Lattas, Stylianos Moschoglou, Jiankang Deng, Bernhard Kainz, and Stefanos Zafeiriou. Arc2face: A foundation model of human faces. CoRR, abs/2403.11641, 2024. 5, 6, 8

    2024 arXiv

  20. [28]

    SDXL: improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. SDXL: improving latent diffusion models for high-resolution image synthesis. InInternational Confer- ence on Learning Representations (ICLR). OpenReview.net,

  21. [29]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  22. [30]

    3d facial expres- sions through analysis-by-neural-synthesis

    George Retsinas, Panagiotis Paraskevas Filntisis, Radek Danecek, Victoria Fern ´andez Abrevaya, Anastasios Rous- sos, Timo Bolkart, and Petros Maragos. 3d facial expres- sions through analysis-by-neural-synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision an...

    2024

  23. [31]

    Encoding in style: a stylegan encoder for image-to-image translation

    Elad Richardson, Yuval Alaluf, Or Patashnik, Yotam Nitzan, Yaniv Azar, Stav Shapiro, and Daniel Cohen-Or. Encoding in style: a stylegan encoder for image-to-image translation. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 2287–22...

    2021

  24. [32]

    Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer

    Robin Rombach, A. Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10674–10685, 2021. 2

    2022

  25. [33]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, pages 10684– 10695, 2022. 2, 7, 12, 13, 16

    2022

  26. [34]

    Nataniel Ruiz, Eunji Chong, and James M. Rehg. Fine- grained head pose estimation without keypoints. In The IEEE Conference on Computer Vision and Pattern Recog- nition (CVPR) Workshops, 2018. 6, 12

    2018

  27. [35]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In CVPR, pages 22500–22510, 2023. 2

    2023

  28. [36]

    Hyperdreambooth: Hypernetworks for fast personalization of text-to-image models

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Wei Wei, Tingbo Hou, Yael Pritch, Neal Wadhwa, Michael Rubinstein, and Kfir Aberman. Hyperdreambooth: Hypernetworks for fast personalization of text-to-image models. arXiv preprint arXiv:2307.06949, 2023. 3

    2023 arXiv

  29. [37]

    Instant- booth: Personalized text-to-image generation without test- time finetuning

    Jing Shi, Wei Xiong, Zhe Lin, and Hyun Joon Jung. Instant- booth: Personalized text-to-image generation without test- time finetuning. arXiv preprint arXiv:2304.03411, 2023. 3

    2023 arXiv

  30. [38]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. ArXiv, abs/2010.02502, 2020. 2

    2010 arXiv

  31. [39]

    Designing an encoder for stylegan image manipulation

    Omer Tov, Yuval Alaluf, Yotam Nitzan, Or Patashnik, and Daniel Cohen-Or. Designing an encoder for stylegan image manipulation. ACM Transactions on Graphics (TOG), 40(4): 1–14, 2021. 2

    2021

  32. [40]

    p+: Extended textual conditioning in text-to- image generation

    Andrey V oynov, Qinghao Chu, Daniel Cohen-Or, and Kfir Aberman. p+: Extended textual conditioning in text-to- image generation. arXiv preprint arXiv:2303.09522, 2023. 2

    2023 arXiv

  33. [41]

    Cos- face: Large margin cosine loss for deep face recognition

    Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. Cos- face: Large margin cosine loss for deep face recognition. In CVPR, pages 5265–5274. Computer Vision Foundation / IEEE Computer Society, 2018. 1, 2

    2018

  34. [42]

    Moa: Mixture-of-attention for subject-context disentanglement in personalized image gen- eration

    Kuan-Chieh Wang, Daniil Ostashev, Yuwei Fang, Sergey Tulyakov, and Kfir Aberman. Moa: Mixture-of-attention for subject-context disentanglement in personalized image gen- eration. arXiv preprint arXiv:2404.11565, 2024. 3

    2024 arXiv

  35. [43]

    Instantid: Zero-shot identity-preserving gener- ation in seconds

    Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, and An- thony Chen. Instantid: Zero-shot identity-preserving gener- ation in seconds. arXiv preprint arXiv:2401.07519, 2024. 3, 6, 12, 13, 16

    2024 arXiv

  36. [44]

    One-shot free-view neural talking-head synthesis for video conferenc- ing

    Ting-Chun Wang, Arun Mallya, and Ming-Yu Liu. One-shot free-view neural talking-head synthesis for video conferenc- ing. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), 2021. 5

    2021

  37. [45]

    VFHQ: A high-quality dataset and bench- mark for video face super-resolution

    Liangbin Xie, Xintao Wang, Honglun Zhang, Chao Dong, and Ying Shan. VFHQ: A high-quality dataset and bench- mark for video face super-resolution. In CVPR Workshops, pages 656–665. IEEE, 2022. 5

    2022

  38. [46]

    Flux-controlnet collections

    XLabs-AI. Flux-controlnet collections. https : / / huggingface . co / XLabs - AI / flux - controlnet - collections , 2024. Accessed: 2024-11-13. 11, 12

    2024

  39. [47]

    Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models. arXiv preprint arxiv:2308.06721 ,

    arXiv

  40. [48]

    Celebv-text: A large-scale facial text-video dataset

    Jianhui Yu, Hao Zhu, Liming Jiang, Chen Change Loy, Wei- dong Cai, and Wayne Wu. Celebv-text: A large-scale facial text-video dataset. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR) , pages 14805–14814. IEEE, 2023. 5

    2023

  41. [49]

    Adding conditional control to text-to-image diffusion models, 2023

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models, 2023. 3, 6

    2023

  42. [50]

    General facial representation learn- ing in a visual-linguistic manner

    Yinglin Zheng, Hao Yang, Ting Zhang, Jianmin Bao, Dong- dong Chen, Yangyu Huang, Lu Yuan, Dong Chen, Ming Zeng, and Fang Wen. General facial representation learn- ing in a visual-linguistic manner. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogn...

    2022

  43. [51]

    Celebv- hq: A large-scale video facial attributes dataset

    Hao Zhu, Wayne Wu, Wentao Zhu, Liming Jiang, Siwei Tang, Li Zhang, Ziwei Liu, and Chen Change Loy. Celebv- hq: A large-scale video facial attributes dataset. In Pro- ceedings of the European Conference on Computer Vision (ECCV), pages 650–667. Springer, 2022. 5

    2022

  44. [52]

    Webface260m: A benchmark unveiling the power of million-scale deep face recognition

    Zheng Zhu, Guan Huang, Jiankang Deng, Yun Ye, Junjie Huang, Xinze Chen, Jiagang Zhu, Tian Yang, Jiwen Lu, Da- long Du, et al. Webface260m: A benchmark unveiling the power of million-scale deep face recognition. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and...

    2021

  45. [54]

    and Stable Diffusion (SD) [33] as the base model, re- spectively. Our IPA Omni-ID trained by the simple flow matching loss without any advanced techniques such as LoRA [21], ID loss [17], aligment loss [17], stacked embed- ding [25], IdentityNet [43], achieves the highest ID p...

  46. [55]

    Our Omni-ID with IP-Adapter without any other regularization achieves the highest ID preservation

    and PhotoMakerV2 [25] use SDXL [28] as the base model, generating1024 × 1024 samples, which are resized to 512 × 512 to show with other methods side by side. Our Omni-ID with IP-Adapter without any other regularization achieves the highest ID preservation

  47. [2023]

    2, 3, 5, 7, 11, 12, 15

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.