Turn That Frown Upside Down: FaceID Customization via Cross-Training Data

Eduard Hovy; Guoyin Wang; Jiwei Li; Shuhe Wang; Tianwei Zhang; Xiaofei Sun; Xiaoya Li

REVIEW 3 major objections 6 minor 40 references

Turn That Frown Upside Down: FaceID Customization via Cross-Training Data

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A cross-training dataset lets face-ID models edit expressions without losing identity.

desk verdict A genuinely useful dataset and a clean data-level trick, but the 'significant improvement' claim needs an identity-disjoint evaluation before I'd trust it fully. read the letter →

arxiv 2501.15407 v1 pith:Z5SGH7AY submitted 2025-01-26 cs.CV cs.AIcs.LG

Shuhe Wang , Xiaoya Li , Xiaofei Sun , Guoyin Wang , Tianwei Zhang , Jiwei Li , Eduard Hovy This is my paper · ORCID

classification cs.CVcs.AIcs.LG

keywords FaceIDcustomizationcross-trainingdatasetdiffusionmodelstext-to-imagegenerationidentitypreservationfacialattributeeditingIP-AdapterInstant

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that existing FaceID customization models can reproduce the input face but cannot follow prompts to change the same person's expression, pose, or adornments. It introduces CrossFaceID, a dataset of roughly 40,000 text-image pairs from about 2,000 celebrities, each appearing in about 20 images with varied facial attributes, captioned by GPT-4o. The training recipe pairs one photo of a person as the image target with a different photo of the same person as the face-identity condition, forcing the model to generate the identity with altered facial features. Fine-tuning IP-Adapter and InstantID this way improves customization metrics and human ratings while keeping identity-similarity scores stable. If correct, this means a data change alone, without architectural changes, can give face-ID personalization models a basic face-editing ability.

What carries the argument

The central object is the CrossFaceID dataset combined with a cross-training data arrangement. The dataset is built by crawling about 60,000 celebrity images, filtering to roughly 40,596 images with faces at least 4 percent of the image and resolution at least 512x512, and annotating each with GPT-4o captions that emphasize facial features. The load-bearing training trick is the input-output pairing: the image target $x_0$ and the text condition $C_{\text{text}}$ come from one photo, while the face-identity condition $C_{\text{id}}$ is a face embedding extracted from a different photo of the same person. This forces the diffusion model to reconstruct the appearance described by the caption using an identity that is not tied to that exact photo, teaching it to vary expressions, poses, and adornments while keeping the person recognizable. Only the Image Adapter and IdentityNet parameters are updated; the base text-to-image model stays frozen.

What would settle it

Run the same fine-tuned models on a test set of faces whose identities are provably absent from CrossFaceID training, either non-celebrity volunteers or a held-out celebrity set with an explicit identity-disjointness check; if the customization gains (CLIP-T and human customization scores) shrink to the level of the unfine-tuned baselines, the central claim would be refuted.

Watch

Extended reading notes

Core claim

The central claim is that the inability of FaceID customization models to modify facial attributes is a training-data problem, not an architecture problem. CrossFaceID supplies identity-preserving variation: for each person, multiple images of the same identity under different expressions, angles, and adornments, each annotated with a free-form caption focused on facial features. During fine-tuning, the model receives the text caption of one image and a face embedding extracted from a different image of the same person, so the denoising objective is to reconstruct a specific appearance while the identity condition pulls from another view of that identity. This cross arrangement is what teaches the model to separate identity from transient facial attributes. The paper reports that the fine-tuned models match the original IP-Adapter and InstantID on identity-fidelity metrics while improving text-alignment metrics and human customization scores.

Load-bearing premise

The evaluation assumes that celebrity test images collected from the Internet represent ordinary users and that test identities do not overlap with the 1,626 training celebrities; the paper does not establish either condition.

Editorial extensions

If this is right

Fine-tuning existing FaceID customization models on CrossFaceID improves their ability to follow prompts that change expressions, poses, and accessories, with human customization scores rising from about 1.65 to 4.21 for the LAION baseline.
The same fine-tuning leaves identity fidelity approximately unchanged, according to Face Sim and human fidelity scores.
A data-only fix works for both IP-Adapter and InstantID, suggesting the limitation is shared across decoupled cross-attention FaceID architectures.
Because code, dataset, and trained models are released, other face-ID customization methods can be retrained or fine-tuned on the same data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

We infer that the reported gains have not yet been shown to transfer to identities absent from the training collection, because the paper does not state that test identities are disjoint from the 1,626 training celebrities.
We infer that the cross-training pairing is a general recipe: any task with multiple images of the same instance under varying attributes (pose, clothing, age) could use the same input/output swap.
We infer that deployment would need to address consent, since training on scraped celebrity photos may raise right-of-publicity concerns not discussed in the paper.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The dataset and the cross-training setup are the real contribution here. Using one image of a person as the visual target and a different same-person image as the identity reference is a simple, clean data-level trick that I haven't seen stated before. If the released data matches the description, CrossFaceID gives the community something useful: controlled same-person variation that existing training pairs lack.

The paper does well on the basics. The collection pipeline is described concretely. The GPT-4o annotation protocol is sensible. The baselines are the right ones (IP-Adapter and InstantID), and the qualitative figures show visible gains in expression, pose, and accessory editing. The loss is just the standard diffusion loss with a swapped face condition, so there is no hidden circularity or fitted constants in the objective. The related work correctly identifies that current datasets use identical input-output faces.

The soft spots are real but not fatal. The biggest one is the evaluation's failure to establish identity-disjoint generalization. CrossFaceID-test and the human-eval set are also celebrity images collected from the Internet, and the paper never says these identities are disjoint from the 1,626 training celebrities. If they overlap, part of the reported customization gain could be memory of that person's alternate photos rather than a learned ability to edit a novel face. That is a missing experiment—a held-out-identity split or a cross-dataset test—rather than a contradiction in the paper.

Second, the numbers are reported without error bars or significance tests. On CrossFaceID-test, the CLIP-T improvements are +0.04 to +0.06, which is small even if consistent. The human evaluation uses 10 raters and no inter-rater reliability. These do not sink the claim, but they do make the word "significantly" carry more weight than the statistics justify. Third, the 4% face-area threshold is hand-tuned and validated through the same evaluation, so a sensitivity check would be good. Finally, a minor inconsistency: the abstract says roughly 2,000 persons and 40,000 images, while Table 1 reports 1,626 celebrities and 40,596 images. That is worth cleaning up.

Overall, the central claim is plausible and the contribution is concrete. The paper deserves a serious referee; the reviewer should ask for identity-disjoint evaluation and error bars. I'd engage with this work and would likely cite the dataset.

Referee Report

3 major / 6 minor

Summary. The paper proposes CrossFaceID, a dataset of approximately 40,000 text-image pairs from about 1,626 celebrities, with multiple images per person capturing variations in expression, pose, and accessories. The authors fine-tune two state-of-the-art FaceID customization models, IP-Adapter and InstantID, on this dataset using a cross-training scheme in which the face condition is taken from a different image of the same person than the target image, forcing the model to apply text-specified modifications while preserving identity. The paper reports automatic metrics (CLIP-T, CLIP-I, Face Sim) and human evaluations showing improved customization with maintained fidelity, and it releases code, data, and models.

Significance. If the evaluation concerns are addressed, CrossFaceID could be a useful resource for face customization research, directly targeting a known limitation of current FaceID methods: the inability to modify facial attributes while preserving identity. The public release of the dataset, trained models, and code is a concrete strength that enables reproducibility and follow-up work. However, the current evidence for the central claim—that fine-tuning on CrossFaceID improves customization on novel identities—is weakened by the potential identity overlap between training and test sets and by the absence of statistical testing, so the significance of the reported improvements is not yet established.

major comments (3)

[§3.1, §5.2, §5.3] The test sets (CrossFaceID-test and the 200 human-evaluation faces) appear to be drawn from the same celebrity-image distribution as the training set (1,626 celebrities), but the paper never states that the test identities are disjoint from training identities. If a test identity appeared in training, the model could memorize that person's attribute variants rather than learn a general face-editing operation, which would inflate the reported CLIP-T, CLIP-I, and human Customization scores in Tables 2 and 3. The authors should provide identity-disjoint train/test splits or report the overlap, and ideally include a cross-dataset evaluation on identities never seen in training.
[§5.2, §5.3, Tables 2 and 3] All quantitative results are reported as point estimates without error bars, confidence intervals, or significance tests. The automatic metric improvements are small (e.g., CLIP-T +0.06, Face Sim +0.01 in Table 2), and the human evaluation uses only 10 participants and 200 faces with no inter-rater agreement or variance reported. The claim that customization is 'significantly improved' is therefore not statistically supported.
[§5.2, Table 2] The use of CLIP-I on CrossFaceID-test is questionable. CLIP-I measures similarity between the input and generated images, and on a customization task where the generated image should differ from the input (e.g., a different expression or angle), a higher CLIP-I could indicate that the model is making insufficient modifications, not that it is better at customization. The paper correctly notes that Face Sim is unsuitable for CrossFaceID-test for this reason, but does not explain why CLIP-I, also a similarity metric, is appropriate. The authors should clarify the interpretation of CLIP-I on this test set or replace it with a more task-appropriate metric.

minor comments (6)

[Table 1] 'Medium Images Per Celebrity' appears to be a typo for 'Median Images Per Celebrity.'
[Abstract and Table 1] The abstract says 'approximately 2,000 persons' while Table 1 reports 1,626 celebrities; please make the numbers consistent.
[§3.2] The 4% face-area threshold is a hand-tuned free parameter; the paper states it was 'determined through iterative refinements and validated via human and model evaluations' but provides no details or ablation. Please add a sensitivity analysis or at least describe the validation procedure.
[§4.2.1] The notation for the training triples is introduced as (y_image, y_text, y_face) but later the superscripts i and j are used inconsistently (e.g., y^i_image vs y_i_image). Please standardize the notation.
[Figure 2 caption] The caption says 'Rise' for an expression, which is likely a typo for 'Raise' or 'Eyes up'; please clarify.
[§5.1.1] The claim that the LAION-trained model demonstrates performance comparable to the official InstantID model is supported only by qualitative examples in Figure 4; please provide quantitative evidence for this baseline equivalence.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the fine-tuning experiment uses a standard diffusion objective and external metrics, so the central claim is not equivalent to its inputs.

full rationale

The paper's central claim is empirical: fine-tuning IP-Adapter and InstantID on the CrossFaceID dataset improves face customization while preserving FaceID fidelity. The training loss in Eq. 10 is the standard diffusion denoising objective, with an input image x0, text condition Ctext, and face condition Cid drawn from a different image of the same person. No fitted parameter is renamed as a prediction, and no equation reduces by construction to the reported improvement. The evaluation uses external metrics (CLIP-T, CLIP-I, Face Sim) and human ratings rather than quantities derived from the training objective. The paper does not rely on a self-citation chain or an imported uniqueness theorem; its baselines are cited external methods, and the CrossFaceID dataset is a new contribution. The main validity concern is that CrossFaceID-test and the human-evaluation celebrity faces (Sections 5.2 and 5.3) are not stated to be disjoint from the 1,626 training celebrities (Section 3.1), so memorization could inflate the reported gains. That is an evaluation-bias or generalization concern, not a circular derivation, and per the review rules it does not raise the circularity score. Therefore no circular steps are identified.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical entities are postulated. The paper's central claim rests on dataset-construction choices (the three filtering thresholds above) and on domain assumptions about annotation quality, identity matching, and metric validity. These are the main unverified inputs the reader must accept.

free parameters (3)

face_min_area_ratio = 0.04 (4%)
Hand-tuned threshold in image filtering, refined iteratively and validated via human and model evaluations (Section 3.2, rule 3). It controls which images enter the dataset and thus affects the trained model.
max_faces_per_image = 3
Manual rule limiting the number of faces in each image to at most three (Section 3.2, rule 1). This is a design choice that influences training examples.
min_image_resolution = 512x512
Manual resolution threshold for filtering (Section 3.2, rule 2). It affects image quality and the facial detail available for training.

assumptions (4)

domain assumption Diffusion models can learn facial modifications from cross-conditioned pairs where the visual input and the identity reference are different images of the same person.
The training objective in Eq. 10 assumes the standard noise-prediction loss will transfer attribute-editing ability to inference. This is an empirical assumption, not formally derived.
domain assumption GPT-4o annotations accurately capture the facial attributes that users will request.
The generated captions form the text condition during training. Wrong or inconsistent captions would teach incorrect text-to-face mappings.
domain assumption The face recognition model Antelopev2 reliably matches images of the same celebrity across poses, expressions, and accessories.
Cross-training pairs faces of the same person using face embeddings. If identity matching is noisy, the pairs are mislabeled and the training signal degrades.
domain assumption CLIP-T, CLIP-I, and Face Sim are valid proxies for customization and identity preservation.
The paper uses these metrics to claim success, supplementing them with human evaluation. The metrics are not calibrated against end-user behavior in this paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Turn That Frown Upside Down: FaceID Customization via Cross-Training Data." pith.science (2026). https://pith.science/paper/Z5SGH7AY

@misc{pith2026250115407,
  author       = {Pith},
  title        = {Pith review of: Turn That Frown Upside Down: FaceID Customization via Cross-Training Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z5SGH7AY}},
  note         = {Machine review of arXiv:2501.15407}
}

read the original abstract

Existing face identity (FaceID) customization methods perform well but are limited to generating identical faces as the input, while in real-world applications, users often desire images of the same person but with variations, such as different expressions (e.g., smiling, angry) or angles (e.g., side profile). This limitation arises from the lack of datasets with controlled input-output facial variations, restricting models' ability to learn effective modifications. To address this issue, we propose CrossFaceID, the first large-scale, high-quality, and publicly available dataset specifically designed to improve the facial modification capabilities of FaceID customization models. Specifically, CrossFaceID consists of 40,000 text-image pairs from approximately 2,000 persons, with each person represented by around 20 images showcasing diverse facial attributes such as poses, expressions, angles, and adornments. During the training stage, a specific face of a person is used as input, and the FaceID customization model is forced to generate another image of the same person but with altered facial features. This allows the FaceID customization model to acquire the ability to personalize and modify known facial features during the inference stage. Experiments show that models fine-tuned on the CrossFaceID dataset retain its performance in preserving FaceID fidelity while significantly improving its face customization capabilities. To facilitate further advancements in the FaceID customization field, our code, constructed datasets, and trained models are fully available to the public.

Figures

Figures reproduced from arXiv: 2501.15407 by the authors.

**Figure 1.** An example illustrating the limitations of existing FaceID customization models in generating images of the same individual with [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

**Figure 2.** Images sampled from our constructed CrossFaceID dataset are presented. From top to bottom, the display includes six persons, [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** The distribution of various facial features (e.g., expres [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: The results demonstrate the performance of FaceID customization models in maintaining FaceID fidelity. For models, “InstantID” [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: The results of the performance for FaceID customization models in customizing or editing FaceID. Here, ”InstantID” represents [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 16 canonical work pages

[1]

Gpt-4 technical report

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 , 2023

arXiv 2023
[2]

ediff-i: Text-to-image dif- fusion models with an ensemble of expert denoisers

Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Ji- aming Song, Qinsheng Zhang, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, et al. ediff-i: Text-to-image dif- fusion models with an ensemble of expert denoisers. arXiv preprint arXiv:2211.01324, 2022

arXiv 2022
[3]

Photoverse: Tuning-free image customization with text-to-image diffusion models

Li Chen, Mengyi Zhao, Yiheng Liu, Mingxu Ding, Yangyang Song, Shizun Wang, Xu Wang, Hao Yang, Jing Liu, Kang Du, et al. Photoverse: Tuning-free image customization with text-to-image diffusion models. arXiv preprint arXiv:2309.05793, 2023

arXiv 2023
[4]

Dreamidentity: Enhanced ed- itability for efficient face-identity preserved image genera- tion

Zhuowei Chen, Shancheng Fang, Wei Liu, Qian He, Mengqi Huang, and Zhendong Mao. Dreamidentity: Enhanced ed- itability for efficient face-identity preserved image genera- tion. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 1281–1289, 2024

work page 2024
[5]

Diffusion models beat gans on image synthesis

Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural informa- tion processing systems, 34:8780–8794, 2021

work page 2021
[6]

Cogview: Mastering text-to-image generation via transformers

Ming Ding, Zhuoyi Yang, Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, et al. Cogview: Mastering text-to-image generation via transformers. Advances in neural information processing systems, 34:19822–19835, 2021

2021
[7]

Cogview2: Faster and better text-to-image generation via hierarchical transformers

Ming Ding, Wendi Zheng, Wenyi Hong, and Jie Tang. Cogview2: Faster and better text-to-image generation via hierarchical transformers. Advances in Neural Information Processing Systems, 35:16890–16902, 2022

work page 2022
[8]

An image is worth one word: Personalizing text-to- image generation using textual inversion

Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patash- nik, Amit H Bermano, Gal Chechik, and Daniel Cohen- Or. An image is worth one word: Personalizing text-to- image generation using textual inversion. arXiv preprint arXiv:2208.01618, 2022

arXiv 2022

Show all 40 references

[9]

Lcm- lookahead for encoder-based text-to-image personalization

Rinon Gal, Or Lichter, Elad Richardson, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. Lcm- lookahead for encoder-based text-to-image personalization. arXiv preprint arXiv:2404.03620, 2(3):4, 2024

2024 arXiv
[10]

Deep autoregressive networks

Karol Gregor, Ivo Danihelka, Andriy Mnih, Charles Blun- dell, and Daan Wierstra. Deep autoregressive networks. pages 1242–1250, 2014

2014
[11]

Pixelvae: A latent variable model for natural im- ages

Ishaan Gulrajani, Kundan Kumar, Faruk Ahmed, Adrien Ali Taiga, Francesco Visin, David Vazquez, and Aaron Courville. Pixelvae: A latent variable model for natural im- ages. arXiv preprint arXiv:1611.05013, 2016

2016 arXiv
[12]

Denoising dif- fusion probabilistic models

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020

2020
[13]

Composer: Creative and controllable im- age synthesis with composable conditions

Lianghua Huang, Di Chen, Yu Liu, Yujun Shen, Deli Zhao, and Jingren Zhou. Composer: Creative and controllable im- age synthesis with composable conditions. arXiv preprint arXiv:2302.09778, 2023

2023 arXiv
[14]

Gpt-4o system card

Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perel- man, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Weli- hinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024

2024 arXiv
[15]

Diffface: Diffusion-based face swapping with facial guid- ance

Kihong Kim, Yunho Kim, Seokju Cho, Junyoung Seo, Jisu Nam, Kychul Lee, Seungryong Kim, and KwangHee Lee. Diffface: Diffusion-based face swapping with facial guid- ance. arXiv preprint arXiv:2212.13344, 2022

2022 arXiv
[16]

Multi-concept customization of text-to-image diffusion

Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi-concept customization of text-to-image diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1931–1941, 2023

1931
[17]

Photomaker: Customizing re- alistic human photos via stacked id embedding

Zhen Li, Mingdeng Cao, Xintao Wang, Zhongang Qi, Ming- Ming Cheng, and Ying Shan. Photomaker: Customizing re- alistic human photos via stacked id embedding. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8640–8650, 2024

2024
[18]

Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

2021 arXiv
[19]

Portraitbooth: A versatile portrait model for fast identity-preserved personalization

Xu Peng, Junwei Zhu, Boyuan Jiang, Ying Tai, Donghao Luo, Jiangning Zhang, Wei Lin, Taisong Jin, Chengjie Wang, and Rongrong Ji. Portraitbooth: A versatile portrait model for fast identity-preserved personalization. InProceedings of the IEEE/CVF Conference on Computer Vision a...

2024
[20]

Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis

Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023

2023 arXiv
[21]

Learning transferable visual models from natural language supervi- sion

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

2021
[22]

Exploring the limits of transfer learning with a unified text-to-text transformer

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67, 2020

2020
[23]

Hierarchical text-conditional image gen- eration with clip latents

Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gen- eration with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3, 2022

2022 arXiv
[24]

Zero-shot text-to-image generation

Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International confer- ence on machine learning, pages 8821–8831. Pmlr, 2021

2021
[25]

High-resolution image synthesis with latent diffusion models

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

2022
[26]

Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 2250...

2023
[27]

Photorealistic text-to-image diffusion models with deep language understanding

Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

2022
[28]

Denoising diffusion implicit models

Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020

2010 arXiv
[29]

Score-based generative modeling through stochastic differential equa- tions

Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. arXiv preprint arXiv:2011.13456, 2020

2011 arXiv
[30]

Face0: Instantaneously conditioning a text-to- image model on a face

Dani Valevski, Danny Lumen, Yossi Matias, and Yaniv Leviathan. Face0: Instantaneously conditioning a text-to- image model on a face. InSIGGRAPH Asia 2023 Conference Papers, pages 1–10, 2023

2023
[31]

Conditional image genera- tion with pixelcnn decoders

Aaron Van den Oord, Nal Kalchbrenner, Lasse Espeholt, Oriol Vinyals, Alex Graves, et al. Conditional image genera- tion with pixelcnn decoders. Advances in neural information processing systems, 29, 2016

2016
[32]

Neural discrete representation learning

Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information pro- cessing systems, 30, 2017

2017
[33]

Attention is all you need

A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017

2017
[34]

Instantid: Zero-shot identity-preserving generation in seconds

Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, Anthony Chen, Huaxia Li, Xu Tang, and Yao Hu. Instantid: Zero-shot identity-preserving generation in seconds. arXiv preprint arXiv:2401.07519, 2024

2024 arXiv
[35]

Fastcomposer: Tuning-free multi- subject image generation with localized attention

Guangxuan Xiao, Tianwei Yin, William T Freeman, Fr ´edo Durand, and Song Han. Fastcomposer: Tuning-free multi- subject image generation with localized attention. Interna- tional Journal of Computer Vision, pages 1–20, 2024

2024
[36]

Imagere- ward: Learning and evaluating human preferences for text- to-image generation

Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagere- ward: Learning and evaluating human preferences for text- to-image generation. Advances in Neural Information Pro- cessing Systems, 36, 2024

2024
[37]

Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models

Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models. arXiv preprint arXiv:2308.06721, 2023

2023 arXiv
[38]

Inserting anybody in diffusion models via celeb ba- sis

Ge Yuan, Xiaodong Cun, Yong Zhang, Maomao Li, Chenyang Qi, Xintao Wang, Ying Shan, and Huicheng Zheng. Inserting anybody in diffusion models via celeb ba- sis. arXiv preprint arXiv:2306.00926, 2023

2023 arXiv
[39]

Adding conditional control to text-to-image diffusion models

Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023

2023
[40]

Diffswap: High-fidelity and con- trollable face swapping via 3d-aware masked diffusion

Wenliang Zhao, Yongming Rao, Weikang Shi, Zuyan Liu, Jie Zhou, and Jiwen Lu. Diffswap: High-fidelity and con- trollable face swapping via 3d-aware masked diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 8568–8577, 2023

2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Gpt-4 technical report

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 , 2023

arXiv 2023

[2] [2]

ediff-i: Text-to-image dif- fusion models with an ensemble of expert denoisers

Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Ji- aming Song, Qinsheng Zhang, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, et al. ediff-i: Text-to-image dif- fusion models with an ensemble of expert denoisers. arXiv preprint arXiv:2211.01324, 2022

arXiv 2022

[3] [3]

Photoverse: Tuning-free image customization with text-to-image diffusion models

Li Chen, Mengyi Zhao, Yiheng Liu, Mingxu Ding, Yangyang Song, Shizun Wang, Xu Wang, Hao Yang, Jing Liu, Kang Du, et al. Photoverse: Tuning-free image customization with text-to-image diffusion models. arXiv preprint arXiv:2309.05793, 2023

arXiv 2023

[4] [4]

Dreamidentity: Enhanced ed- itability for efficient face-identity preserved image genera- tion

Zhuowei Chen, Shancheng Fang, Wei Liu, Qian He, Mengqi Huang, and Zhendong Mao. Dreamidentity: Enhanced ed- itability for efficient face-identity preserved image genera- tion. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 1281–1289, 2024

work page 2024

[5] [5]

Diffusion models beat gans on image synthesis

Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural informa- tion processing systems, 34:8780–8794, 2021

work page 2021

[6] [6]

Cogview: Mastering text-to-image generation via transformers

Ming Ding, Zhuoyi Yang, Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, et al. Cogview: Mastering text-to-image generation via transformers. Advances in neural information processing systems, 34:19822–19835, 2021

2021

[7] [7]

Cogview2: Faster and better text-to-image generation via hierarchical transformers

Ming Ding, Wendi Zheng, Wenyi Hong, and Jie Tang. Cogview2: Faster and better text-to-image generation via hierarchical transformers. Advances in Neural Information Processing Systems, 35:16890–16902, 2022

work page 2022

[8] [8]

An image is worth one word: Personalizing text-to- image generation using textual inversion

Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patash- nik, Amit H Bermano, Gal Chechik, and Daniel Cohen- Or. An image is worth one word: Personalizing text-to- image generation using textual inversion. arXiv preprint arXiv:2208.01618, 2022

arXiv 2022