Pith. sign in

REVIEW 4 major objections 4 minor 54 references

ComposeAnyone: Controllable Layout-to-Human Generation with Decoupled Multimodal Conditions

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single diffusion model turns color-block sketches, text, and reference photos into aligned human images.

desk verdict The decoupled conditioning formulation and ComposeHuman dataset are real contributions, but the empirical claims rest on an in-domain versus zero-shot comparison and a circular VLM judge, so the paper needs a fair re-run before the numbers can be trusted. read the letter →

arxiv 2501.12173 v1 pith:RRE4AIXN submitted 2025-01-21 cs.CV

Shiyue Zhang , Zheng Chong , Xi Lu , Wenqing Zhang , Haoxiang Li , Xujie Zhang , Jiehui Huang , Xiao Dong
show 1 more author
Xiaodan Liang
This is my paper · ORCID
classification cs.CV
keywords controllablehumanimagegenerationlayout-to-imagedecoupledmultimodalconditionshand-drawnlayoutdiffusionmodelssubject-drivenvirtualtry-onComposedataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ComposeAnyone claims a single diffusion model can generate a human image from a hand-drawn color-block layout in which each body part (face, top, bottom, shoes) is described separately, either by text or by a reference photo, mixed freely across parts. This removes the usual pairing requirement between text and image conditions, so text-only, image-only, and mixed-modality prompts all work in one framework. The paper builds the ComposeHuman dataset with decoupled per-component text and reference-image annotations, and reports that the resulting model aligns better with layout, text, and reference images than existing layout-guided and subject-driven baselines on VITON-HD, DressCode, and DeepFashion. A sympathetic reader would take the core claim as: spatial control plus per-part multimodal conditioning can be combined without sacrificing fidelity or consistency.

What carries the argument

The load-bearing mechanism is the decoupled multimodal condition injection: per-component text embeddings from CLIP are concatenated along the channel dimension, while VAE latents of the layout and of pixel-level concatenated reference images are concatenated along the spatial dimension, letting a single U-Net see all modalities without a dedicated fusion network. The second piece is inference-time cross-attention modulation, where the cross-attention map $A^{(i)}$ of each component description is multiplied elementwise by the binary mask $M_i$ of its color block in the hand-drawn layout ($A^*_{(i)} = A_{(i)} \odot M_i$), forcing each description's semantics to concentrate inside its spatial region. Training uses an SNR-weighted MSE denoising loss, and the color-block layout is constructed by fitting ellipses and rectangles to SCHP human-parsing masks.

What would settle it

Collect a set of genuine free-hand color-block layouts drawn by non-expert users (with overlaps and irregular shapes) and measure the paper's own Spatial Accuracy (Jaccard, Dice, SSIM weighted 0.25/0.25/0.50 between GroundingDINO-detected parts and the layout) as well as the VLM rate. If these scores drop substantially relative to layouts fitted from SCHP masks on the same prompts, the advertised hand-drawn layout control is not robust to real user input. A simpler targeted version: generate with a deliberately irregular layout (a star-shaped or heavily overlapping color block) and check whether the cross-attention modulation in Eq. (18) keeps the text description inside the intended region.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that decoupling text and image conditions per human component, and aligning each with a color-blocked layout, yields controllable layout-to-human generation across three modes. The input is a hand-drawn layout of ellipses and rectangles, one color block per component; each block can be paired with a text description, a reference image, or both. The model is a Stable Diffusion U-Net, initialized from InstructPix2Pix, in which the VAE latents of the layout and of concatenated reference images are injected by spatial concatenation and CLIP text embeddings by channel concatenation, with no extra feature network. At inference, the cross-attention map for each component's text is modulated by the binary mask of its color block (Eq. 18), which the ablations credit with sharper spatial and semantic alignment. Experiments on VITON-HD, DressCode, and DeepFashion report better alignment and image quality than GLIGEN, DenseDiffusion, MultiDiffusion, InstanceDiffusion, MIGC for layout guidance and IP-Adapter, λ-ECLIPSE, ELITE, CustomNet, AnyDoor for subject-driven generation.

Load-bearing premise

The layout control depends on the assumption that a user's free-hand color-block drawing will resemble the ellipse-and-rectangle layouts that the model saw during training, which were fitted automatically from SCHP segmentation masks; if real users draw looser, overlapping, or irregular shapes, the spatial alignment and the Eq. (18) mask modulation may degrade.

Editorial extensions

If this is right

  • A single trained model covers text-only, image-only, and mixed-modality human generation, since text and image conditions are sampled independently per component during training.
  • Users can add or remove accessories such as hats and bags by editing the hand-drawn layout instead of rewriting prompts, because each color block maps to a distinct component.
  • Multiple reference images can be supplied at once via pixel-level concatenation, avoiding sequential per-image conditioning.
  • Ablations show cross-attention modulation (CA) improves spatial accuracy and text fidelity, implying that mask-based attention steering is a working alternative to extra feature networks.
  • The ComposeHuman dataset provides decoupled component-level text and image annotations, giving a reusable training resource for layout-to-human generation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same decoupled conditioning recipe could transfer to non-human subjects (animals, vehicles, furniture) wherever component-level segmentation exists, because nothing in the mechanism is person-specific.
  • A testable extension: collect free-hand layouts from real users and compare generation fidelity against SCHP-fitted layouts; if the gap is large, the next version should train on synthetically perturbed or genuinely hand-drawn layouts.
  • Since each color block is independently conditioned, the method suggests a part-localized editing protocol where modifying one block leaves other components untouched, an effect that could be verified by measuring per-region drift before and after editing.
  • The stochastic 'text or image' drop strategy may also serve as a general recipe for other multi-condition generation tasks, reducing the need for paired text-image training data.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes ComposeAnyone, a diffusion-based framework for generating human images from hand-drawn layouts (color-blocked ellipses/rectangles), component-level text descriptions, and reference images, in a decoupled or mixed-modality manner. It introduces the ComposeHuman dataset, built from VITON-HD, DressCode, and DeepFashion, with CogVLM2-generated captions, SAM/SCHP-extracted component references, and fitted layout shapes. The method concatenates reference and layout latents along spatial dimensions and applies cross-attention modulation at inference. Quantitative comparisons on layout-guided and subject-driven tasks are reported, claiming superior alignment and image quality.

Significance. The proposed framework has the useful goal of unifying text-only, image-only, and mixed-modality human generation with spatial layout control in a single diffusion model, and the ComposeHuman dataset could be a resource for future work. However, the paper's central quantitative claim is not supported by its current experimental protocol. The evaluation is in-domain for the proposed model but zero-shot for the baselines, the headline VLM metric is computed with the same model used to generate training annotations, and the ablation results show a trade-off between alignment and fidelity rather than a clear win. The hand-drawn layout interaction is not validated with real drawings. Thus the idea has merit, but the empirical evidence as presented is insufficient.

major comments (4)
  1. [Section 4.1; Tables 1-2] The quantitative comparisons in Tables 1 and 2 are confounded by the training/evaluation protocol. The ComposeHuman training set is sampled from the training splits of VITON-HD, DressCode, and DeepFashion (Section 4.1), and evaluation is performed on the test splits of these same datasets. All baseline methods (GLIGEN, DenseDiffusion, MultiDiffusion, InstanceDiffusion, MIGC in Table 1; IP-Adapter, lambda-ECLIPSE, ELITE, CustomNet, AnyDoor in Table 2) are used off-the-shelf without fine-tuning. Consequently, ComposeAnyone is in-domain while the baselines are zero-shot, which can account for a substantial portion of the large FID/KID differences (e.g., FID 18.33 vs 87.10 in Table 1; 11.62 vs 32.53 in Table 2). The central claim that ComposeAnyone achieves better alignment and quality is therefore not supported by the evidence as presented. The authors should either fine-tune the baselines on the ComposeHuman training set or evaluate all methods under a zero-shot protocol on datasets not seen during training.
  2. [Section 3.2; Section 4.3] The VLM Rate metric is circular with respect to the training data. The component-level textual annotations in the ComposeHuman dataset are generated by CogVLM2 (Section 3.2, Figure 3), and the VLM Rate in Section 4.3 uses CogVLM2 as the judge to evaluate alignment between generated images and the same style of component descriptions. Since the model is explicitly trained to match CogVLM2's annotations, a high VLM Rate partly reflects fitting to the judge rather than true multimodal alignment. The authors should use an independent VLM (or human raters) for the evaluation metric, and should report results with both CogVLM2 and an alternative judge to demonstrate robustness.
  3. [Table 3; Section 4.6] The ablation results do not consistently support the claim that cross-attention modulation (CA) markedly enhances controllability. In Table 3, comparing '-CA+CFG' and '+CA+CFG' shows that CA improves VLM Rate by only 0.26 points (80.35 to 80.61) and Spatial Accuracy by 1.82 points (67.76 to 69.58), but worsens FID from 14.85 to 20.24, KID from 1.87 to 6.37, and LPIPS from 0.0735 to 0.1242. The same pattern holds when CFG is off. This trade-off between alignment metrics and image fidelity should be acknowledged and analyzed, rather than presenting CA as an unqualified improvement in controllability.
  4. [Section 3.2; Eqs. (3)-(6); Section 4] The 'hand-drawn layout' capability is not validated with actual hand-drawn inputs. All layouts used in training and evaluation are fitted ellipses and rectangles derived from SCHP segmentation masks via Eqs. (3)-(6). Real user drawings are likely to involve looser, overlapping, or irregular shapes, and the model's performance on such inputs is unknown. The paper should either include a user study with free-form drawings or an evaluation with layout perturbations to substantiate the claim that arbitrary color-block drawings work as described.
minor comments (4)
  1. [Section 4.1] The ComposeHuman dataset is not released in the paper; its size after cleaning and the distribution of components are not reported, which hampers reproducibility.
  2. [Section 3.3, Eq. (18)] In Eq. (18), the binary mask Mi is applied directly to the cross-attention map; the paper should clarify whether Mi is resized to the attention map resolution and whether it is normalized to preserve the total attention mass.
  3. [Section 3.2] The description of the 'Text or Image' random selection in Section 3.2 is brief; the probability of dropping text or image, and the handling of the corresponding prompts, should be specified.
  4. [References] The references are incomplete in some entries (e.g., [2] and [7] lack publication venue and page numbers).

Circularity Check

1 steps flagged · score 4.0 of 10

Evaluation loop circularity: the same CogVLM2 model writes the component captions used as training conditions and also serves as the VLM Rate judge, so the headline text-alignment metric is not independent of the annotation source.

  1. other [Section 3.2 (Text-Based Conditions) and Section 4.3 (Evaluation Metrics, 'VLM Rate'); Tables 1 and 2]
    "We utilize CogVLM2 [17] to expand the fine-grained textual descriptions for each human image by applying component-level queries with standardized formatting to attributes such as face, top, bottom, and shoes... VLM Rate. Since CLIP may not capture intricate details effectively, we use CogVLM2 [17] to query and assess the alignment of instance features, facilitating more sophisticated, detailed, and varied evaluation."

    The text conditions used to train ComposeAnyone and to run the reported comparisons are generated by CogVLM2, and the paper's headline VLM Rate metric is computed by that same CogVLM2 system acting as judge. The model is therefore trained to satisfy CogVLM2-generated captions and then scored by CogVLM2's own assessment of whether the image matches those captions. This makes VLM Rate a measure of agreement with the annotation source rather than an independent measure of text alignment, so part of the favorable VLM Rate gap in Tables 1 and 2 is built into the evaluation loop. The other reported metrics (FID, KID, spatial accuracy via GroundingDINO, and CLIP score) do not share this specific circularity, so the issue is confined to one headline metric rather than the whole method.

full rationale

The derivation chain of the generation method itself is not circular: ComposeAnyone fine-tunes a Stable Diffusion backbone with an MSE noise-prediction loss, and the layout and reference-image conditioning are defined by spatial/channel concatenation and cross-attention modulation (Section 3.3), not by the evaluation metrics. The main circularity found is evaluative: the same VLM, CogVLM2, writes the component-level captions in Section 3.2 and then judges text alignment in Section 4.3. Because the model is trained on CogVLM2's annotations and evaluated by CogVLM2's own notion of alignment, the VLM Rate metric is self-referential rather than an external check. I do not count the in-domain versus zero-shot baseline comparison as circularity; it is a fairness and validity concern about how much of the FID/KID gap reflects training on the same datasets, but it is not a reduction of a claimed result to its own inputs. The Limitation section acknowledges that VLM-based annotations may introduce inaccuracies, but it does not address this evaluator overlap. No load-bearing self-citation chain, uniqueness theorem, or ansatz-smuggling citation is present. Since the central claim of better alignment also has independent support from spatial accuracy and image-quality metrics, the appropriate score is 4 rather than higher.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on several domain assumptions about pretrained models, annotation quality, and layout representativeness, plus a few hand-chosen thresholds and weights. No new physical entities are introduced.

free parameters (3)
  • SSIM threshold for mask cross-validation = 0.75
    Section 3.2: hand-chosen threshold to decide whether SAM and SCHP segmentation agree. This controls which samples enter ComposeHuman and thus affects all downstream metrics.
  • Spatial Accuracy metric weights = 0.25 (Jaccard), 0.25 (Dice), 0.50 (SSIM)
    Section 4.3: the composite spatial accuracy score is a hand-weighted combination of three overlap measures. Changing the weights changes the reported ranking and the absolute numbers.
  • SNR loss weighting = w_i = 1/SNR(t_i)
    Section 3.3, Eq. (12)-(13): the loss weighting is a design choice not derived from first principles. It affects training dynamics and the final model quality.
assumptions (5)
  • domain assumption Pretrained Stable Diffusion (via InstructPix2Pix) provides a suitable generative prior for human images.
    Section 4.2: the backbone is initialized with InstructPix2Pix weights. The method relies on this prior to generate realistic humans and garments.
  • domain assumption CogVLM2 produces accurate component-level text annotations that can serve as training targets.
    Section 3.2: CogVLM2 is used to write fine-grained captions for face, top, bottom, and shoes. If these annotations are noisy or biased, the model learns those biases and the VLM Rate evaluation inherits them.
  • ad hoc to paper Color-blocked ellipse and rectangle layouts fitted from SCHP masks are representative of user hand-drawn layouts.
    Section 3.2: hand-drawn layouts are approximated by least-squares ellipses and rotated rectangles. The claim that users can draw layouts assumes this representation is faithful and that training on fitted layouts transfers to arbitrary sketches.
  • ad hoc to paper Concatenating reference latents along the width dimension is sufficient to condition on multiple reference images.
    Section 3.3, Eq. (9)-(10): multi-reference conditioning is implemented as spatial concatenation. The paper does not compare with other fusion mechanisms, so this is a design assumption.
  • ad hoc to paper Cross-attention modulation with a binary mask improves layout-text alignment without hurting fidelity.
    Section 3.3, Eq. (18): the inference-time attention modulation is heuristic. The ablation in Table 3 shows mixed effects on image quality metrics (SSIM, FID, KID worsen with CA+CFG), so this assumption is only partially supported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ComposeAnyone: Controllable Layout-to-Human Generation with Decoupled Multimodal Conditions." pith.science (2026). https://pith.science/paper/RRE4AIXN

@misc{pith2026250112173,
  author       = {Pith},
  title        = {Pith review of: ComposeAnyone: Controllable Layout-to-Human Generation with Decoupled Multimodal Conditions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RRE4AIXN}},
  note         = {Machine review of arXiv:2501.12173}
}
read the original abstract

Building on the success of diffusion models, significant advancements have been made in multimodal image generation tasks. Among these, human image generation has emerged as a promising technique, offering the potential to revolutionize the fashion design process. However, existing methods often focus solely on text-to-image or image reference-based human generation, which fails to satisfy the increasingly sophisticated demands. To address the limitations of flexibility and precision in human generation, we introduce ComposeAnyone, a controllable layout-to-human generation method with decoupled multimodal conditions. Specifically, our method allows decoupled control of any part in hand-drawn human layouts using text or reference images, seamlessly integrating them during the generation process. The hand-drawn layout, which utilizes color-blocked geometric shapes such as ellipses and rectangles, can be easily drawn, offering a more flexible and accessible way to define spatial layouts. Additionally, we introduce the ComposeHuman dataset, which provides decoupled text and reference image annotations for different components of each human image, enabling broader applications in human image generation tasks. Extensive experiments on multiple datasets demonstrate that ComposeAnyone generates human images with better alignment to given layouts, text descriptions, and reference images, showcasing its multi-task capability and controllability.

Figures

Figures reproduced from arXiv: 2501.12173 by the authors.

Figure 1
Figure 1. ComposeAnyone is capable of generating high-quality human images with decoupled multimodal conditions, such as captions, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of ComposeAnyone. (a)Data Preparation. We leverage CogVLM2, SAM, and SCHP to enrich the image-based try-on datasets with fine-grained textual descriptions, hand-drawn layouts, and human component sets. (b)Training Procedure. We begin by employing the VAE encoder and the CLIP encoder to extract image and text embeddings, respectively, subsequently injecting the embeddings into the U-Net through concatenation… view at source ↗
Figure 3
Figure 3. An example of fine-grained text description of [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison with subject-driven methods. ComposeAnyone demonstrates high fidelity in matching specific features [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparison with layout-guided text-to-image methods. ComposeAnyone demonstrates a high level of congruity with [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 7
Figure 7. Figure 7: Ablation results of ComposeAnyone. The red boxes in [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 6
Figure 6. Figure 6: Visual results of ComposeAnyone, highlighting its abil [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 28 canonical work pages

  1. [1]

    Spatext: Spatio-textual representation for con- trollable image generation

    Omri Avrahami, Thomas Hayes, Oran Gafni, Sonal Gupta, Yaniv Taigman, Devi Parikh, Dani Lischinski, Ohad Fried, and Xi Yin. Spatext: Spatio-textual representation for con- trollable image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 18370–18380, 2023. 2, 3

    work page 2023

  2. [2]

    Multidiffusion: Fusing diffusion paths for controlled image generation

    Omer Bar-Tal, Lior Yariv, Yaron Lipman, and Tali Dekel. Multidiffusion: Fusing diffusion paths for controlled image generation. arXiv preprint arXiv:2302.08113, 2023. 2, 3, 5, 7

    arXiv 2023

  3. [3]

    Sutherland, Michael Arbel, and Arthur Gretton

    Mikołaj Bi ´nkowski, Danica J. Sutherland, Michael Arbel, and Arthur Gretton. Demystifying mmd gans, 2021. 7

    work page 2021

  4. [4]

    In- structpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. In- structpix2pix: Learning to follow image editing instructions. arXiv preprint arXiv:2211.09800, 2022. 6

    arXiv 2022

  5. [5]

    Photoverse: Tuning-free image customization with text-to-image diffusion models

    Li Chen, Mengyi Zhao, Yiheng Liu, Mingxu Ding, Yangyang Song, Shizun Wang, Xu Wang, Hao Yang, Jing Liu, Kang Du, et al. Photoverse: Tuning-free image customization with text-to-image diffusion models. arXiv preprint arXiv:2309.05793, 2023. 2, 3

    arXiv 2023

  6. [6]

    Training-free layout control with cross-attention guidance

    Minghao Chen, Iro Laina, and Andrea Vedaldi. Training-free layout control with cross-attention guidance. In Proceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 5343–5353, 2024. 2, 3

    2024

  7. [7]

    Anydoor: Zero-shot object-level im- age customization

    Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao. Anydoor: Zero-shot object-level im- age customization. arXiv preprint arXiv:2307.09481, 2023. 2, 3, 5, 7, 8

    arXiv 2023

  8. [8]

    Layoutdiffuse: Adapting foundational dif- fusion models for layout-to-image generation.arXiv preprint arXiv:2302.08908, 2023

    Jiaxin Cheng, Xiao Liang, Xingjian Shi, Tong He, Tianjun Xiao, and Mu Li. Layoutdiffuse: Adapting foundational dif- fusion models for layout-to-image generation.arXiv preprint arXiv:2302.08908, 2023. 3

    arXiv 2023

Show all 54 references
  1. [9]

    Pose guided multi-person image generation from text

    Soon Yau Cheong, Armin Mustafa, and Andrew Gilbert. Pose guided multi-person image generation from text. arXiv preprint arXiv:2203.04907, 1, 2022. 2

    2022 arXiv

  2. [10]

    Viton-hd: High-resolution virtual try-on via misalignment-aware normalization

    Seunghwan Choi, Sunghyun Park, Minsoo Lee, and Jaegul Choo. Viton-hd: High-resolution virtual try-on via misalignment-aware normalization. In Proc. of the IEEE conference on computer vision and pattern recognition (CVPR), 2021. 6, 7

    2021

  3. [11]

    Measures of the amount of ecologic association between species

    Lee R Dice. Measures of the amount of ecologic association between species. Ecology, 26(3):297–302, 1945. 7

    1945

  4. [12]

    Stylegan-human: A data-centric odyssey of human genera- tion

    Jianglin Fu, Shikai Li, Yuming Jiang, Kwan-Yee Lin, Chen Qian, Chen Change Loy, Wayne Wu, and Ziwei Liu. Stylegan-human: A data-centric odyssey of human genera- tion. In European Conference on Computer Vision , pages 1–19. Springer, 2022. 2

    2022

  5. [13]

    An image is worth one word: Personalizing text-to- image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patash- nik, Amit H Bermano, Gal Chechik, and Daniel Cohen- Or. An image is worth one word: Personalizing text-to- image generation using textual inversion. arXiv preprint arXiv:2208.01618, 2022. 2, 3

    2022 arXiv

  6. [14]

    A versatile benchmark for de- tection, pose estimation, segmentation and re-identification of clothing images

    Yuying Ge, Ruimao Zhang, Lingyun Wu, Xiaogang Wang, Xiaoou Tang, and Ping Luo. A versatile benchmark for de- tection, pose estimation, segmentation and re-identification of clothing images. CVPR, 2019. 6, 8

    2019

  7. [15]

    Context- aware layout to image generation with enhanced object ap- pearance

    Sen He, Wentong Liao, Michael Ying Yang, Yongxin Yang, Yi-Zhe Song, Bodo Rosenhahn, and Tao Xiang. Context- aware layout to image generation with enhanced object ap- pearance. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 15049– 1...

    2021

  8. [16]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 3

    2020

  9. [17]

    Cogvlm2: Visual language mod- els for image and video understanding

    Wenyi Hong, Weihan Wang, Ming Ding, Wenmeng Yu, Qingsong Lv, Yan Wang, Yean Cheng, Shiyu Huang, Jun- hui Ji, Zhao Xue, et al. Cogvlm2: Visual language mod- els for image and video understanding. arXiv preprint arXiv:2408.16500, 2024. 3, 7

    2024 arXiv

  10. [18]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 2, 3

    2021 arXiv

  11. [19]

    ´Etude comparative de la distribution florale dans une portion des alpes et des jura

    Paul Jaccard. ´Etude comparative de la distribution florale dans une portion des alpes et des jura. Bull Soc Vaudoise Sci Nat, 37:547–579, 1901. 7

    1901

  12. [20]

    Text2human: Text-driven controllable human image generation

    Yuming Jiang, Shuai Yang, Haonan Qiu, Wayne Wu, Chen Change Loy, and Ziwei Liu. Text2human: Text-driven controllable human image generation. ACM Transactions on Graphics (TOG), 41(4):1–11, 2022. 2

    2022

  13. [21]

    Dense text-to-image generation with attention modulation

    Yunji Kim, Jiyoung Lee, Jin-Hwa Kim, Jung-Woo Ha, and Jun-Yan Zhu. Dense text-to-image generation with attention modulation. In ICCV, 2023. 2, 3, 5, 7

    2023

  14. [22]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 3, 5

    2013 arXiv

  15. [23]

    Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick. Segment anything. arXiv:2304.02643, 2023. 4

    2023 arXiv

  16. [24]

    Multi-concept customization of text-to-image diffusion

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi-concept customization of text-to-image diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1931–1941, 2023. 2, 3

    1931

  17. [25]

    Self- correction for human parsing

    Peike Li, Yunqiu Xu, Yunchao Wei, and Yi Yang. Self- correction for human parsing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020. 4

    2020

  18. [26]

    Gligen: Open-set grounded text-to-image generation

    Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jian- wei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded text-to-image generation. CVPR,

  19. [27]

    Image synthesis from layout with locality- aware mask adaption

    Zejian Li, Jingyu Wu, Immanuel Koh, Yongchuan Tang, and Lingyun Sun. Image synthesis from layout with locality- aware mask adaption. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pages 13819– 13828, 2021. 3

    2021

  20. [28]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499, 2023. 7 9

    2023 arXiv

  21. [29]

    Dress code: High- resolution multi-category virtual try-on, 2022

    Davide Morelli, Matteo Fincato, Marcella Cornia, Federico Landi, Fabio Cesari, and Rita Cucchiara. Dress code: High- resolution multi-category virtual try-on, 2022. 6, 7, 8

    2022

  22. [30]

    λ-eclipse: Multi-concept personalized text-to-image diffusion models by leveraging clip latent space

    Maitreya Patel, Sangmin Jung, Chitta Baral, and Yezhou Yang. λ-eclipse: Multi-concept personalized text-to-image diffusion models by leveraging clip latent space. arXiv preprint arXiv:2402.05195, 2024. 2, 3, 7, 8

    2024 arXiv

  23. [31]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  24. [32]

    High-resolution image syn- thesis with latent diffusion models, 2021

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models, 2021. 2, 3, 6

    2021

  25. [33]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 2250...

    2023

  26. [34]

    Humangan: A generative model of hu- man images

    Kripasindhu Sarkar, Lingjie Liu, Vladislav Golyanik, and Christian Theobalt. Humangan: A generative model of hu- man images. In 2021 International Conference on 3D Vision (3DV), pages 258–267. IEEE, 2021. 2

    2021

  27. [35]

    pytorch-fid: FID Score for PyTorch

    Maximilian Seitzer. pytorch-fid: FID Score for PyTorch. https://github.com/mseitzer/pytorch-fid ,

  28. [36]

    In- stantbooth: Personalized text-to-image generation without test-time finetuning

    Jing Shi, Wei Xiong, Zhe Lin, and Hyun Joon Jung. In- stantbooth: Personalized text-to-image generation without test-time finetuning. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 8543–8552, 2024. 2, 3

    2024

  29. [37]

    Image synthesis from reconfig- urable layout and style

    Wei Sun and Tianfu Wu. Image synthesis from reconfig- urable layout and style. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10531– 10540, 2019. 3

    2019

  30. [38]

    Object-centric image genera- tion from layouts

    Tristan Sylvain, Pengchuan Zhang, Yoshua Bengio, R Devon Hjelm, and Shikhar Sharma. Object-centric image genera- tion from layouts. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 2647–2655, 2021

    2021

  31. [39]

    Interactive image synthesis with panoptic layout generation

    Bo Wang, Tao Wu, Minfeng Zhu, and Peng Du. Interactive image synthesis with panoptic layout generation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7783–7792, 2022. 3

    2022

  32. [40]

    Instantid: Zero-shot identity-preserving generation in seconds

    Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, Anthony Chen, Huaxia Li, Xu Tang, and Yao Hu. Instantid: Zero-shot identity-preserving generation in seconds. arXiv preprint arXiv:2401.07519, 2024. 2, 3

    2024 arXiv

  33. [41]

    Instancediffusion: Instance-level control for image generation, 2024

    Xudong Wang, Trevor Darrell, Sai Saketh Rambhatla, Rohit Girdhar, and Ishan Misra. Instancediffusion: Instance-level control for image generation, 2024. 2, 3, 5, 7

    2024

  34. [42]

    Image quality assessment: from error visibility to structural similarity

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Si- moncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004. 4, 7

    2004

  35. [43]

    Elite: Encoding visual con- cepts into textual embeddings for customized text-to-image generation

    Yuxiang Wei, Yabo Zhang, Zhilong Ji, Jinfeng Bai, Lei Zhang, and Wangmeng Zuo. Elite: Encoding visual con- cepts into textual embeddings for customized text-to-image generation. arXiv preprint arXiv:2302.13848, 2023. 2, 3, 5, 7, 8

    2023 arXiv

  36. [44]

    Fastcomposer: Tuning-free multi- subject image generation with localized attention

    Guangxuan Xiao, Tianwei Yin, William T Freeman, Fr ´edo Durand, and Song Han. Fastcomposer: Tuning-free multi- subject image generation with localized attention. Interna- tional Journal of Computer Vision, pages 1–20, 2024. 2, 3

    2024

  37. [45]

    R&b: Region and boundary aware zero- shot grounded text-to-image generation

    Jiayu Xiao, Henglei Lv, Liang Li, Shuhui Wang, and Qing- ming Huang. R&b: Region and boundary aware zero- shot grounded text-to-image generation. arXiv preprint arXiv:2310.08872, 2023. 3

    2023 arXiv

  38. [46]

    Boxdiff: Text-to-image synthesis with training-free box-constrained diffusion

    Jinheng Xie, Yuexiang Li, Yawen Huang, Haozhe Liu, Wen- tian Zhang, Yefeng Zheng, and Mike Zheng Shou. Boxdiff: Text-to-image synthesis with training-free box-constrained diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7452–7461, 2023

    2023

  39. [47]

    Reco: Region-controlled text-to-image genera- tion

    Zhengyuan Yang, Jianfeng Wang, Zhe Gan, Linjie Li, Kevin Lin, Chenfei Wu, Nan Duan, Zicheng Liu, Ce Liu, Michael Zeng, et al. Reco: Region-controlled text-to-image genera- tion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 14246–14255,

  40. [48]

    Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models. arXiv preprint arXiv:2308.06721,

    arXiv

  41. [49]

    Customnet: Zero-shot object customization with variable-viewpoints in text-to-image dif- fusion models, 2023

    Ziyang Yuan, Mingdeng Cao, Xintao Wang, Zhongang Qi, Chun Yuan, and Ying Shan. Customnet: Zero-shot object customization with variable-viewpoints in text-to-image dif- fusion models, 2023. 2, 3, 5, 7, 8

    2023

  42. [50]

    Humandif- fusion: a coarse-to-fine alignment diffusion framework for controllable text-driven person image generation

    Kaiduo Zhang, Muyi Sun, Jianxin Sun, Binghao Zhao, Kunbo Zhang, Zhenan Sun, and Tieniu Tan. Humandif- fusion: a coarse-to-fine alignment diffusion framework for controllable text-driven person image generation. arXiv preprint arXiv:2211.06235, 2022. 2

    2022 arXiv

  43. [51]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018. 7

    2018

  44. [52]

    Layoutdiffusion: Controllable diffu- sion model for layout-to-image generation

    Guangcong Zheng, Xianpan Zhou, Xuewei Li, Zhongang Qi, Ying Shan, and Xi Li. Layoutdiffusion: Controllable diffu- sion model for layout-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 22490–22499, 2023. 2, 3

    2023

  45. [53]

    clip-score: CLIP Score for Py- Torch

    SUN Zhengwentai. clip-score: CLIP Score for Py- Torch. https : / / github . com / taited / clip - score, 2023. Version 0.1.1. 7

    2023

  46. [54]

    Migc: Multi-instance generation controller for text-to-image synthesis, 2024

    Dewei Zhou, You Li, Fan Ma, Xiaoting Zhang, and Yi Yang. Migc: Multi-instance generation controller for text-to-image synthesis, 2024. 2, 3, 5, 7 10

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.