Pith. sign in

REVIEW 4 major objections 7 minor 1 cited by

PatchDPO: Patch-level DPO for Finetuning-free Personalized Image Generation

T0 review · 4 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper introduces PatchDPO, an extra training stage that scores each image patch against the reference image and uses those scores as spatial training weights, improving pre-trained finetuning-free personalized generation models on…

desk verdict PatchDPO is a practical, internally supported booster for finetuning-free personalized generation, but the DPO label and uncontrolled SOTA comparison overstate what is shown. read the letter →

arxiv 2412.03177 v2 pith:NNVKMQSG submitted 2024-12-04 cs.CV

Qihan Huang , Weilong Dai , Jinlong Liu , Wanggui He , Hao Jiang , Mingli Song , Jie Song This is my paper · ORCID
classification cs.CV
keywords personalizedimagegenerationdirectpreferenceoptimizationpatch-levelqualitydiffusionmodelsfinetuning-freeself-supervisedfeaturelearningDreamBenchmulti-objectpersonalization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PatchDPO is an additional training stage for finetuning-free personalized image generation. Instead of comparing two whole images as traditional DPO does, it estimates the quality of each image patch by matching patch features between the generated image and the reference image, then uses those per-patch scores as weights in the diffusion training loss. The paper reports that this patch-level feedback improves multiple pre-trained personalized models and reaches state-of-the-art average scores on DreamBench for single-object generation and on Concept101 for multi-object generation. The practical point is that alignment feedback for image generation is more useful when it says where the image is wrong, not just that one image is better than another.

What carries the argument

The load-bearing object is the patch-quality map $p(x)\in\mathbb{R}^{H\times W}$, computed for each patch by Eq. (3) as $p(x_{\mathrm{gen}}[h,w])=\max_{i,j}\, \langle f(x_{\mathrm{gen}})[h,w], f(x_{\mathrm{ref}})[i,j]\rangle / (\|f(x_{\mathrm{gen}})[h,w]\|\,\|f(x_{\mathrm{ref}})[i,j]\|)$, the maximum cosine similarity between that patch's feature vector and every patch feature vector in the other image. The features come from a vision transformer (ViT) finetuned with a self-supervised loss that enforces consistency between an augmented image and the spatial transformation of the original's features, plus a regularization toward the frozen starting model; the authors select layer-7 features, which raise the HPatches matching score $S_{\mathrm{patch}}$ from 68.4% to 83.7%. This quality map drives the training loss as element-wise weights: the generated image is reconstructed with weight $\tilde{p}(x_{\mathrm{gen}})$ and the reference image with weight $1-\tilde{p}(x_{\mathrm{ref}})$, so high-quality generated patches are reinforced while reference patches that the generated image fails to reproduce are pushed in.

What would settle it

Run PatchDPO with the patch-quality weights randomly permuted across spatial positions, keeping their distribution intact, and compare DINO and CLIP-I on DreamBench with Table 4 row (5). If the improvement over the original IP-Adapter-Plus persists under permutation, the quality signal is not the causal driver and the gain comes from the extra reconstruction training on the synthetic dataset rather than from patch-level feedback.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that DPO-style alignment for personalized image generation fails when it compares whole images, because generated images typically diverge from the reference only in localized patches; global win/lose labels then reward bad regions of the winning image and penalize good regions of the losing image. PatchDPO replaces that global comparison with a per-patch quality estimate: each patch of the generated image is scored by its maximum cosine similarity to any patch of the reference image, and the resulting quality map is applied as spatial weights in a two-term reconstruction loss. The paper reports that this training recipe raises the DreamBench average of IP-Adapter-Plus from 0.600 to 0.619 (DINO 0.692 to 0.727, CLIP-I 0.826 to 0.838) and that the same recipe improves other pre-trained models and multi-object benchmarks such as Concept101.

Load-bearing premise

The load-bearing premise is that the maximum cosine similarity between a generated patch's learned features and any patch in the reference image correctly measures whether that patch is faithfully generated; the paper validates this on a generic patch-matching benchmark, not on patch quality in personalized generation.

Editorial extensions

If this is right

  • PatchDPO operates as an add-on training stage over an already-trained model, so any existing finetuning-free personalization model can be improved without changing its architecture or test-time procedure.
  • Per-patch feedback avoids the failure mode of whole-image DPO, where a winning image's poor regions are rewarded and a losing image's good regions are penalized.
  • The improvement concentrates in image-fidelity metrics (DINO and CLIP-I) while text alignment (CLIP-T) stays roughly flat, consistent with training pairs sharing the same text prompt.
  • Higher patch-matching accuracy on HPatches translates into larger generation gains, so further improvements to the patch-feature extractor should directly improve personalization quality.
  • The same weighted-training recipe transfers to multi-object personalization benchmarks, indicating that local patch feedback matters regardless of how many objects the prompt references.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One extension the paper leaves implicit is turning the same patch-quality map into preference pairs for a standard DPO loss, which would let per-patch weights and pairwise comparison be combined rather than treated as alternatives.
  • The quality heatmaps could serve as an interpretable user-facing diagnostic: they already highlight which regions of a generated image are not faithful to the reference, so they could be output alongside the image instead of only used internally during training.
  • Because the reference images in the training set are generated with clean backgrounds, the estimator may carry a bias toward simple scenes; testing on real user photos with cluttered backgrounds would show whether that bias limits the gain.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes PatchDPO, an additional training stage for finetuning-free personalized image generation. It constructs a synthetic dataset of reference/generated image pairs using ChatGPT and Stable Diffusion, estimates per-patch quality by matching patch features from a finetuned ViT between generated and reference images, and trains the personalized generation model with a weighted reconstruction loss that upweights high-quality generated patches and downweights low-quality ones, while also reconstructing the reference image with complementary weights. Experiments on DreamBench report improved DINO/CLIP-I over IP-Adapter-Plus and state-of-the-art averages, with ablations on training data, loss, and patch feature quality.

Significance. If the claims hold, PatchDPO is a useful post-hoc training recipe for finetuning-free personalization: it is model-agnostic in principle, releases code, validates the patch feature extractor on the external HPatches benchmark, and provides a controlled ablation (Table 4) showing gains over the base IP-Adapter-Plus. The significance is primarily practical rather than theoretical. The main-text evidence, however, is narrower than the abstract claims: only single-object results on IP-Adapter-Plus appear in full, with baselines copied from other papers, and the multi-object and multi-model claims are deferred to the appendix.

major comments (4)
  1. [Section 4.2.1, Eq. (3); Implementation Details] Eq. (3) defines patch quality as the maximum cosine similarity between a generated patch feature and any reference patch feature. This rewards a generated patch that matches a different object part exactly as much as one that matches the correct part, so the training signal can reinforce identity-mismatched content. The object-masking statement in Implementation Details ('only estimates the patch quality of object') is never operationalized: no mask source, resolution, or treatment of multi-object images is given. The HPatches evaluation in Table 1 validates patch retrieval against ground-truth correspondences, not the validity of p as a reward for personalized generation. Please specify the masking procedure and provide evidence that p correlates with perceived local fidelity (e.g., human ratings or correspondence-based evaluation) for this task.
  2. [Section 5.1, Tables 2 and 3] The claim of state-of-the-art performance rests on baseline numbers copied from the respective papers ('The results of baseline methods are from their paper'), which may use different CLIP/DINO versions, evaluation subsets, and sampling counts. The only controlled comparison is the IP-Adapter-Plus row in Table 4. Without a uniform protocol or, at minimum, error bars and significance tests across seeds, the margin over prior methods (e.g., Avg. 0.619 vs. 0.600) cannot be assessed as a genuine improvement. Please report baselines under a shared protocol or provide variance estimates for the claimed gains.
  3. [Section 1 and Section 5.1] The abstract and introduction claim state-of-the-art performance on multi-object personalized image generation, but the main text contains no quantitative multi-object results: Concept101 and MultiDreamBench numbers are deferred to appendix S2.2, and the claim of improving multiple pre-trained models (IP-Adapter, ELITE) is likewise deferred to appendix S2.3. The main text as written supports only single-object results on IP-Adapter-Plus. Please move these results into the main text or temper the claims.
  4. [Section 4.2.2, Eq. (4)] The self-supervised loss uses Aug(f(x)) without defining how an image augmentation acts on a feature map. For rotations, the feature-map transformation is ambiguous, and the equation cannot be reproduced from the text. Since this loss is the only training signal for the patch-quality extractor, please define Aug on feature maps precisely (including positional encoding handling) or provide the implementation in the appendix.
minor comments (7)
  1. [Section 5.1] Two consecutive headings both read 'Qualitative comparisons'; please rename the second heading, for example 'Multi-object qualitative comparison'.
  2. [Section 4.2.1] The text says the patch features are acquired from 'the last feature maps', while Table 1 and the final design use the 7th-layer features; please reconcile the wording.
  3. [Section 4.2.2] Eq. (4) uses the same symbol Aug for image augmentation and for the feature-map transformation; please use distinct notation.
  4. [Table 4] The names Dours and Dnatural are used before being introduced; define them in the table caption or in the surrounding text.
  5. [References] References [33] and [34] are the same paper; please remove the duplicate.
  6. [Section 4.3] The method does not actually use the DPO loss of Eq. (2); it uses a weighted reconstruction loss. Consider renaming the method or explicitly motivating why the name PatchDPO is retained.
  7. [Section 5.3] Figures 5 and 6 are referenced in non-sequential order; reorder the references for readability.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the patch-quality reward is an independent feature-similarity definition, and the reported gains are measured on external DreamBench and Concept101 benchmarks.

full rationale

The claimed derivation chain is not circular. PatchDPO consists of three independently specified stages: (i) constructing a synthetic dataset with ChatGPT prompts and Stable Diffusion reference images; (ii) defining patch quality via Eq. (3) as max cosine similarity between a generated patch feature and reference patch features, with the feature extractor improved by the self-supervised augmentation-consistency loss of Eq. (4) and validated on the external HPatches benchmark; (iii) fine-tuning the personalized generation model with the patch-weighted reconstruction loss in Section 4.3, whose weights are computed from those features. Nothing in the loss is equivalent to the evaluation metrics DINO, CLIP-I, or CLIP-T; those metrics are computed on DreamBench and Concept101 after training and are not used to fit any parameter of the reward or the model. The self-citations (refs. 13-16, 39, etc.) are contextual references to the authors' earlier interpretability and personalization papers; they do not supply a uniqueness theorem, ansatz, or fitted quantity on which the central result depends. A separate concern is that Eq. (3) can reward identity-mismatched patches and the object-masking step is unspecified; however, that is a correctness and reproducibility risk, not a circular reduction of the output to the input. Therefore no circularity step is exhibited, and the paper is self-contained against external benchmarks.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central claim rests on a chain of empirical assumptions rather than on classic fitted parameters. I list the hand-selected layer, the unspecified normalization and object-mask procedures, and the key domain assumptions about synthetic data transfer and max cosine patch similarity. No new physical or conceptual entities are introduced; PatchDPO is a training procedure, not a new object.

free parameters (4)
  • Layer index for patch features = layer 7 of ViT-Base
    Selected by the highest Spatch on HPatches (83.7 percent); this hand-picked design choice determines the quality scores that feed the training loss.
  • Patch quality normalization = unspecified, values normalized to [0, 1] via upsampling
    Section 4.3 says tilde p is upsampled from p with a normalization operation, but the exact operation is not given; the loss weights depend on it.
  • Training hyperparameters = LR 3e-5 for generation, LR 1e-1 for self-supervised training, 30,000 steps, batch size 4 per GPU on 8 GPUs, 50,000…
    Chosen by hand and not swept; these settings contribute to the reported results.
  • Object mask for restricting patch quality = not stated
    The paper says only object patch quality is estimated to remove background interference, but never describes how object masks are obtained.
assumptions (6)
  • standard math Diffusion noise-prediction MSE training and the DPO/RLHF objectives from the cited literature are accepted as background.
    Invoked in Section 3 as the base training objective and the motivation for preference-based alignment.
  • domain assumption Pre-trained and self-supervised-finetuned ViT feature map positions correspond to image patches.
    Used in Section 4.2.1 to define f(x)[h,w] as the feature of patch x[h,w].
  • domain assumption Maximum cosine similarity between a generated patch and any reference patch is a valid measure of patch generation quality.
    This is Eq. (3), the core of the patch-quality estimator and the reward signal for training.
  • domain assumption Minimizing the reweighted reconstruction loss improves generation fidelity rather than merely overfitting to the reference image.
    Underlies the model optimization stage in Section 4.3.
  • domain assumption Synthetic data from ChatGPT prompts plus Stable Diffusion references plus target model outputs transfers to real DreamBench and Concept101 images.
    The constructed D_ours dataset in Section 4.1 is the only training data used for the PatchDPO stage.
  • domain assumption HPatches Spatch is a valid proxy for patch-quality estimation accuracy in personalized generation.
    Used in Section 4.2.1 to select the layer and to justify the self-supervised training of the vision model.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PatchDPO: Patch-level DPO for Finetuning-free Personalized Image Generation." pith.science (2026). https://pith.science/paper/NNVKMQSG

@misc{pith2026241203177,
  author       = {Pith},
  title        = {Pith review of: PatchDPO: Patch-level DPO for Finetuning-free Personalized Image Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NNVKMQSG}},
  note         = {Machine review of arXiv:2412.03177}
}
read the original abstract

Finetuning-free personalized image generation can synthesize customized images without test-time finetuning, attracting wide research interest owing to its high efficiency. Current finetuning-free methods simply adopt a single training stage with a simple image reconstruction task, and they typically generate low-quality images inconsistent with the reference images during test-time. To mitigate this problem, inspired by the recent DPO (i.e., direct preference optimization) technique, this work proposes an additional training stage to improve the pre-trained personalized generation models. However, traditional DPO only determines the overall superiority or inferiority of two samples, which is not suitable for personalized image generation because the generated images are commonly inconsistent with the reference images only in some local image patches. To tackle this problem, this work proposes PatchDPO that estimates the quality of image patches within each generated image and accordingly trains the model. To this end, PatchDPO first leverages the pre-trained vision model with a proposed self-supervised training method to estimate the patch quality. Next, PatchDPO adopts a weighted training approach to train the model with the estimated patch quality, which rewards the image patches with high quality while penalizing the image patches with low quality. Experiment results demonstrate that PatchDPO significantly improves the performance of multiple pre-trained personalized generation models, and achieves state-of-the-art performance on both single-object and multi-object personalized image generation. Our code is available at https://github.com/hqhQAQ/PatchDPO.

Figures

Figures reproduced from arXiv: 2412.03177 by the authors.

Figure 1
Figure 1. The generated images (Images 1 & 2 & 3) are commonly [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. PatchDPO has three stages: (1) Data construction; (2) Patch quality estimation; (3) Model optimization. The stage (2) is split [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Qualitative comparisons of different methods on single-object & multi-object personalized image generation. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Qualitative ablation experiment. Sample 1 Sample 2 Sample 3 [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Selective Timestep Weighting and Advantage-Based Replay for Sample-Efficient Diffusion RLHF

    cs.LG 2026-07 conditional novelty 5.0 of 10

    Two plug-and-play strategies — per-timestep advantage weighting and advantage-based trajectory replay — improve diffusion RLHF sample efficiency up to 6× across five reward functions.

Reference graph

Works this paper leans on

48 extracted references · 36 canonical work pages · cited by 1 Pith paper

  1. [1]

    Computing re- ceptive fields of convolutional neural networks.Distill, 2019

    Andr ´e Araujo, Wade Norris, and Jack Sim. Computing re- ceptive fields of convolutional neural networks.Distill, 2019. https://distill.pub/2019/computing-receptive-fields. 5

    work page 2019

  2. [2]

    Con- stitutional ai: Harmlessness from ai feedback

    Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Con- stitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073, 2022. 3

    arXiv 2022

  3. [3]

    Hpatches: A benchmark and evaluation of handcrafted and learned local descriptors

    Vassileios Balntas, Karel Lenc, Andrea Vedaldi, and Krys- tian Mikolajczyk. Hpatches: A benchmark and evaluation of handcrafted and learned local descriptors. In CVPR 2017, pages 3852–3861. IEEE, 2017. 2, 4

    work page 2017

  4. [4]

    Training diffusion models with reinforce- ment learning

    Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforce- ment learning. In ICLR 2024. OpenReview.net, 2024. 3

    work page 2024

  5. [5]

    This looks like that: Deep learning for interpretable image recognition

    Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan Su. This looks like that: Deep learning for interpretable image recognition. In NeurIPS 2019, pages 8928–8939, 2019. 4

    work page 2019

  6. [6]

    Wenhu Chen, Hexiang Hu, Chitwan Saharia, and William W. Cohen. Re-imagen: Retrieval-augmented text-to-image gen- erator. In ICLR 2023. OpenReview.net, 2023. 6

    work page 2023

  7. [7]

    Kevin Clark, Paul Vicol, Kevin Swersky, and David J. Fleet. Directly fine-tuning diffusion models on differentiable re- wards. In ICLR 2024. OpenReview.net, 2024. 3

    work page 2024

  8. [8]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR 2009, pages 248–255. Ieee, 2009. 2

    work page 2009

Show all 48 references
  1. [9]

    Re- inforcement learning for fine-tuning text-to-image diffusion models

    Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Moham- mad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Re- inforcement learning for fine-tuning text-to-image diffusion models. In NeurIPS 2023, 2023. 3

    2023

  2. [10]

    An image is worth one word: Personalizing text-to-image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit Haim Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image generation using textual inversion. In ICLR 2023. OpenRe- view.net, 2023. 2, 6

    2023

  3. [11]

    Mix-of-show: Decentralized low-rank adaptation for multi-concept customization of diffusion models

    Yuchao Gu, Xintao Wang, Jay Zhangjie Wu, Yujun Shi, Yun- peng Chen, Zihan Fan, Wuyou Xiao, Rui Zhao, Shuning Chang, Weijia Wu, Yixiao Ge, Ying Shan, and Mike Zheng Shou. Mix-of-show: Decentralized low-rank adaptation for multi-concept customization of diffusion models. In Neur...

    2023

  4. [12]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. In NeurIPS 2020, 2020. 3

    2020

  5. [13]

    Evaluation and improvement of interpretability for self-explainable part- prototype networks

    Qihan Huang, Mengqi Xue, Wenqi Huang, Haofei Zhang, Jie Song, Yongcheng Jing, and Mingli Song. Evaluation and improvement of interpretability for self-explainable part- prototype networks. In ICCV 2023, pages 2011–2020. IEEE,

    2023

  6. [14]

    Resolving multi-condition confusion for finetuning-free personalized image generation

    Qihan Huang, Siming Fu, Jinlong Liu, Hao Jiang, Yipeng Yu, and Jie Song. Resolving multi-condition confusion for finetuning-free personalized image generation. arXiv preprint arXiv:2409.17920, 2024. 5

    2024 arXiv

  7. [15]

    On the concept trustworthiness in concept bottleneck models

    Qihan Huang, Jie Song, Jingwen Hu, Haofei Zhang, Yong Wang, and Mingli Song. On the concept trustworthiness in concept bottleneck models. In AAAI 2024 , pages 21161– 21168, 2024. 4

    2024

  8. [16]

    LG-CA V: train any concept activation vector with lan- guage guidance

    Qihan Huang, Jie Song, Mengqi Xue, Haofei Zhang, Bingde Hu, Huiqiong Wang, Hao Jiang, Xingen Wang, and Mingli Song. LG-CA V: train any concept activation vector with lan- guage guidance. In NeurIPS 2024, 2024. 4

    2024

  9. [17]

    Perceiver: General perception with iterative attention

    Andrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals, Andrew Zisserman, and Jo ˜ao Carreira. Perceiver: General perception with iterative attention. In ICML 2021 , pages 4651–4664. PMLR, 2021. 6

    2021

  10. [18]

    Taming encoder for zero fine-tuning image customization with text-to-image diffusion models

    Xuhui Jia, Yang Zhao, Kelvin CK Chan, Yandong Li, Han Zhang, Boqing Gong, Tingbo Hou, Huisheng Wang, and Yu-Chuan Su. Taming encoder for zero fine-tuning image customization with text-to-image diffusion models. arXiv preprint arXiv:2304.02642, 2023. 2

    2023 arXiv

  11. [19]

    Turning frequency to resolution: Video super-resolution via event cameras

    Yongcheng Jing, Yiding Yang, Xinchao Wang, Mingli Song, and Dacheng Tao. Turning frequency to resolution: Video super-resolution via event cameras. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7772–7781, 2021. 5

    2021

  12. [20]

    Deep graph reprogramming

    Yongcheng Jing, Chongbin Yuan, Li Ju, Yiding Yang, Xin- chao Wang, and Dacheng Tao. Deep graph reprogramming. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24345–24354, 2023. 5

    2023

  13. [21]

    Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross B

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chlo´e Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross B. Girshick. Segment anything. In ICCV 2023, pages 3992–4003. IEEE, 2023. 7

    2023

  14. [22]

    Multi-concept customization of text-to-image diffusion

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi-concept customization of text-to-image diffusion. InCVPR 2023, pages 1931–1941. IEEE, 2023. 1, 2, 6

    2023

  15. [23]

    Dongxu Li, Junnan Li, and Steven C. H. Hoi. Blip-diffusion: Pre-trained subject representation for controllable text-to- image generation and editing. In NeurIPS 2023, 2023. 2, 6

    2023

  16. [24]

    Cones: Concept neurons in diffusion models for customized generation

    Zhiheng Liu, Ruili Feng, Kai Zhu, Yifei Zhang, Kecheng Zheng, Yu Liu, Deli Zhao, Jingren Zhou, and Yang Cao. Cones: Concept neurons in diffusion models for customized generation. In ICML 2023 , pages 21548–21566. PMLR,

    2023

  17. [25]

    Wenjie Luo, Yujia Li, Raquel Urtasun, and Richard S. Zemel. Understanding the effective receptive field in deep convolutional neural networks. In NeurIPS, pages 4898– 4906, 2016. 5

    2016

  18. [26]

    Subject- diffusion: Open domain personalized text-to-image gener- ation without test-time fine-tuning

    Jian Ma, Junhao Liang, Chen Chen, and Haonan Lu. Subject- diffusion: Open domain personalized text-to-image gener- ation without test-time fine-tuning. In SIGGRAPH 2024 , page 25. ACM, 2024. 1, 2, 6

    2024

  19. [27]

    Teaching language models to support answers with verified quotes

    Jacob Menick, Maja Trebacz, Vladimir Mikulik, John Aslanides, Francis Song, Martin Chadwick, Mia Glaese, Su- sannah Young, Lucy Campbell-Gillingham, Geoffrey Irving, et al. Teaching language models to support answers with verified quotes. arXiv preprint arXiv:2203.11147, 2022. 3

    2022 arXiv

  20. [28]

    Kosmos-g: Generating images in context with multimodal large language models

    Xichen Pan, Li Dong, Shaohan Huang, Zhiliang Peng, Wenhu Chen, and Furu Wei. Kosmos-g: Generating images in context with multimodal large language models. In ICLR

  21. [29]

    lambda-eclipse: Multi-concept personalized text-to- image diffusion models by leveraging clip latent space.arXiv preprint arXiv:2402.05195, 2024

    Maitreya Patel, Sangmin Jung, Chitta Baral, and Yezhou Yang. lambda-eclipse: Multi-concept personalized text-to- image diffusion models by leveraging clip latent space.arXiv preprint arXiv:2402.05195, 2024. 6

    2024 arXiv

  22. [30]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 6

    2023 arXiv

  23. [31]

    Aligning text-to-image diffusion models with reward backpropagation

    Mihir Prabhudesai, Anirudh Goyal, Deepak Pathak, and Katerina Fragkiadaki. Aligning text-to-image diffusion models with reward backpropagation. arXiv preprint arXiv:2310.03739, 2023. 3

    2023 arXiv

  24. [32]

    Manning, Stefano Ermon, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christo- pher D. Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In NeurIPS 2023, 2023. 1, 3, 4

    2023

  25. [33]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR 2022, pages 10674–10685. IEEE, 2022. 2

    2022

  26. [34]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR 2022, pages 10674–10685. IEEE, 2022. 3

    2022

  27. [35]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In CVPR 2023 , pages 22500–22510. IEEE,

    2023

  28. [36]

    Instant- booth: Personalized text-to-image generation without test- time finetuning

    Jing Shi, Wei Xiong, Zhe Lin, and Hyun Joon Jung. Instant- booth: Personalized text-to-image generation without test- time finetuning. In CVPR 2024, pages 8543–8552. IEEE,

    2024

  29. [37]

    Generative multimodal models are in-context learners

    Quan Sun, Yufeng Cui, Xiaosong Zhang, Fan Zhang, Qiy- ing Yu, Yueze Wang, Yongming Rao, Jingjing Liu, Tiejun Huang, and Xinlong Wang. Generative multimodal models are in-context learners. In CVPR 2024, pages 14398–14409. IEEE, 2024. 6

    2024

  30. [38]

    Diffusion model align- ment using direct preference optimization

    Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model align- ment using direct preference optimization. In CVPR 2024, pages 8228–8238. IEEE, 2024. 3, 7

    2024

  31. [39]

    Ms-diffusion: Multi-subject zero-shot im- age personalization with layout guidance

    X Wang, Siming Fu, Qihan Huang, Wanggui He, and Hao Jiang. Ms-diffusion: Multi-subject zero-shot im- age personalization with layout guidance. arXiv preprint arXiv:2406.07209, 2024. 6

    2024 arXiv

  32. [40]

    ELITE: encoding visual con- cepts into textual embeddings for customized text-to-image generation

    Yuxiang Wei, Yabo Zhang, Zhilong Ji, Jinfeng Bai, Lei Zhang, and Wangmeng Zuo. ELITE: encoding visual con- cepts into textual embeddings for customized text-to-image generation. In ICCV 2023, pages 15897–15907. IEEE, 2023. 2, 6

    2023

  33. [41]

    Fastcomposer: Tuning-free multi- subject image generation with localized attention

    Guangxuan Xiao, Tianwei Yin, William T Freeman, Fr ´edo Durand, and Song Han. Fastcomposer: Tuning-free multi- subject image generation with localized attention. arXiv preprint arXiv:2305.10431, 2023. 2

    2023 arXiv

  34. [42]

    Omnigen: Unified image generation

    Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xin- grun Xing, Ruiran Yan, Shuting Wang, Tiejun Huang, and Zheng Liu. Omnigen: Unified image generation. arXiv preprint arXiv:2409.11340, 2024. 6

    2024 arXiv

  35. [43]

    SFD2: semantic-guided feature detection and description

    Fei Xue, Ignas Budvytis, and Roberto Cipolla. SFD2: semantic-guided feature detection and description. In CVPR 2023, pages 5206–5216. IEEE, 2023. 4

    2023

  36. [44]

    Protopformer: Con- centrating on prototypical parts in vision transformers for in- terpretable image recognition

    Mengqi Xue, Qihan Huang, Haofei Zhang, Jingwen Hu, Jie Song, Mingli Song, and Canghong Jin. Protopformer: Con- centrating on prototypical parts in vision transformers for in- terpretable image recognition. In IJCAI 2024, pages 1516–

    2024

  37. [45]

    Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models. arXiv preprint arXiv:2308.06721,

    arXiv

  38. [46]

    Patel, Haochen Wang, Xun Huang, Ting- Chun Wang, Ming-Yu Liu, and Yogesh Balaji

    Yu Zeng, Vishal M. Patel, Haochen Wang, Xun Huang, Ting- Chun Wang, Ming-Yu Liu, and Yogesh Balaji. Jedi: Joint- image diffusion models for finetuning-free personalized text- to-image generation. In CVPR 2024 , pages 6786–6795. IEEE, 2024. 1, 2, 6

    2024

  39. [47]

    Ssr-encoder: Encoding selective subject representation for subject-driven generation

    Yuxuan Zhang, Yiren Song, Jiaming Liu, Rui Wang, Jinpeng Yu, Hao Tang, Huaxia Li, Xu Tang, Yao Hu, Han Pan, et al. Ssr-encoder: Encoding selective subject representation for subject-driven generation. In CVPR 2024, pages 8069–8078. IEEE, 2024. 2, 6

    2024

  40. [2024]

    OpenReview.net, 2024. 6

    2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.