Pith. sign in

REVIEW 4 major objections 5 minor 43 references

DreamCache: Finetuning-Free Lightweight Personalized Image Generation via Feature Caching

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read DreamCache claims that caching reference-image features from a single low-noise timestep of a frozen diffusion U-Net, plus 25 million adapter parameters, delivers state-of-the-art zero-shot personalized image generation without…

desk verdict DreamCache is a genuinely efficient and plausible personalization method, but the SOTA claim is too strong for its own numbers and the zero-shot generalization evidence is thinner than the abstract suggests. read the letter →

arxiv 2411.17786 v1 pith:RIPJLQD4 submitted 2024-11-26 cs.CV cs.AIcs.LG

Emanuele Aiello , Umberto Michieli , Diego Valsesia , Mete Ozay , Enrico Magli This is my paper · ORCID
classification cs.CVcs.AIcs.LG
keywords personalizedimagegenerationfeaturecachingzero-shotsubject-drivendiffusionmodelsattentionadaptersfinetuning-freepersonalizationsynthetictrainingdataStable
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes DreamCache, a method for personalized text-to-image generation that requires no fine-tuning at test time. It caches features of a reference image from a few layers of a frozen diffusion U-Net, computed once at a single low-noise timestep with no text prompt, then injects these cached features into the denoising process through small learned attention adapters. The claim is that this single cache carries enough subject identity to match or beat methods that process the reference image at every denoising step or use large image encoders, while adding only 25 million parameters (about 42 MB). If correct, personalized generation becomes nearly as cheap as ordinary text-to-image generation, which would matter for mobile and real-time deployment.

What carries the argument

The load-bearing object is the feature cache $\mathcal{H}_{\mathrm{FC}} = \{ h_{\mathrm{ref},L} : L \in \mathcal{L} \}$, where $h_{\mathrm{ref},L} = \epsilon_\theta(I_{\mathrm{ref}} + n_t, \varnothing, t; l)$ evaluated at $t=1$ and selected layers $\mathcal{L}$ (the middle bottleneck and every second decoder layer). This single-timestep, text-free cache decouples reference encoding from the iterative sampling process, so the reference is processed once rather than at every denoising step. The conditioning adapter then modulates the generated image's features via $q = W_Q h$, $k_c = W_K h_{\mathrm{ref}}$, $v_c = W_V h_{\mathrm{ref}}$, $a_c = \mathrm{softmax}(q k_c^T / \sqrt{d}) v_c$, $a = W_{\mathrm{proj}}([a; a_c])$, trained with the standard score-matching loss.

What would settle it

A controlled experiment comparing DreamCache on a real reference image versus a heavily stylized version of the same subject (for example, the same face rendered as an oil painting) would test whether the cache retains identity; a large DINO drop on the stylized input while text alignment stays high would show that the synthetic triplet training does not cover stylistic inputs.

Watch

Extended reading notes

Core claim

DreamCache's central claim is that a single forward pass through a pretrained Stable Diffusion denoiser, at timestep $t=1$ with a null prompt, yields a multi-resolution feature cache of the reference subject sufficient to condition personalized generation. The method uses a conditioning adapter composed of a cross-attention block between generated-image features and cached reference features, a concatenation of self- and cross-attention outputs, and a projection layer, inserted in the middle bottleneck and every second decoder layer of the U-Net. Trained on a synthetic dataset of roughly 400,000 triplets (caption, generated image, foreground-segmented reference), these adapters generalize zero-shot to real subjects, achieving DINO scores of 0.713 on SD 1.5 and 0.767 on SD 2.1 on the DreamBooth benchmark, with inference at about 3.88 seconds per image.

Load-bearing premise

The conditioning adapters learn subject identity from synthetic triplets in which the reference is an SD-XL-generated object pasted on white; their zero-shot transfer to arbitrary real user reference images, including stylized or abstract subjects, is validated only on 30 DreamBooth subjects and the paper explicitly flags abstract and stylistic images as an edge case.

Editorial extensions

If this is right

  • Personalization requires no user-provided caption for the reference image, since caching uses a null text prompt.
  • Because the U-Net is frozen and the cache is computed in one forward pass, the method can switch between personalized and non-personalized generation with no retraining, supporting plug-and-play deployment.
  • At about 3.88 seconds and 42 MB of extra storage per subject, DreamCache is an order of magnitude lighter than encoder-based and reference-based alternatives, making it viable on resource-constrained devices.
  • Foreground-masked metrics (MCLIP-I 0.906, MDINO 0.837) indicate the method preserves subject appearance without replicating the reference background.
  • Scaling the synthetic training set from 50K to 400K samples improves image alignment, indicating that the adapters benefit from more triplet data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same cache could in principle be reused across multiple outputs for one subject, so per-image cost should drop further when generating batches for the same reference — a batching extension the paper does not test.
  • Because caching happens at $t=1$ with a null prompt, the method treats the reference image as an unlabelled visual token; this suggests it could pair with retrieval-augmented generation, where a cache is fetched per prompt rather than per user.
  • The masked-metric gains over JeDi and ToffeeNet suggest that separating the cached subject from its background is doing much of the work; a testable extension is to replace the SAM foreground mask with a learned saliency predictor to handle non-segmentable subjects.
  • The method's reliance on a fixed low-noise timestep implies that the cache stores a fairly clean visual snapshot; feeding deliberately corrupted or stylized references at test time would stress whether the adapters have learned a general notion of subject identity beyond the synthetic training distribution.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper proposes DreamCache, a finetuning-free personalized image generation method that caches reference-image features from a subset of layers of a frozen Stable Diffusion U-Net at a single low-noise timestep, using a null text prompt. Lightweight attention-based conditioning adapters (25M parameters) inject these cached features into the denoiser during sampling, and the adapters are trained on a synthetically generated dataset of triplets (caption, target image, reference foreground on white background). The method is evaluated on DreamBooth across SD 1.5 and SD 2.1 backbones, with quantitative metrics (DINO, CLIP-I, CLIP-T), inference time, and ablations over conditioning placement, dataset size, and reference encoding. The authors claim state-of-the-art image and text alignment with an order-of-magnitude reduction in extra parameters and faster inference than reference-based counterparts.

Significance. If the stated performance holds, DreamCache is a useful practical contribution: it decouples reference processing from iterative sampling by caching once, requires only 42 MB of extra storage in FP16, and preserves the base model via plug-and-play adapters. The paper gives explicit credit for reproducibility by describing an open-source synthetic data pipeline that they plan to release, and the ablations (Tables 4, 5, 7, S2, S3) are informative about the design choices. However, the central novelty is incremental over BootPIG and IP-Adapter: it combines BootPIG's synthetic-triplet idea with a caching scheme that avoids per-step reference U-Net inference. The significance of the efficiency claim is real, but the 'state-of-the-art' quality claim is not fully supported by the evidence provided, and the zero-shot generalization risk is under-validated.

major comments (4)
  1. [Abstract; Section 4.1, Table 2] The abstract and Section 4.1 claim state-of-the-art image and text alignment, but Table 2 does not support this claim on a per-metric basis: Subject-Diffusion has a higher DINO score (0.771 vs. 0.767) and BootPig has a higher CLIP-T score (0.311 vs. 0.298 for DreamCache SD 1.5 and 0.301 for SD 2.1). The claim is defensible only as a better Pareto balance between image and text similarity, not as 'state-of-the-art' on both. The lack of error bars or significance tests makes this overstatement especially problematic, since the differences are within a few hundredths of a point. Please revise the claim to be metric-specific and add confidence intervals or significance tests for the main comparisons.
  2. [Section 3.3; Section 4.3; Table 7] The synthetic training data is generated in a closed loop: the reference image is the foreground of the target image pasted on white, so the adapter sees a perfect pixel-level alignment between the cached reference and the target foreground during training. This allows the model to learn a 'copy' shortcut rather than a generalizable identity-binding mechanism; at inference, no target image exists, and the shortcut would not apply. The only zero-shot evidence is the 30-subject DreamBooth benchmark, with no held-out split across domains and no images resembling the 'abstract or stylistic' references that the paper itself lists as a limitation in Section 5. Please provide stronger evidence of generalization: e.g., cross-dataset evaluation on a broader subject set, an analysis of whether attention maps exploit exact foreground alignment, or an ablation that adds a small foreground-distribution shift between training references and test references. This concern is load-bearing because the finetuning-free claim depends on true zero-shot transfer.
  3. [Section 4.2, Table 3; Implementation Details] The efficiency comparison in Table 3 is internally inconsistent: Section 4 states that DreamCache generates images with 50 sampling steps, but Table 3's footnote says the reported times are for 100 timesteps. It is unclear whether the DreamCache row uses 50 steps or 100 steps, and whether the baselines use their own step counts. Without this clarification, the reported speed advantage over BootPig and other methods is not a fair comparison. Please report wall-clock times under matched settings (same number of sampling steps, same GPU, same batch configuration) or explicitly state the step counts used for each method.
  4. [Section 4.3, Table 7; Section 3.3] The LAION-5M ablation in Table 7 is informative but does not adequately control for the shortcut confounding. The LAION condition reuses target images as references, which removes the pixel-identical foreground alignment, so the observed drop in CLIP-T (0.298 to 0.242) could be due to the loss of the copy shortcut, to the absence of triplet structure, or to the differing caption quality. Comparing synthetic scenes with the foreground object pasted on a new background (not just white) versus the current foreground-on-white reference would isolate the effect of pixel-identical foregrounds. This distinction is important because the paper uses the LAION comparison to argue for the triplet structure, but the current design conflates shortcut availability with triplet structure.
minor comments (5)
  1. [Table 2] The method name 'Jedi' should be 'JeDi' to match the reference and the rest of the text.
  2. [Tables 4 and 7] There are typographical comma-period inconsistencies (e.g., '0,282' in Table 4 and '0,301' in Table 7 should be decimal points). Please standardize the decimal separator.
  3. [Section 3.1, Eq. (2)] The definition of the noisy reference is slightly ambiguous: 'Iref + nt' is written without specifying that nt is sampled according to the forward diffusion process at timestep t=1. Clarify the notation so that nt is recognized as the t-loss noise realization.
  4. [Section 5, Limitations] The paper lists 'highly abstract or stylistic images' and 'multi-subject generation' as limitations, but these are not empirically characterized. It would be helpful to add a small qualitative or quantitative analysis of failure cases to substantiate the stated boundaries of the method.
  5. [Section 3.3] The paper mentions releasing the synthetic dataset and code, but no link or placeholder is provided. Adding a reproducibility URL would strengthen the paper's reproducibility claims.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: caching, adapter training, and DreamBooth evaluation form an independent derivation chain.

full rationale

The claimed derivation chain is not circular. Reference features H_FC are defined by forward passes of a frozen U-Net (Eqs. 1-2), the conditioning adapters are defined by Eqs. 3-5, and the adapter weights are optimized by the score-matching loss in Eq. 6. The training triplets used in Eq. 6 are generated externally to the method via Llama 3.2 captions, SD-XL images, and SAM/Grounding-DINO segmentations (Sec. 3.3). Evaluation is on DreamBooth subjects using standard DINO/CLIP metrics, with comparisons against external baselines (Table 2). No equation reduces the predicted output to the cached input by construction, and no load-bearing claim rests on a self-citation; the reference list contains no self-citations by the present authors. The pixel-overlap between reference and target in the synthetic training set is a legitimate generalization concern, but it is a training-distribution property, not a circular derivation: at inference the reference and target are different images, and the reported DreamBooth scores are empirical evidence rather than consequences of the training loss. Likewise, choosing hyperparameters via ablations on the same benchmark is a model-selection weakness, but it does not make the reported metrics equal to fitted inputs. Therefore none of the enumerated circularity classes is present.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on two domain assumptions: a single low-noise forward pass of the frozen denoiser provides a sufficient subject representation, and synthetic training triplets generalize to real references. The design hyperparameters (layer subset, timestep, guidance scales, dataset size, fusion type) are all chosen empirically on the evaluation benchmark rather than derived.

free parameters (5)
  • Cached layer subset L = middle bottleneck layer plus every second decoder layer
    Selected by ablation (Table 5) to maximize CLIP-I/CLIP-T on the DreamBooth evaluation set.
  • Caching timestep = t=1
    Selected by ablation (Table S3); t=1 gives best CLIP-I/CLIP-T on the evaluation set.
  • Guidance scales (combined CFG) = sT=7.5, sI varied (Fig S2)
    Chosen empirically; the paper explores the sampling space and does not fix a single operating point.
  • Synthetic dataset size = 400K triplets
    Chosen by scaling study (Table 7); 400K is the largest tested and gives best CLIP-I.
  • Adapter fusion type = Spatial Concat (concat of self- and cross-attention outputs)
    Selected by ablation (Table 4) as the best tradeoff of CLIP-I/CLIP-T and parameter count.
assumptions (5)
  • domain assumption Features of the frozen SD U-Net at timestep t=1 with null text prompt encode sufficient subject identity for personalization.
    Sec 3.1 and Table S3 validate t=1 empirically, but the paper provides no theoretical or mechanistic argument that a single low-noise forward pass captures identity across arbitrary subjects.
  • domain assumption The synthetic dataset of triplets (caption, target image, reference foreground) generated with SD-XL, Llama 3.2, and SAM/Grounding-DINO is a sufficient training distribution for zero-shot personalization on real images.
    Sec 3.3 describes the pipeline; Table 7 shows scaling helps, but generalization beyond 30 DreamBooth subjects is untested.
  • domain assumption DINO and CLIP cosine similarity scores are valid proxies for subject fidelity and text alignment.
    Section 4 uses these metrics; the paper itself notes CLIP-I is unreliable for Kosmos-G because of background interference, yet still relies on CLIP-I in ablations.
  • standard math The combined classifier-free guidance equation (Sec S4) correctly extends CFG to two conditioning modalities.
    Supp S4 gives the equation; it is a standard linear extrapolation assumption.
  • domain assumption Foreground segmentation of reference images is accurate enough that background features do not leak into the cache.
    The paper foreground-segments with SAM/Grounding DINO (Sec 3.1) and shows attention maps are localized, but segmentation errors on complex images would break the subject representation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DreamCache: Finetuning-Free Lightweight Personalized Image Generation via Feature Caching." pith.science (2026). https://pith.science/paper/RIPJLQD4

@misc{pith2026241117786,
  author       = {Pith},
  title        = {Pith review of: DreamCache: Finetuning-Free Lightweight Personalized Image Generation via Feature Caching},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RIPJLQD4}},
  note         = {Machine review of arXiv:2411.17786}
}
read the original abstract

Personalized image generation requires text-to-image generative models that capture the core features of a reference subject to allow for controlled generation across different contexts. Existing methods face challenges due to complex training requirements, high inference costs, limited flexibility, or a combination of these issues. In this paper, we introduce DreamCache, a scalable approach for efficient and high-quality personalized image generation. By caching a small number of reference image features from a subset of layers and a single timestep of the pretrained diffusion denoiser, DreamCache enables dynamic modulation of the generated image features through lightweight, trained conditioning adapters. DreamCache achieves state-of-the-art image and text alignment, utilizing an order of magnitude fewer extra parameters, and is both more computationally effective and versatile than existing models.

Figures

Figures reproduced from arXiv: 2411.17786 by the authors.

Figure 1
Figure 1. DreamCache is a finetuning-free personalized image generation method that achieves an optimal balance between sub￾ject fidelity, memory efficiency, and adherence to text prompts. Encoder-based methods [8, 16, 21, 36, 39] utilize dedi￾cated image encoders, such as CLIP [24] or DINO [3], to extract relevant features from reference images. While these encoders can produce high-quality results, they are often large, req… view at source ↗
Figure 2
Figure 2. Personalized generations by DreamCache. The first column contains reference images. The generated images correspond to the text prompts above each column. Textual Inversion [7]) or multiple (like in P+ [35]) input token embeddings. Later methods [1, 9, 33, 38] build on these, with innovations like Perfusion [33] using dynamic rank-1 updates to prevent overfitting while keeping encod￾ings lightweight. However, all fi… view at source ↗
Figure 3
Figure 3. Overview of DreamCache. Original U-Net layers are shown in violet, while the novel components introduced by DreamCache are highlighted in green. During personalization, features from selected layers of the diffusion denoiser are cached from a single timestep, using a null text prompt. These cached features serve as reference-specific information. During generation, conditioning adapters inject the cached features in… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Visual comparison. Personalized generations on sample concepts. DreamCache preserves reference concept appearance and does not suffer from background interference. BLIP-D [14] and Kosmos-G [20] cannot faithfully preserve visual details from the reference [PITH_FULL_IM…
Figure 5
Figure 5. Figure 5: Visualization of reference image impact. Cross￾attention maps between cached reference features and features of the image under generation. Left: attention map at layers at 16 × 16 resolution (left reference, right generated). Right: 32 × 32. Attention values are highl…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 19 canonical work pages

  1. [1]

    Moab Arar, Rinon Gal, Yuval Atzmon, Gal Chechik, Daniel Cohen-Or, Ariel Shamir, and Amit H. Bermano. Domain- agnostic tuning-encoder for fast personalization of text-to- image models. In SIGGRAPH Asia 2023 Conference Papers, pages 1–10, 2023. 1, 3

    work page 2023

  2. [2]

    In- structpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. In- structpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18392–18402, 2023. 2

    2023

  3. [3]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 1

    work page 2021

  4. [4]

    Re-imagen: Retrieval-augmented text-to-image gen- erator

    Wenhu Chen, Hexiang Hu, Chitwan Saharia, and William W Cohen. Re-imagen: Retrieval-augmented text-to-image gen- erator. In The Eleventh International Conference on Learning Representations, 2022. 6

    work page 2022

  5. [5]

    Subject-driven text-to-image generation via apprenticeship learning

    Wenhu Chen, Hexiang Hu, Yandong Li, Nataniel Rui, Xuhui Jia, Ming-Wei Chang, and William W Cohen. Subject-driven text-to-image generation via apprenticeship learning. arXiv preprint arXiv:2304.00186, 2023. 2, 3, 6

    arXiv 2023

  6. [6]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Ab- hishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783,

    arXiv

  7. [7]

    An image is worth one word: Personalizing text-to-image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit Haim Bermano, Gal Chechik, and Daniel Cohen-or. An image is worth one word: Personalizing text-to-image generation using textual inversion. In The Eleventh Interna- tional Conference on Learning Representations, 2022. 1, 2, 3, 6

    work page 2022

  8. [8]

    Encoder-based domain tuning for fast personalization of text-to-image models

    Rinon Gal, Moab Arar, Yuval Atzmon, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. Encoder-based domain tuning for fast personalization of text-to-image models. ACM Transactions on Graphics (TOG), 42(4):1–13, 2023. 1, 3

    work page 2023

Show all 43 references
  1. [9]

    Vico: Plug-and-play visual condition for personalized text-to- image generation

    Shaozhe Hao, Kai Han, Shihao Zhao, and Kwan-Yee K Wong. Vico: Plug-and-play visual condition for personalized text-to- image generation. arXiv preprint arXiv:2306.00971, 2023. 1, 3, 7, 8

    2023 arXiv

  2. [10]

    Clipscore: A reference-free evaluation metric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7514–7528, 2021. 6

    2021

  3. [11]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 1

    2020

  4. [12]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. arXiv preprint arXiv:2304.02643, 2023. 5

    2023 arXiv

  5. [13]

    Multi-concept customization of text- to-image diffusion

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shecht- man, and Jun-Yan Zhu. Multi-concept customization of text- to-image diffusion. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 1931–1941, 2023. 1, 2, 6

    1931

  6. [14]

    Blip- diffusion: Pre-trained subject representation for control- lable text-to-image generation and editing

    Dongxu Li, Junnan Li, and Steven CH Hoi. Blip- diffusion: Pre-trained subject representation for control- lable text-to-image generation and editing. arXiv preprint arXiv:2305.14720, 2023. 1, 2, 3, 6, 7, 5

    2023 arXiv

  7. [15]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499, 2023. 5

    2023 arXiv

  8. [16]

    Subject-diffusion: Open domain personalized text-to-image generation without test-time fine-tuning

    Jian Ma, Junhao Liang, Chen Chen, and Haonan Lu. Subject-diffusion: Open domain personalized text-to-image generation without test-time fine-tuning. arXiv preprint arXiv:2307.11410, 2023. 1, 2, 3, 5, 6, 7

    2023 arXiv

  9. [17]

    Learning-to-cache: Accelerating diffusion transformer via layer caching

    Xinyin Ma, Gongfan Fang, Michael Bi Mi, and Xinchao Wang. Learning-to-cache: Accelerating diffusion transformer via layer caching. arXiv preprint arXiv:2406.01733, 2024. 4

    2024 arXiv

  10. [18]

    Deepcache: Accelerating diffusion models for free

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. Deepcache: Accelerating diffusion models for free. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15762–15772, 2024. 4

    2024

  11. [19]

    Tadam: Task dependent adaptive metric for improved few-shot learning

    Boris Oreshkin, Pau Rodr ´ıguez L ´opez, and Alexandre La- coste. Tadam: Task dependent adaptive metric for improved few-shot learning. Advances in neural information processing systems, 31, 2018. 4

    2018

  12. [20]

    Kosmos-g: Generating images in con- text with multimodal large language models

    Xichen Pan, Li Dong, Shaohan Huang, Zhiliang Peng, Wenhu Chen, and Furu Wei. Kosmos-g: Generating images in con- text with multimodal large language models. arXiv preprint arXiv:2310.02992, 2023. 1, 2, 3, 6, 7, 5

    2023 arXiv

  13. [21]

    λ-eclipse: Multi-concept personalized text-to-image diffu- sion models by leveraging clip latent space

    Maitreya Patel, Sangmin Jung, Chitta Baral, and Yezhou Yang. λ-eclipse: Multi-concept personalized text-to-image diffu- sion models by leveraging clip latent space. arXiv preprint arXiv:2402.05195, 2024. 1, 3

    2024 arXiv

  14. [22]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 1

    2023 arXiv

  15. [23]

    Bootpig: Bootstrapping zero-shot personalized image generation capabilities in pretrained diffusion models

    Senthil Purushwalkam, Akash Gokul, Shafiq Joty, and Nikhil Naik. Bootpig: Bootstrapping zero-shot personalized image generation capabilities in pretrained diffusion models. arXiv preprint arXiv:2401.13974, 2024. 1, 2, 3, 5, 6, 7

    2024 arXiv

  16. [24]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  17. [25]

    Grounded sam: Assembling open-world models for diverse visual tasks

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al. Grounded sam: Assembling open-world models for diverse visual tasks. arXiv preprint arXiv:2401.14159,

    arXiv

  18. [26]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 5 9

    2022

  19. [27]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven gen- eration

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven gen- eration. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 2...

  20. [28]

    Pho- torealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Pho- torealistic text-to-image diffusion models with deep language understanding. Advances in Neural Informati...

    2022

  21. [29]

    Laion-5b: An open large-scale dataset for training next gen- eration image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next gen- eration image-text models. Advances in Neural Infor...

    2022

  22. [30]

    Prototypical networks for few-shot learning

    Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. Advances in neural informa- tion processing systems, 30, 2017. 4

    2017

  23. [31]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International confer- ence on machine learning, pages 2256–2265. PMLR, 2015. 1

    2015

  24. [32]

    Generative modeling by estimating gradients of the data distribution

    Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019. 5

    2019

  25. [33]

    Key-locked rank one editing for text-to-image personalization

    Yoad Tewel, Rinon Gal, Gal Chechik, and Yuval Atzmon. Key-locked rank one editing for text-to-image personalization. In ACM SIGGRAPH 2023 Conference Proceedings, pages 1–11, 2023. 1, 3

    2023

  26. [34]

    Meta-learning for semi-supervised few- shot classification

    Eleni Triantafillou, Hugo Larochelle, Jake Snell, Josh Tenen- baum, Kevin Jordan Swersky, Mengye Ren, Richard Zemel, and Sachin Ravi. Meta-learning for semi-supervised few- shot classification. In International Conference on Learning Representations, 2018. 4

    2018

  27. [35]

    p+: Extended textual conditioning in text-to-image generation

    Andrey V oynov, Qinghao Chu, Daniel Cohen-Or, and Kfir Aberman. p+: Extended textual conditioning in text-to-image generation. arXiv preprint arXiv:2303.09522, 2023. 1, 3

    2023 arXiv

  28. [36]

    Elite: Encoding visual concepts into textual embeddings for customized text-to-image generation

    Yuxiang Wei, Yabo Zhang, Zhilong Ji, Jinfeng Bai, Lei Zhang, and Wangmeng Zuo. Elite: Encoding visual concepts into textual embeddings for customized text-to-image generation. 2023 IEEE/CVF International Conference on Computer Vi- sion (ICCV), pages 15897–15907, 2023. 1, 2, 3, 5, 6, 7

    2023

  29. [37]

    Cache me if you can: Acceler- ating diffusion models through block caching

    Felix Wimbauer, Bichen Wu, Edgar Schoenfeld, Xiaoliang Dai, Ji Hou, Zijian He, Artsiom Sanakoyeu, Peizhao Zhang, Sam Tsai, Jonas Kohler, et al. Cache me if you can: Acceler- ating diffusion models through block caching. In Proceedings of the IEEE/CVF Conference on Computer Vis...

    2024

  30. [38]

    Controllable textual inversion for personalized text-to-image generation

    Jianan Yang, Haobo Wang, Yanming Zhang, Ruixuan Xiao, Sai Wu, Gang Chen, and Junbo Zhao. Controllable textual inversion for personalized text-to-image generation. arXiv preprint arXiv:2304.05265, 2023. 1, 3

    2023 arXiv

  31. [39]

    Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models. arXiv preprint arXiv:2308.06721,

    arXiv

  32. [40]

    Jedi: Joint- image diffusion models for finetuning-free personalized text- to-image generation

    Yu Zeng, Vishal M Patel, Haochen Wang, Xun Huang, Ting- Chun Wang, Ming-Yu Liu, and Yogesh Balaji. Jedi: Joint- image diffusion models for finetuning-free personalized text- to-image generation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recogni...

    2024

  33. [41]

    Cus- tomization assistant for text-to-image generation

    Yufan Zhou, Ruiyi Zhang, Jiuxiang Gu, and Tong Sun. Cus- tomization assistant for text-to-image generation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9182–9191, 2024. 2, 3, 6

    2024

  34. [42]

    Toffee: Efficient million-scale dataset construction for subject-driven text-to-image generation

    Yufan Zhou, Ruiyi Zhang, Kaizhi Zheng, Nanxuan Zhao, Jiux- iang Gu, Zichao Wang, Xin Eric Wang, and Tong Sun. Toffee: Efficient million-scale dataset construction for subject-driven text-to-image generation. arXiv preprint arXiv:2406.09305,

    arXiv

  35. [2024]

    A photo of [object], [object positioning in scene] [scene]

    1, 2, 3, 6 10 DreamCache: Finetuning-Free Lightweight Personalized Image Generation via Feature Caching Supplementary Material Table S1. Masked metrics quantitative evaluation. Method MCLIP-I ( ↑) MDINO ( ↑) DreamBooth 0.868 0.712 Custom Diffusion 0.864 0.711 JeDI 0.876 0.751 ...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.