Pith. sign in

REVIEW 4 major objections 6 minor 65 references

Opt-In Art: Learning Art Styles Only from Few Examples

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A diffusion model trained exclusively on photographs can learn to imitate a painter's style from as few as nine examples, matching models whose training data contained hundreds of millions of art images.

desk verdict A well-executed empirical study with a genuinely new setup and a real result, but the style-generalization claim rests on circular evaluation; it deserves peer review with requests for held-out tests. read the letter →

arxiv 2412.00176 v3 pith:3Y55EHJX submitted 2024-11-29 cs.CV

Hui Ren , Joanna Materzynska , Rohit Gandikota , David Bau , Antonio Torralba This is my paper · ORCID
classification cs.CV
keywords artisticstyletransferfew-shotadaptationdiffusionmodelsLoRAfine-tuningdataattributionopt-inconsenttext-to-imagegenerationphotography-onlytraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a text-to-image model must ever see paintings to learn an artistic style. The authors build Blank Canvas Diffusion, a latent diffusion model trained only on about 9 million photographs that survive a two-stage art filter, then fine-tune a lightweight LoRA adapter on 9 to 50 curated paintings from an artist. They report that this adapted model matches Stable Diffusion 1.4, a model whose training set is estimated to contain over 190 million art-like images, on both automatic style metrics and user studies of style resemblance. The result matters because it suggests artistic style generation does not require massive art-rich pretraining, opening a path to opt-in style licensing while also warning that a handful of examples can reproduce a style.

What carries the argument

The argument rests on two engineered components. First, the Blank Canvas Dataset and Diffusion model: a two-stage filter (caption keyword blocking plus CLIP cosine-similarity thresholding against art-related terms) removes paintings and other visual art from the SAM-LLava-Captions10M photo corpus, and the VAE and U-Net are trained from scratch on the remaining photographs with a BERT text encoder, deliberately avoiding CLIP so no visual art knowledge leaks through the text embedding. Second, the Art Style Adapter: a LoRA fine-tune applied only to the U-Net's up blocks, trained with a style loss $\|\epsilon_{\theta\cup\theta'}(X_t, C^*, t) - \epsilon\|^2$ and a content loss $\|\epsilon_{\theta\cup\theta'}(X_t, C, t) - \epsilon_\theta(X_t, C, t)\|^2$, where $C^* = C + \text{''in the style of } V^* \text{ art''}$; the content loss keeps style and content disentangled, and at inference the adapter's injection timestep controls stylization strength. This combination isolates style as a low-rank residual that can be learned from a few examples.

What would settle it

A concrete experiment: take two identical latent-diffusion backbones trained on the same 9 million photographs, but add 5% of the very art images the filter excludes to one version's pretraining; apply the same LoRA adapter for a fixed style (say, van Gogh) to both and compare CSD style scores. If the art-added model scores materially higher, prior artistic exposure is doing work the paper denies; if the scores are flat, the no-art-prior claim holds. A sharper probe is to remove the 0.71% residual art images from the photo-only training set and re-run adaptation: under the paper's thesis, style fidelity should be unchanged.

Watch

Extended reading notes

Core claim

The paper's central claim is that artistic style is not a latent capacity that requires large-scale prior exposure to paintings; it can be assembled from a photograph-only backbone plus a small, carefully selected set of style exemplars. Post-adaptation, Blank Canvas Diffusion with the Art Style Adapter achieves style fidelity comparable to Stable Diffusion 1.4 on image stylization and artistic generation, and controlled experiments that fix architecture and data scale show no significant difference between models pretrained on art-containing versus art-free 100,000-image subsets (CSD 0.32 versus 0.35). Training-free style transfer methods such as StyleAligned work well on Stable Diffusion but fail on Blank Canvas Diffusion, whereas the adapter succeeds on the art-free backbone, indicating that the LoRA adapter supplies the style signal that the pretraining withheld. Data attribution analysis shows that both natural photographs and the few style examples contribute to the generated images, supporting the view that style emerges from reorganizing photographic priors rather than from memorizing paintings.

Load-bearing premise

The claim depends on the assumption that the photograph-only training set truly contains no meaningful artistic style prior, but manual inspection found 71 art-related images (0.71%) in a 10,000-image sample and the paper concedes that photography itself carries aesthetic choices, so 'no prior exposure to paintings' is an approximation.

Editorial extensions

If this is right

  • Art-style licensing can become truly opt-in: a base model can be trained with no paintings at all, and a rights-holder can approve a small set of reference works that a LoRA adapter then turns into a usable style.
  • Training-data restrictions alone will not prevent style replication, since 9 to 50 examples suffice to mimic a style; the paper argues copyright discussions must therefore address attribution and consent rather than only dataset curation.
  • Training-free style-transfer methods (StyleAligned, StyleID) succeed on art-rich backbones and fail on Blank Canvas Diffusion, showing that those methods exploit stylistic priors baked into pretraining; the adapter closes that gap without such priors.
  • The content loss (weight $w=50$) lifts style score from 0.14 to 0.29 on the stylization task, demonstrating that separating style from content is what makes few-example style learning stable and transferable.
  • A controlled pretraining experiment with fixed architecture and 100,000-image datasets (art-free versus art-containing) yields nearly identical adaptation quality, isolating prior art exposure as irrelevant to the adapter's success.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own statistics—71 art-related images (0.71%) in a 10,000-sample manual inspection of the filtered dataset, predominantly sculptures—mean 'no prior art' is an approximation; a sharper test would measure style fidelity as the residual art fraction is varied toward zero, predicting a flat curve if the thesis holds.
  • The data attribution results suggest the mechanism is style as a transformation of photographic priors rather than memory of paintings; a direct probe would be to train the adapter on a backbone pretrained on synthetic non-scene images (textures, noise) and check whether any artistic style survives, which would reveal how much of the effect depends on real-world photographic structure.
  • The finding that a deterministic adapter on an art-free backbone rivals an art-trained model complicates the 'memorization' narrative of diffusion models: if style can be composed from natural-image priors plus a few examples, then resemblance to an artist may not require any copying of the artist's works, which would shift legal and technical debates from training-data provenance to the few-shot
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper investigates whether a text-to-image diffusion model pretrained exclusively on photographs can learn an artistic style from a small number of examples. The authors construct Blank Canvas Diffusion, a latent diffusion model trained from scratch on the Blank Canvas Dataset, a filtered subset of SAM-LLaVA-Captions10M from which captions and CLIP scores are used to remove paintings and other non-photographic art. Style adaptation is performed with a LoRA adapter trained on 9 to 50 manually curated paintings per artist using a style loss plus a content-preservation loss. Evaluation uses the Contrastive Style Descriptor (CSD) against the adaptation set, content metrics (ViT-c, CLIP), a Mechanical Turk user study, and data attribution. The central claim is that a model with no prior exposure to paintings can match the style fidelity of Stable Diffusion 1.4, suggesting that artistic style generation is not contingent on artistic pretraining.

Significance. If the central claim holds, the paper is significant for two communities: it challenges the assumption that artistic style generation requires large-scale exposure to art in pretraining, and it informs copyright/opt-in debates by showing that few-shot adaptation can replicate a style. The paper has several genuine strengths: the pretraining data is deliberately and transparently filtered, the VAE and U-Net are trained from scratch with a language-only text encoder to avoid art leakage from CLIP, the authors report a manual audit of residual art content, they include a controlled filtering experiment (Section 6.2), and they supplement automatic metrics with a user study and a real-artist interview. These choices make the empirical setup more informative than a typical style-transfer paper. However, the evaluation is currently too weak to support the strongest form of the claim, because style fidelity is measured against the same images used for adaptation and no held-out-artist evaluation is reported.

major comments (4)
  1. [Section 6.2 (CSD metric)] The automatic style metric is computed as the mean CSD score between a generated image and images from the Art-Adaptation set, which is the same set used to train the LoRA adapter via Eq. (3). This makes the style-fidelity evaluation circular: a model that overfits or memorizes the adaptation examples can score high without generalizing to unseen works by the same artist. The abstract's phrase 'generalize an artistic style' (Fig. 1) therefore is not actually tested. Please add a held-out evaluation using paintings by the same artist that were not used for adapter training, and compare against a memorization baseline (e.g., a model that retrieves or copies the closest training example).
  2. [Section 6.2 (user study)] The user study asks participants to compare outputs against 'three reference images showing the style of an artist,' but the paper does not state whether these reference images are disjoint from the adapter training set. If they are the same as the training examples, the preference results face the same circularity as the CSD metric. Please specify this explicitly and, ideally, run the study with held-out reference artworks so that the question is about style generalization rather than example matching.
  3. [Abstract, Section 4, Section 7] The claim of 'no prior exposure to paintings' or 'without access to any painting-related content' is stronger than the evidence. Table 1 reports that manual inspection of a 10,000-image sample of the final Blank Canvas Dataset still finds 71 art-related images (0.71%), and Section 7 concedes that photography itself carries inherent artistic style. The BERT text encoder also has linguistic knowledge of art concepts. I recommend qualifying the claim to 'minimal exposure to non-photographic art' and adding an analysis (e.g., removing the residual 0.71% or measuring sensitivity to them) to show that the residual art does not drive the adaptation results.
  4. [Section 6.2 (Data Attribution)] The data attribution results are interpreted as evidence that the adapter 'generalizes' and that natural images contribute to style generation, but the top attributed images in Fig. 9 frequently include Art-Style Examples, which is exactly what would be expected if the generated image is close to the training set. Without a baseline (e.g., attribution for a model trained to copy the adaptation set, or comparison of attribution to held-out works), the data attribution does not distinguish style abstraction from example memorization. This concern does not invalidate the method, but it means the attribution analysis does not currently support the generalization claim.
minor comments (6)
  1. [Table 1 and Section 4.1] The text says 'Post-filtering analysis ... revealed only 72 images' while Table 1 reports 71; please align the numbers.
  2. [Equation (1)] The noise term should be √(β_t)ε under the standard DDPM parameterization; as written, the variance of the added noise is β_t^2, not β_t.
  3. [Figure 5 caption] The caption appears garbled (e.g., '?㗃!'), likely a rendering issue; please ensure the final PDF displays the equation and loss terms correctly.
  4. [Section 6.2 (few-shot curation)] The manual curation of '9 to 50 paintings ... with similar color composition, brushstroke techniques, and artistic content' makes the adaptation set unusually homogeneous; please discuss how the results might change if the few-shot examples are less curated or more diverse.
  5. [Section 6.2 (user study details)] Please specify in the user study description how many reference images were shown per artist and how many evaluation rounds each worker completed, so that the reported percentages can be interpreted.
  6. [Section 6.2 (baseline description)] The statement that 'baselines rely on extensive Stable Diffusion 1.4 training on large art-rich datasets' is accurate but should also acknowledge that StyleAligned uses a single reference image while the proposed adapter uses multiple style references; Section K already mentions this, but it should be stated in the main evaluation.

Circularity Check

1 steps flagged · score 6.0 of 10

Style-fidelity evaluation is computed against the same Art-Adaptation set used to train the adapter, so the quantitative 'generalization' claim is partially circular.

  1. fitted input called prediction [Sec. 5, Eq. (3) and Sec. 6.2, 'Art Style Adaptation']
    "For a given target artistic image, we define the following loss: LS(θ′) =∥ϵθ∪θ′(Xt, C∗, t) − ϵ|∥2 , (3) ... We refer to this loss as style loss. ... To evaluate art style similarity, we use the Contrastive Style Descriptor (CSD) [54] ... For each sample, we compute the mean CSD score between a generated image and images from the Art-Adaptation set."

    The Art-Style Adapter is optimized on the Art-Adaptation set A (Eq. 3), and the headline automatic style metric is the mean CSD between a generated image and images from that same set A. A model that overfits or memorizes its 9-50 training paintings will therefore score high on CSD without demonstrating style generalization. The paper explicitly claims generalization ('we find that it is possible to adapt a model that is trained without paintings to generalize an artistic style'), but no held-out same-artist images, no cross-artist discrimination test, and no memorization baseline are reported.

full rationale

The paper is largely self-contained: Blank Canvas Diffusion is trained from scratch on a filtered photograph dataset, the LoRA adapter is a standard open adaptation method, and comparisons to CommonCanvas, SD1.4, StyleAligned, and other baselines are external. No load-bearing self-citation chain or imported uniqueness theorem is present. The central circularity is confined to the style-fidelity evaluation: the quantitative CSD score and, in all likelihood, the user-study references are drawn from the same small Art-Adaptation set on which the adapter was trained. That makes the 'style score' a training-set-similarity measure by construction, so the paper's quantitative claim of matching art-trained models on style is not a held-out test of style generalization. The qualitative figure-level outputs, the data-attribution analysis, and the Alan Kenny interview do provide some independent evidence that a style has been acquired, which prevents a score of 8 or 10. A held-out same-artist evaluation or memorization baseline would remove the circularity.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The central claim depends less on invented entities than on the completeness of the art filter and the choice of evaluation metrics. The free parameters listed above are all tuned or manually chosen, with the style example set being the most consequential for the reported few-shot performance.

free parameters (8)
  • Content loss weight w = 50
    Selected from a sweep; w=20,50,100 give similar CSD, w=0 much worse, so the exact value is not critical but it is tuned.
  • LoRA rank = 1
    Rank 1 outperforms rank 64 on style score (CSD 0.29 vs 0.21); the choice is empirical.
  • LoRA scale = 1.0
    Default scale used in all experiments.
  • CLIP art-filter threshold = 17
    Chosen by manually inspecting images ranked by CLIP score; threshold set to remove art-related content.
  • Adapter start timestep = 800 for image stylization, 0 for artistic generation
    Timestep for injecting adapter is manually chosen based on qualitative inspection; earlier injection gives more style.
  • Style example set size = 9 to 50 per artist, average 21.88
    Examples are manually curated for consistent style, not a random sample.
  • Adapter training steps = 1,000
    Fixed training schedule for all adapters.
  • VAE training mixture = 104,145 SAM images plus 115,294 filtered COCO images
    Arbitrary mixture used to improve VAE reconstruction; not justified from first principles.
assumptions (5)
  • ad hoc to paper The two-stage filtering (caption keywords plus CLIP similarity below threshold 17) removes essentially all non-photographic visual art from the training set.
    This is the load-bearing premise for the 'no prior exposure to paintings' claim; the paper's own manual inspection found 71 art-related images (0.71%) in a 10k sample after filtering, so the axiom holds only approximately.
  • domain assumption BERT text encoder contains no visual or pixel-level art knowledge.
    Used to avoid art leakage through text embeddings; BERT has linguistic knowledge of art, but the paper argues it has no visual art knowledge.
  • domain assumption SA-1B images are photographs captured by cameras and are suitable as an art-free pretraining source.
    The paper relies on the origin of SA-1B to claim a photographic base, then filters residual art.
  • domain assumption Contrastive Style Descriptor (CSD) is a valid measure of artistic style similarity.
    CSD is used as the main automatic style metric; its validity for this purpose is taken from prior work.
  • domain assumption The user study provides reliable judgments after filtering out workers who failed the reliability task.
    Self-reported filtering of crowdworkers is assumed to remove bias toward photographic images.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Opt-In Art: Learning Art Styles Only from Few Examples." pith.science (2026). https://pith.science/paper/3Y55EHJX

@misc{pith2026241200176,
  author       = {Pith},
  title        = {Pith review of: Opt-In Art: Learning Art Styles Only from Few Examples},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3Y55EHJX}},
  note         = {Machine review of arXiv:2412.00176}
}
read the original abstract

We explore whether pre-training on datasets with paintings is necessary for a model to learn an artistic style with only a few examples. To investigate this, we train a text-to-image model exclusively on photographs, without access to any painting-related content. We show that it is possible to adapt a model that is trained without paintings to an artistic style, given only few examples. User studies and automatic evaluations confirm that our model (post-adaptation) performs on par with state-of-the-art models trained on massive datasets that contain artistic content like paintings, drawings or illustrations. Finally, using data attribution techniques, we analyze how both artistic and non-artistic datasets contribute to generating artistic-style images. Surprisingly, our findings suggest that high-quality artistic outputs can be achieved without prior exposure to artistic data, indicating that artistic style generation can occur in a controlled, opt-in manner using only a limited, carefully selected set of training examples.

Figures

Figures reproduced from arXiv: 2412.00176 by the authors.

Figure 1
Figure 1. (a) We introduce Blank Canvas Diffusion, a carefully curated text-to-image model trained only on photographs, serving as the [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Examples of images included and excluded from the [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Our Blank Canvas Diffusion model shows limited style [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (32 more)
Figure 4
Figure 4. Figure 4: Our model has no prior knowledge of paintings. It not [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: The generated image should match the style of a small exemplar dataset when prompted with a caption [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Comparison of Blank Canvas Diffusion art generation (top row) with Stable Diffusion 1.4 generated images (bottom row). [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: (Left) Results of the Perceptual User Study; Blank Can [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Comparion of our method and other image stylization baselines for the artist Van Gogh. All captions contain a suffix “in the style [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Data attribution experiments on stylized images reveals [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: Manual inspection tool [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: In a random sample of 10000 images (total: 71), we [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]
Figure 12
Figure 12. Figure 12: Histogram of the CLIP score of images with the word [PITH_FULL_IMAGE:figures/full_fig_p012_12.png]
Figure 13
Figure 13. Figure 13: Qualitative comparison of images generated with Blank [PITH_FULL_IMAGE:figures/full_fig_p013_13.png]
Figure 15
Figure 15. Figure 15: Through textual inversion using paintings by van Gogh, [PITH_FULL_IMAGE:figures/full_fig_p014_15.png]
Figure 16
Figure 16. Figure 16: Plug-and-Play editing on our model. We provide both [PITH_FULL_IMAGE:figures/full_fig_p014_16.png]
Figure 17
Figure 17. Figure 17: Dreambooth editing on our model. We send 7 barn [PITH_FULL_IMAGE:figures/full_fig_p015_17.png]
Figure 18
Figure 18. Figure 18: Art generation results using Art Adapter at different [PITH_FULL_IMAGE:figures/full_fig_p015_18.png]
Figure 20
Figure 20. Figure 20: Additional qualitative experiments showing diverse art generations and top five attributed images from both the Blank Canvas 17 [PITH_FULL_IMAGE:figures/full_fig_p017_20.png]
Figure 21
Figure 21. Figure 21: Generated artwork in the style of Alan Kenny (created and displayed with the artist’s permission) showcases the top-5 influential [PITH_FULL_IMAGE:figures/full_fig_p018_21.png]
Figure 22
Figure 22. Figure 22: Additional qualitative experiments of the art imitation of the interviewed artist Alan Kenny. [PITH_FULL_IMAGE:figures/full_fig_p018_22.png]
Figure 23
Figure 23. Figure 23: Additional qualitative experiments. Art Example Dataset Vincent van Gogh Image stylization Artistic Generation A group of young boys playing soccer on a city street at night. Reference Image Stylized Image Variations Text-conditioned Generations A black SUV driving do…
Figure 24
Figure 24. Figure 24: Additional qualitative experiments. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_24.png]
Figure 25
Figure 25. Figure 25: Additional qualitative experiments. Art Example Dataset Gustav Klimt Image stylization Artistic Generation A close-up view of a gold watch with a white face and a black background. Reference Image Stylized Image Variations Text-conditioned Generations A group of four …
Figure 26
Figure 26. Figure 26: Additional qualitative experiments. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_26.png]
Figure 27
Figure 27. Figure 27: Additional qualitative experiments. Art Example Dataset Henri Matisse Image stylization Artistic Generation A beautiful scene of two horses standing on a grassy hill. Reference Image Stylized Image Variations Text-conditioned Generations A group of people posing toget…
Figure 28
Figure 28. Figure 28: Additional qualitative experiments. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_28.png]
Figure 29
Figure 29. Figure 29: Additional qualitative experiments. Art Example Dataset Gerhard Richter Image stylization Artistic Generation A large hot air balloon floating in the sky above a city. Reference Image Stylized Image Variations Text-conditioned Generations A brown dog standing on a dir…
Figure 30
Figure 30. Figure 30: Additional qualitative experiments. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_30.png]
Figure 31
Figure 31. Figure 31: Additional qualitative experiments. Art Example Dataset Image stylization Artistic Generation Three scary looking characters, each with their own unique appearance. Reference Image Stylized Image Variations Text-conditioned Generations A tall building with a row of ba…
Figure 32
Figure 32. Figure 32: Additional qualitative experiments. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_32.png]
Figure 33
Figure 33. Figure 33: Additional qualitative experiments. Art Example Dataset Roy Lichtenstein Image stylization Artistic Generation A large, modern house with a swimming pool in the backyard. Reference Image Stylized Image Variations Text-conditioned Generations A church with a tall steep…
Figure 34
Figure 34. Figure 34: Additional qualitative experiments. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_34.png]
Figure 35
Figure 35. Figure 35: Additional qualitative experiments. Art Example Dataset M.C. Escher Image stylization Artistic Generation A beautiful garden with a large pond. Reference Image Stylized Image Variations Text-conditioned Generations A backyard with a wooden dining table surrounded by c…
Figure 36
Figure 36. Figure 36: Additional qualitative experiments. 25 [PITH_FULL_IMAGE:figures/full_fig_p025_36.png]
Figure 37
Figure 37. Figure 37: Additional qualitative experiments. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_37.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

65 extracted references · 34 canonical work pages

  1. [1]

    Andersen v

    Jane Andersen. Andersen v. stability ai: The landmark case unpacking the copyright risks of ai image generators. NYU Journal of Intellectual Property and Entertainment Law ,

    work page

  2. [2]

    In- structpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. In- structpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18392–18402, 2023. 6

    2023

  3. [3]

    Extracting training data from diffu- sion models

    Nicolas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagiel- ski, Vikash Sehwag, Florian Tramer, Borja Balle, Daphne Ip- polito, and Eric Wallace. Extracting training data from diffu- sion models. In 32nd USENIX Security Symposium (USENIX Security 23), pages 5253–5270, 2023. 2

    2023

  4. [4]

    Towards memorization-free diffusion models

    Chen Chen, Daochang Liu, and Chang Xu. Towards memorization-free diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8425–8434, 2024. 2

    2024

  5. [5]

    Pixart-α: Fast training of dif- fusion transformer for photorealistic text-to-image synthesis,

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-α: Fast training of dif- fusion transformer for photorealistic text-to-image synthesis,

  6. [6]

    Style injec- tion in diffusion: A training-free approach for adapting large- scale diffusion models for style transfer

    Jiwoo Chung, Sangeek Hyun, and Jae-Pil Heo. Style injec- tion in diffusion: A training-free approach for adapting large- scale diffusion models for style transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8795–8805, 2024. 16

    work page 2024

  7. [7]

    Style injec- tion in diffusion: A training-free approach for adapting large- scale diffusion models for style transfer

    Jiwoo Chung, Sangeek Hyun, and Jae-Pil Heo. Style injec- tion in diffusion: A training-free approach for adapting large- scale diffusion models for style transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8795–8805, 2024. 3

    2024

  8. [8]

    Stable diffusion v1.4, 2022

    CompVis. Stable diffusion v1.4, 2022. Accessed: 2025-03-

    work page 2022

Show all 65 references
  1. [9]

    Stable diffusion v1-4 model card, 2022

    CompVis. Stable diffusion v1-4 model card, 2022. 5

    2022

  2. [10]

    Diffusionshield: A wa- termark for copyright protection against generative diffusion models

    Yingqian Cui, Jie Ren, Han Xu, Pengfei He, Hui Liu, Lichao Sun, Yue Xing, and Jiliang Tang. Diffusionshield: A wa- termark for copyright protection against generative diffusion models. arXiv preprint arXiv:2306.04642, 2023. 2

    2023 arXiv

  3. [11]

    Multiresolution textual inversion

    Giannis Daras and Alexandros G Dimakis. Multiresolution textual inversion. arXiv preprint arXiv:2211.17115, 2022. 3

    2022 arXiv

  4. [12]

    Ambient diffu- sion: Learning clean distributions from corrupted data

    Giannis Daras, Kulin Shah, Yuval Dagan, Aravind Gol- lakota, Alex Dimakis, and Adam Klivans. Ambient diffu- sion: Learning clean distributions from corrupted data. Ad- vances in Neural Information Processing Systems , 36:288– 313, 2023. 2

    2023

  5. [13]

    Consistent diffusion meets tweedie: Training ex- act ambient diffusion models with noisy data

    Giannis Daras, Alexandros G Dimakis, and Constantinos Daskalakis. Consistent diffusion meets tweedie: Training ex- act ambient diffusion models with noisy data. arXiv preprint arXiv:2404.10177, 2024. 2

    2024 arXiv

  6. [14]

    Bert: Pre-training of deep bidirectional trans- formers for language understanding, 2019

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional trans- formers for language understanding, 2019. 4, 5

    2019

  7. [15]

    Image quilting for texture synthesis and transfer

    Alexei A Efros and William T Freeman. Image quilting for texture synthesis and transfer. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2, pages 571–576. 2023. 3

    2023

  8. [16]

    Art and the science of generative ai

    Ziv Epstein, Aaron Hertzmann, Investigators of Human Cre- ativity, Memo Akten, Hany Farid, Jessica Fjeld, Morgan R Frank, Matthew Groh, Laura Herman, Neil Leach, et al. Art and the science of generative ai. Science, 380(6650):1110– 1111, 2023. 1, 2

    2023

  9. [17]

    Bermano, Gal Chechik, and Daniel Cohen-Or

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H. Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image gen- eration using textual inversion, 2022. 3

    2022

  10. [18]

    Erasing concepts from diffusion models, 2023

    Rohit Gandikota, Joanna Materzynska, Jaden Fiotto- Kaufman, and David Bau. Erasing concepts from diffusion models, 2023. 1, 2

    2023

  11. [19]

    Unified concept editing in dif- fusion models

    Rohit Gandikota, Hadas Orgad, Yonatan Belinkov, Joanna Materzy´nska, and David Bau. Unified concept editing in dif- fusion models. arXiv preprint arXiv:2308.14761, 2023. 2

    2023 arXiv

  12. [20]

    Com- moncanvas: Open diffusion models trained on creative- commons images

    Aaron Gokaslan, A Feder Cooper, Jasmine Collins, Lan- dan Seguin, Austin Jacobson, Mihir Patel, Jonathan Fran- kle, Cory Stephenson, and V olodymyr Kuleshov. Com- moncanvas: Open diffusion models trained on creative- commons images. In Proceedings of the IEEE/CVF Con- ference ...

    2024

  13. [21]

    Getty images v

    Sophia Goosens. Getty images v. stability ai: The implica- tions for uk copyright law and licensing, 2024. Accessed: 2025-03-05. 2

    2024

  14. [22]

    This artist is dominating ai-generated art

    Melissa Heikkil ¨a. This artist is dominating ai-generated art. and he’s not happy about it.MIT Technology Review, 125(6): 9–10, 2022. 7

    2022

  15. [23]

    Selective amnesia: A continual learning approach to forgetting in deep generative models,

    Alvin Heng and Harold Soh. Selective amnesia: A continual learning approach to forgetting in deep generative models,

  16. [24]

    Style aligned image generation via shared atten- tion

    Amir Hertz, Andrey V oynov, Shlomi Fruchter, and Daniel Cohen-Or. Style aligned image generation via shared atten- tion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 4775–4785,

  17. [25]

    The choices hidden in photography

    Aaron Hertzmann. The choices hidden in photography. Jour- nal of Vision, 22(11):10–10, 2022. 8

    2022

  18. [26]

    Image analogies

    Aaron Hertzmann, Charles E Jacobs, Nuria Oliver, Brian Curless, and David H Salesin. Image analogies. In Sem- inal Graphics Papers: Pushing the Boundaries, Volume 2 , pages 557–570. 2023. 3

    2023

  19. [27]

    Classifier-free diffusion guidance, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance, 2022. 13 9

    2022

  20. [28]

    Denoising diffu- sion probabilistic models, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models, 2020. 1, 3

    2020

  21. [29]

    All but one: Surgical concept erasing with model preservation in text-to- image diffusion models

    Seunghoo Hong, Juhun Lee, and Simon S Woo. All but one: Surgical concept erasing with model preservation in text-to- image diffusion models. In Proceedings of the AAAI Con- ference on Artificial Intelligence, pages 21143–21151, 2024. 2

    2024

  22. [30]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021. 3, 5

    2021

  23. [31]

    Visual style prompting with swapping self- attention

    Jaeseok Jeong, Junho Kim, Yunjey Choi, Gayoung Lee, and Youngjung Uh. Visual style prompting with swapping self- attention. arXiv preprint arXiv:2402.12974, 2024. 3, 16

    2024 arXiv

  24. [32]

    Cpsample: Classifier protected sampling for guarding training data during diffusion

    Joshua Kazdan, Hao Sun, Jiaqi Han, Felix Petersen, and Stefano Ermon. Cpsample: Classifier protected sampling for guarding training data during diffusion. arXiv preprint arXiv:2409.07025, 2024. 2

    2024 arXiv

  25. [33]

    Diffu- sionclip: Text-guided diffusion models for robust image ma- nipulation

    Gwanghyun Kim, Taesung Kwon, and Jong Chul Ye. Diffu- sionclip: Text-guided diffusion models for robust image ma- nipulation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2426–2435,

  26. [34]

    Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick. Segment anything, 2023. 4

    2023

  27. [35]

    Ablating con- cepts in text-to-image diffusion models

    Nupur Kumari, Bingliang Zhang, Sheng-Yu Wang, Eli Shechtman, Richard Zhang, and Jun-Yan Zhu. Ablating con- cepts in text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 22691–22702, 2023. 2

    2023

  28. [36]

    Multi-concept customization of text-to-image diffusion, 2023

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi-concept customization of text-to-image diffusion, 2023. 14

    2023

  29. [37]

    Mace: Mass concept erasure in diffu- sion models

    Shilin Lu, Zilan Wang, Leyang Li, Yanzhu Liu, and Adams Wai-Kin Kong. Mace: Mass concept erasure in diffu- sion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6430– 6440, 2024. 2

    2024

  30. [38]

    Disguised copyright infringement of latent diffusion models

    Yiwei Lu, Matthew YR Yang, Zuoqiu Liu, Gautam Kamath, and Yaoliang Yu. Disguised copyright infringement of latent diffusion models. arXiv preprint arXiv:2404.06737, 2024. 2

    2024 arXiv

  31. [39]

    One-dimensional adapter to rule them all: Concepts diffusion models and erasing applications

    Mengyao Lyu, Yuhong Yang, Haiwen Hong, Hui Chen, Xuan Jin, Yuan He, Hui Xue, Jungong Han, and Guiguang Ding. One-dimensional adapter to rule them all: Concepts diffusion models and erasing applications. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern R...

    2024

  32. [40]

    A watermark-conditioned diffusion model for ip protection

    Rui Min, Sen Li, Hongyang Chen, and Minhao Cheng. A watermark-conditioned diffusion model for ip protection. arXiv preprint arXiv:2403.10893, 2024. 2

    2024 arXiv

  33. [41]

    Direct unlearning optimization for robust and safe text- to-image models

    Yong-Hyun Park, Sangdoo Yun, Jin-Hwa Kim, Junho Kim, Geonhui Jang, Yonghyun Jeong, Junghyo Jo, and Gayoung Lee. Direct unlearning optimization for robust and safe text- to-image models. arXiv preprint arXiv:2407.21035 , 2024. 2

    2024 arXiv

  34. [42]

    Styleclip: Text-driven manipulation of stylegan imagery

    Or Patashnik, Zongze Wu, Eli Shechtman, Daniel Cohen-Or, and Dani Lischinski. Styleclip: Text-driven manipulation of stylegan imagery. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 2085–2094,

  35. [43]

    Circumventing concept erasure meth- ods for text-to-image generative models

    Minh Pham, Kelly O Marshall, Niv Cohen, Govind Mittal, and Chinmay Hegde. Circumventing concept erasure meth- ods for text-to-image generative models. In The Twelfth In- ternational Conference on Learning Representations , 2023. 2, 14

    2023

  36. [44]

    Robust concept erasure using task vectors

    Minh Pham, Kelly O Marshall, Chinmay Hegde, and Niv Cohen. Robust concept erasure using task vectors. arXiv preprint arXiv:2404.03631, 2024. 2

    2024 arXiv

  37. [45]

    Dead- iff: An efficient stylization diffusion model with disentan- gled representations

    Tianhao Qi, Shancheng Fang, Yanze Wu, Hongtao Xie, Ji- awei Liu, Lang Chen, Qian He, and Yongdong Zhang. Dead- iff: An efficient stylization diffusion model with disentan- gled representations. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognit...

    2024

  38. [46]

    Learning transferable visual models from natural language supervision, 2021

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. 4

    2021

  39. [47]

    High-resolution image syn- thesis with latent diffusion models, 2022

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models, 2022. 3, 4, 5

    2022

  40. [48]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation, 2023

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation, 2023. 3, 14

    2023

  41. [49]

    Laion pop: 600,000 high-resolution images with detailed descriptions, 2023

    Christoph Schuhmann and Peter Bevan. Laion pop: 600,000 high-resolution images with detailed descriptions, 2023. 6

    2023

  42. [50]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in neural in- f...

    2022

  43. [51]

    This artist is dominating ai-generated art

    Tom Simonite. This artist is dominating ai-generated art. and he’s not happy about it. MIT Technology Review, 2022. 2

    2022

  44. [52]

    Styledrop: Text-to-image generation in any style

    Kihyuk Sohn, Nataniel Ruiz, Kimin Lee, Daniel Castro Chin, Irina Blok, Huiwen Chang, Jarred Barber, Lu Jiang, Glenn Entis, Yuanzhen Li, et al. Styledrop: Text-to-image generation in any style. arXiv preprint arXiv:2306.00983 ,

    arXiv

  45. [53]

    Diffusion art or digital forgery? investigating data replication in diffusion models

    Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Diffusion art or digital forgery? investigating data replication in diffusion models. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6048–6058, 2023. 1, 2

    2023

  46. [54]

    Measuring style similarity in diffusion models

    Gowthami Somepalli, Anubhav Gupta, Kamal Gupta, Shra- may Palta, Micah Goldblum, Jonas Geiping, Abhinav Shri- vastava, and Tom Goldstein. Measuring style similarity in diffusion models. arXiv preprint arXiv:2404.01292 , 2024. 1, 2, 6

    2024 arXiv

  47. [55]

    Spawning ai, 2023

    Spawning AI Team. Spawning ai, 2023. 2 10

    2023

  48. [56]

    Plug-and-play diffusion features for text-driven image-to-image translation

    Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text-driven image-to-image translation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1921–1930, 2023. 6, 14

    1921

  49. [57]

    Instantstyle: Free lunch towards style-preserving in text-to-image generation

    Haofan Wang, Matteo Spinelli, Qixun Wang, Xu Bai, Zekui Qin, and Anthony Chen. Instantstyle: Free lunch towards style-preserving in text-to-image generation. arXiv preprint arXiv:2404.02733, 2024. 3, 16

    2024 arXiv

  50. [58]

    Evaluating data attribution for text-to-image models

    Sheng-Yu Wang, Alexei A Efros, Jun-Yan Zhu, and Richard Zhang. Evaluating data attribution for text-to-image models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7192–7203, 2023. 7, 8

    2023

  51. [59]

    A computational approach to studying aesthetic judgments of ambiguous artworks

    Xi Wang, Zoya Bylinskii, Aaron Hertzmann, and Robert Pepperell. A computational approach to studying aesthetic judgments of ambiguous artworks. Psychology of Aesthet- ics, Creativity, and the Arts, 2023. 6

    2023

  52. [60]

    De- tecting, explaining, and mitigating memorization in diffusion models

    Yuxin Wen, Yuchen Liu, Chen Chen, and Lingjuan Lyu. De- tecting, explaining, and mitigating memorization in diffusion models. In The Twelfth International Conference on Learn- ing Representations, 2024. 2

    2024

  53. [61]

    Artists land a win in class action law- suit against a.i

    Richard Whiddington. Artists land a win in class action law- suit against a.i. companies. Artnet News, 2024. 1

    2024

  54. [62]

    Defensive unlearning with adversarial training for robust concept erasure in diffusion models

    Yimeng Zhang, Xin Chen, Jinghan Jia, Yihua Zhang, Chongyu Fan, Jiancheng Liu, Mingyi Hong, Ke Ding, and Sijia Liu. Defensive unlearning with adversarial training for robust concept erasure in diffusion models. arXiv preprint arXiv:2405.15234, 2024. 2

    2024 arXiv

  55. [63]

    A recipe for watermarking dif- fusion models

    Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Ngai- Man Cheung, and Min Lin. A recipe for watermarking dif- fusion models. arXiv preprint arXiv:2303.10137, 2023. 2

    2023 arXiv

  56. [64]

    painting

    Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle- consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision , pages 2223– 2232, 2017. 6 11 Supplementary Material A. Artwo...

    2017

  57. [2024]

    Accessed: 2025-03-05. 2

    2025

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.