Pith. sign in

REVIEW 4 major objections 5 minor 85 references

When and Where do Data Poisons Attack Textual Inversion?

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Data-poisoning attacks on textual inversion strike at low-noise timesteps and outside the concept region, and a three-part training recipe called Safe-Zone Training can neutralize six such poisons while matching clean-data generation…

desk verdict A solid empirical defense paper that earns a serious referee; the main caveats are untested mask robustness and overstated 'all poisons' wording. read the letter →

arxiv 2507.10578 v4 pith:YNNC45MV submitted 2025-07-11 cs.CR cs.AI

Jeremy Styborski , Mingzhi Lyu , Jiayou Lu , Nupur Kapur , Adams Kong This is my paper · ORCID
classification cs.CRcs.AI
keywords datapoisoningtextualinversiondiffusionmodelsadversarialexamplesSafe-ZoneTrainingsemanticsensitivitymapstimesteplearningbiaslossmasking
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Artists and copyright holders sometimes add imperceptible poison perturbations to images to stop diffusion models from learning them. This paper argues that such poisons attack textual inversion, a widely used personalization technique, at a specific time and place: the poison signal is learned mostly at low-to-middle noise timesteps, and it distracts the learned text embedding from the concept object to the whole image. Based on that analysis, the authors propose Safe-Zone Training, which compresses images with JPEG, samples timesteps only above a threshold, and masks the loss to the concept region. On two datasets and six poisons, the recipe restores generation quality to roughly the level of training on clean data, without needing an extra purification network or additional trainable parameters.

What carries the argument

The central diagnostic object is the semantic sensitivity map (SSM): it replaces one token embedding in the prompt with a randomly sampled embedding and maps the resulting change in the noise-prediction output at each spatial location, revealing which image region a token actually controls, something cross-attention maps cannot do cleanly because self-attention in the text encoder entangles token signals. The central defense object is the Safe-Zone Training objective of Eq. 7, which trains the text embedding on JPEG-compressed poisoned images with timesteps sampled only from $t \sim U(t_{\mathrm{th}}, T)$ and multiplies the noise-prediction error elementwise by a binary mask $M_z$ resized from the pixel-space concept mask. The argument is carried by a Gaussian analysis of the noising process: the conditional noise distribution $p(\epsilon \mid z_t, t)$ approaches $\mathcal{N}(0, I)$ at $t \to 0$ and $\mathcal{N}(z_T, 0)$ at $t \to T$, so the noise-prediction error has maximum variance at low timesteps and the informative, gradient-bearing interval is the lower-middle range that the poisons exploit.

What would settle it

Take the strongest loss-maximizing poison, ADM+, and optimize it with a timestep distribution restricted to $t > 600$ instead of the uniform distribution; then train textual inversion with SZT. If DINOv2 similarity on generated images stays near clean levels, the high-timestep restriction is not the load-bearing defense; if it drops toward the no-defense poisoned level, the paper's claim that poisons concentrate at low timesteps is confirmed.

Watch

Extended reading notes

Core claim

The central discovery is that the vulnerability of textual inversion to data poisoning has a precise temporal and spatial structure. Diffusion models concentrate learnable signal in lower-middle timesteps because at $t \to 0$ the optimal noise prediction is zero and carries no gradient, while at $t \to T$ the loss collapses to zero; poisons inherit this bias, so adversarial signals live mainly in $t \in [0, 600]$ on Stable Diffusion v1.5. Spatially, the $\ell_2$ objectives used by diffuser and encoder attacks, combined with the $\ell_\infty$ perturbation bound, encourage every pixel region to contribute, so poisoned training spreads the embedding's attribution across the whole image instead of the concept object. Using semantic sensitivity maps, the paper shows this distraction directly, and then shows that a defense built from JPEG compression, high-timestep sampling, and loss masking can cancel the effect and match clean-training quality across CustomConcept101 and NovelConcepts10 for ADM+, ADM-, SDS+, SDS-, EA, and DA.

Load-bearing premise

The load-bearing premise is that the concept mask covers exactly the novel object in every training image; if the mask misses part of the object or includes background, the masked loss will either discard concept information or let poison signals through, and the paper does not evaluate that failure mode.

Editorial extensions

If this is right

  • A single hyperparameter change, sampling timesteps from $t \ge 600$, already improves poisoned generation quality and costs nothing at inference time.
  • JPEG compression at quality 25 turns bimodal poison noise into Gaussian-like noise and pulls poisoned latents back toward clean power spectra, so it can be bolted onto other defenses as a preprocessing step.
  • Loss masking beats masking the input image or the latent, because it preserves background context in the forward pass while blocking poison gradients outside the concept region.
  • Combined as SZT, the three components lift DINOv2 similarity on poisoned data from about 0.21 to about 0.46, matching clean training on CustomConcept101 and effectively nullifying all six tested poisons.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the paper locates the poison signal at low timesteps, a natural next attack is a poison optimized with a timestep distribution biased toward high timesteps; if that evades the timestep component, the JPEG and masking components would have to carry the defense alone.
  • SZT's spatial component inherits the quality of the concept masks, so a useful stress test is to run SZT on images with off-center, occluded, or poorly segmented concepts and measure how generation quality degrades as mask accuracy falls.
  • The SSM attribution tool is not limited to poison analysis; it can be reused to debug which parts of an image any text token controls in editing and personalization pipelines.
  • Since the defense adds no trainable parameters and only changes preprocessing, timestep sampling, and the loss, the same recipe should transfer to other embedding-based personalization methods; the paper's additional LoRA and CustomDiffusion experiments already point in this direction, though the mechanism was analyzed for TI.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper analyzes when and where data-poisoning attacks affect Textual Inversion (TI) in latent diffusion models. It introduces Semantic Sensitivity Maps (SSMs) to visualize token-level loss attribution, reports that poison signals are concentrated at lower-middle timesteps and spatially spread outside the target concept region, and proposes Safe-Zone Training (SZT), which combines JPEG compression, restriction of TI training to high timesteps, and loss masking on the concept region. The empirical evaluation covers six poisons (ADM+, ADM-, SDS+, SDS-, EA, DA), two datasets, multiple Stable Diffusion backbones, and several personalization methods, with SZT achieving DINOv2 similarity close to clean-data TI. The paper also releases code and data.

Significance. If the result holds, it is a practically significant contribution: SZT is a simple, lightweight defense that avoids regeneration networks and extra trainable parameters, yet it outperforms established defenses across the evaluated poison suite. The paper's strengths include unusually thorough ablations, cross-model transfer experiments, and public code/data release, which make the empirical claims reproducible in principle. The timestep-bias analysis and the SSM visualization are useful conceptual contributions regardless of the defense outcome. However, the evidence as presented leaves two load-bearing gaps: the loss-masking component relies on externally supplied masks whose error sensitivity is not tested, and no variance or statistical significance is reported for any headline number, so the small gaps between SZT and its ablations may not be reliable. The abstract's claim that SZT defends against 'all poisoning attacks' also overstates the six-attack, non-adaptive evaluation. These issues are fixable within the scope of the manuscript, but they need explicit experimental or textual attention before publication.

major comments (4)
  1. [Appendix B, Section 4.2, Eqs. (8)-(11)] The closed-form derivation of the noise-prediction error assumes p(z0) is Gaussian, and the authors explicitly concede in the footnote that p(z0) is 'decidedly not Gaussian' in the multi-image case. As written, Eqs. (8)-(11) therefore do not prove the timestep bias for TI training, which is a central mechanistic claim. The empirical loss and gradient measurements in Figs. 4, 5, 10, and 11 are consistent with the intended conclusion, so I do not view this as a rejection issue, but the derivation should be reframed as a heuristic or as an exact single-image analysis, with the multi-image claim presented as empirically supported rather than derived.
  2. [Section 5.1.1, Eq. (7)] SZT's loss-masking component depends on binary masks from Segment Anything, but the manuscript never evaluates what happens when masks are imperfect. The marginal benefit of masking over JPEG+T600 is small in Table 1 (SZT 0.46 vs. JPEG+LM 0.45 vs. JPEG+T600 0.43 on the CustomConcept101 Psn Avg row), so a mask that excludes part of the concept or includes background could plausibly erase SZT's advantage. Please report how masks are generated (on clean, poisoned, or JPEG-compressed images), include mask-quality ablations such as random erasure, dilation/erosion, or misalignment, and state whether any manual correction was applied.
  3. [Tables 1, 17, and related ablations] No variance or number of seeds is reported for any headline metric. All values are averages across 10 concepts (NovelConcepts10) or 101 concepts (CustomConcept101), and the differences between SZT and JPEG+LM or JPEG+T600 are 0.01-0.03 in DINOv2 similarity. Given the small concept counts and likely seed-to-seed variation, these gaps may not be statistically reliable. Please report per-concept standard deviations or confidence intervals, and preferably rerun the main comparison with multiple seeds, so that the claim that SZT 'matches clean performance' is quantitative.
  4. [Abstract and Section 5.2.4] The claim that SZT is robust against 'all poisoning attacks' overreaches the evidence. The evaluation covers six fixed attacks, all optimized against SD1.5, with no adaptive attacks designed to circumvent JPEG compression, high-timestep sampling, or loss masking, and no attacks targeting the mask component specifically. Since adversarial defenses are typically assessed against an adaptive adversary, please either temper the wording (e.g., 'all six evaluated attacks') or add an adaptive-attack evaluation. This is not a circularity objection; the six-attack results are credible as far as they go.
minor comments (5)
  1. [Section 4.1, Eq. (6)] The definition of SSM is underspecified: the expectation is over beΔn, but the distribution from which the replacement token is sampled is not defined, and the channel-wise reduction is described only in prose. Please define the sampling distribution and the exact reduction.
  2. [Appendix B, Eqs. (10)-(11)] The notation is inconsistent: Eq. (10) uses µx while the derivation is in terms of the latent distribution with mean µz, and Eq. (11) mixes the Gaussian notation with the expression p(ϵ|zt,t) - ϵ. Please fix these notational issues.
  3. [Section 4.4, Fig. 7] The claim that JPEG converts 'bimodal poison noise into unimodal Gaussian-like noise' is supported only by visual histograms. A quantitative distributional comparison, or at least fitted distribution parameters, would make the claim more precise.
  4. [Section 5.2.4 and Appendix J] The final SZT configuration (JPEG quality 25, t≥500, mask dilation 16 pixels) appears to have been selected on NovelConcepts10 and then evaluated on CustomConcept101. This is not circular, but it should be stated explicitly so readers can assess the potential for hyperparameter overfitting.
  5. [Section 4.3] The spatial-distraction mechanism is argued from the convexity of l2 and the l∞ constraint, but the text should mark this as a qualitative argument rather than a proof; Fig. 6 provides empirical support and is the stronger evidence.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the timestep and spatial analyses are derived from the noise-prediction objective, and SZT is ablation-tuned and evaluated out-of-sample.

full rationale

The paper's derivation chain is self-contained and does not reduce to its inputs. The timestep bias is derived from the noising process and the noise-prediction objective (Eqs. 1-4 and Appendix B), then verified on independently collected LAION loss measurements and NovelConcepts10 gradient measurements; no model is fitted to the target SZT results. The spatial distraction claim is supported by loss-in/out measurements and SSM visualizations rather than by assuming the conclusion. SZT (Eq. 7) combines three mechanism-grounded components: JPEG compression, high-timestep sampling, and loss masking. Its hyperparameters (JPEG quality 25, t>=500, 16-pixel dilation) are selected through ablations on NovelConcepts10 and then evaluated on the held-out CustomConcept101 dataset, so the main performance claim is a genuine out-of-sample test. The self-citations present in the paper are background or motivational and do not carry the load-bearing argument. The Rn,Mz ratio in Eq. 16 is a consistency check that loss masking concentrates attribution inside the mask; it is not used to claim SZT's downstream generative-quality gains, which are measured by DINOv2, FID, and CLIP on generated images. No uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in via citation. The paper honestly notes limitations, including the non-Gaussianity of p(z0) in Appendix B, and the remaining concerns are about robustness of the mask component and evaluation breadth, which are correctness risks rather than circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The defense depends on a few hand-chosen hyperparameters (JPEG quality, timestep threshold, mask dilation), three domain assumptions about the noise-prediction model and the attack constraint, and an accurate-segmentation premise for the masking component. No new physical entities are introduced; SSM is a visualization method rather than a postulated entity.

free parameters (3)
  • JPEG quality = 25
    Hand-chosen compression level for SZT; selected via ablation on NovelConcepts10, not derived from an analysis of poison frequency spectra.
  • timestep threshold t_th = 500 (SZT final; T600 in ablations)
    Chosen empirically as the lower bound for uniform timestep sampling in SZT; performance drops if set too high (t>=900) because clean concept signal also decays.
  • mask dilation = 16 pixels
    Selected by ablation over 0, 8, 16, 24 pixels; balances keeping background context vs. focusing on the concept.
assumptions (4)
  • domain assumption The distribution of latent images p(z0) is approximately Gaussian (Appendix B).
    Used to derive the closed-form conditional distribution of noise given a noisy latent, and the claim that noise-prediction variance peaks at t=0; authors admit p(z0) is 'decidedly not Gaussian' but argue empirical loss curves conform.
  • domain assumption At t=0, the trained noise-prediction model outputs near-zero noise independent of z_t and text embedding, so gradients vanish; at t=T it outputs its input, so loss and gradients vanish.
    Underlies the timestep learning-bias analysis (Sec. 4.2); is an empirical property of the fitted model rather than a proven mathematical fact.
  • domain assumption Adversarial perturbations are constrained by a small l-infinity bound (kappa=16/256), so JPEG compression can remove most poison energy.
    Assumed throughout Sec. 4.4 and used by SZT's first component; not valid for unrestricted or low-frequency poisons.
  • domain assumption Segment Anything masks isolate novel-concept regions in all training images (Sec. 5.1.1).
    Loss masking and SZT require accurate concept masks; mask errors are not analyzed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of When and Where do Data Poisons Attack Textual Inversion?." pith.science (2026). https://pith.science/paper/YNNC45MV

@misc{pith2026250710578,
  author       = {Pith},
  title        = {Pith review of: When and Where do Data Poisons Attack Textual Inversion?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YNNC45MV}},
  note         = {Machine review of arXiv:2507.10578}
}
read the original abstract

Poisoning attacks pose significant challenges to the robustness of diffusion models (DMs). In this paper, we systematically analyze when and where poisoning attacks textual inversion (TI), a widely used personalization technique for DMs. We first introduce Semantic Sensitivity Maps, a novel method for visualizing the influence of poisoning on text embeddings. Second, we identify and experimentally verify that DMs exhibit non-uniform learning behavior across timesteps, focusing on lower-noise samples. Poisoning attacks inherit this bias and inject adversarial signals predominantly at lower timesteps. Lastly, we observe that adversarial signals distract learning away from relevant concept regions within training data, corrupting the TI process. Based on these insights, we propose Safe-Zone Training (SZT), a novel defense mechanism comprised of 3 key components: (1) JPEG compression to weaken high-frequency poison signals, (2) restriction to high timesteps during TI training to avoid adversarial signals at lower timesteps, and (3) loss masking to constrain learning to relevant regions. Extensive experiments across multiple poisoning methods demonstrate that SZT greatly enhances the robustness of TI against all poisoning attacks, improving generative quality beyond prior published defenses. Code: www.github.com/JStyborski/Diff_Lab Data: www.github.com/JStyborski/NC10

Figures

Figures reproduced from arXiv: 2507.10578 by the authors.

Figure 1
Figure 1. Applying SZT to TI mitigates poison signals and re [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Cross attention maps (a) and SSMs (b) for tokens “ [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Loss distributions for Stable Diffusion 1.5 on [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figures from the paper (13 more)
Figure 7
Figure 7. Figure 7: Histograms of pixel-space perturbations (relative to [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: Generated images after TI on NovelConcepts10 BlueElephant poisoned by ADM+. Generation prompt: “a [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Comparison between SSMs of clean and AE samples at [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 11
Figure 11. Figure 11: Plots of text embedding loss gradient magnitude as a function of timestep measured throughout training. Values at each timestep [PITH_FULL_IMAGE:figures/full_fig_p013_11.png]
Figure 12
Figure 12. Figure 12: Histograms of pixel-space perturbations (relative to clean images) for poisoned NovelConcept10 images without (red) and with [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]
Figure 14
Figure 14. Figure 14: One example image for each of the 10 concepts con [PITH_FULL_IMAGE:figures/full_fig_p014_14.png]
Figure 15
Figure 15. Figure 15: Probability curves for timestep sampling, displaying [PITH_FULL_IMAGE:figures/full_fig_p017_15.png]
Figure 16
Figure 16. Figure 16: Average Rn,Mz for clean and poisoned versions of NovelConcepts10 throughout TI training. “wLM” denotes TI with loss masking [PITH_FULL_IMAGE:figures/full_fig_p019_16.png]
Figure 17
Figure 17. Figure 17: DINOv2 Similarity versus CLIP Score for all defenses [PITH_FULL_IMAGE:figures/full_fig_p020_17.png]
Figure 18
Figure 18. Figure 18: Images generated after TI on various concepts from CustomConcept101 poisoned by ADM+. Prompt: “a [PITH_FULL_IMAGE:figures/full_fig_p024_18.png]
Figure 19
Figure 19. Figure 19: Images generated after TI on things-bottle1 (CustomConcept101) for various poisons. Prompt: “a [PITH_FULL_IMAGE:figures/full_fig_p024_19.png]
Figure 20
Figure 20. Figure 20: Images generated after TI on various concepts from NovelConcepts10 poisoned by ADM+. Prompt: “a [PITH_FULL_IMAGE:figures/full_fig_p025_20.png]
Figure 21
Figure 21. Figure 21: Images generated after TI on FishDoll (NovelConcepts10) for various poisons. Prompt: “a [PITH_FULL_IMAGE:figures/full_fig_p025_21.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

85 extracted references · 74 canonical work pages

  1. [1]

    Adversarial example detection for dnn models: a review and experimental comparison

    Ahmed Aldahdooh, Wassim Hamidouche, Sid Ahmed Fezza, and Olivier D´eforges. Adversarial example detection for dnn models: a review and experimental comparison. Ar- tificial Intelligence Review, 55(6):4403–4462, 2022. 3

    2022

  2. [2]

    Stable Diffusion Web UI, 2022

    AUTOMATIC1111. Stable Diffusion Web UI, 2022. 1

    2022

  3. [3]

    Tim Brooks, Aleksander Holynski, and Alexei A. Efros. In- structpix2pix: Learning to follow image editing instructions,

  4. [4]

    Poisoning and back- dooring contrastive learning

    Nicholas Carlini and Andreas Terzis. Poisoning and back- dooring contrastive learning. In International Conference on Learning Representations, 2022. 2

    2022

  5. [5]

    Towards evaluating the robustness of neural networks, 2017

    Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks, 2017. 2

    2017

  6. [6]

    Diffusionguard: A robust defense against malicious diffusion-based image editing

    June Suk Choi, Kyungmin Lee, Jongheon Jeong, Saining Xie, Jinwoo Shin, and Kimin Lee. Diffusionguard: A robust defense against malicious diffusion-based image editing. In The Thirteenth International Conference on Learning Rep- resentations, 2025. 1, 2, 3

    work page 2025

  7. [8]

    Zico Kolter

    Jeremy M Cohen, Elan Rosenfeld, and J. Zico Kolter. Certi- fied adversarial robustness via randomized smoothing, 2019. 3

    work page 2019

  8. [9]

    Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V

    Ekin D. Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V . Le. Randaugment: Practical automated data augmentation with a reduced search space, 2019. 3

    work page 2019

Show all 85 references
  1. [10]

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR09, 2009. 14

    2009

  2. [11]

    Terrance DeVries and Graham W. Taylor. Improved regular- ization of convolutional neural networks with cutout, 2017. 3

    2017

  3. [12]

    Diffusion models beat gans on image synthesis, 2021

    Prafulla Dhariwal and Alex Nichol. Diffusion models beat gans on image synthesis, 2021. 2

    2021

  4. [13]

    Dolatabadi, Sarah Erfani, and Christopher Leckie

    Hadi M. Dolatabadi, Sarah Erfani, and Christopher Leckie. The devil’s advocate: Shattering the illusion of unexploitable data using diffusion models, 2024. 3

    2024

  5. [14]

    Adversarial examples make strong poisons, 2021

    Liam Fowl, Micah Goldblum, Ping yeh Chiang, Jonas Geip- ing, Wojtek Czaja, and Tom Goldstein. Adversarial examples make strong poisons, 2021. 3

    2021

  6. [15]

    Bermano, Gal Chechik, and Daniel Cohen-Or

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H. Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image gen- eration using textual inversion, 2022. 1, 2, 3, 7

    2022

  7. [16]

    Wichmann, and Wieland Brendel

    Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Imagenet-trained CNNs are biased towards texture; increas- ing shape bias improves accuracy and robustness. In Inter- national Conference on Learning Representations, 2019. 3

    2019

  8. [17]

    Goodfellow, Jonathon Shlens, and Christian Szegedy

    Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples, 2015. 1, 2

    2015

  9. [18]

    Mix-of-show: Decentralized low-rank adaptation for multi-concept customization of diffusion models, 2023

    Yuchao Gu, Xintao Wang, Jay Zhangjie Wu, Yujun Shi, Yun- peng Chen, Zihan Fan, Wuyou Xiao, Rui Zhao, Shuning Chang, Weijia Wu, Yixiao Ge, Ying Shan, and Mike Zheng Shou. Mix-of-show: Decentralized low-rank adaptation for multi-concept customization of diffusion models, 2023. 2

    2023

  10. [19]

    Svdiff: Compact param- eter space for diffusion fine-tuning, 2023

    Ligong Han, Yinxiao Li, Han Zhang, Peyman Milanfar, Dimitris Metaxas, and Feng Yang. Svdiff: Compact param- eter space for diffusion fine-tuning, 2023. 2

    2023

  11. [20]

    Prompt-to-prompt image editing with cross attention control, 2022

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt image editing with cross attention control, 2022. 1, 2

    2022

  12. [21]

    Clipscore: A reference-free evaluation met- ric for image captioning, 2022

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation met- ric for image captioning, 2022. 7

    2022

  13. [22]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium, 2018

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium, 2018. 7

    2018

  14. [23]

    Classifier-free diffusion guidance, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance, 2022. 2

    2022

  15. [24]

    Denoising diffu- sion probabilistic models, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models, 2020. 2, 3

    2020

  16. [25]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021. 1, 2, 26

    2021

  17. [26]

    Unlearnable examples: Making personal data unexploitable, 2021

    Hanxun Huang, Xingjun Ma, Sarah Monazam Erfani, James Bailey, and Yisen Wang. Unlearnable examples: Making personal data unexploitable, 2021. 3

    2021

  18. [27]

    Adversarial attacks on neural network policies, 2017

    Sandy Huang, Nicolas Papernot, Ian Goodfellow, Yan Duan, and Pieter Abbeel. Adversarial attacks on neural network policies, 2017. 2

    2017

  19. [28]

    Robust pre-training by adversarial contrastive learn- ing, 2020

    Ziyu Jiang, Tianlong Chen, Ting Chen, and Zhangyang Wang. Robust pre-training by adversarial contrastive learn- ing, 2020. 2

    2020

  20. [29]

    Pnp inversion: Boosting diffusion-based editing with 3 lines of code

    Xuan Ju, Ailing Zeng, Yuxuan Bian, Shaoteng Liu, and Qiang Xu. Pnp inversion: Boosting diffusion-based editing with 3 lines of code. InThe Twelfth International Conference on Learning Representations, 2024. 2

    2024

  21. [30]

    Adversarial self-supervised contrastive learning, 2020

    Minseon Kim, Jihoon Tack, and Sung Ju Hwang. Adversarial self-supervised contrastive learning, 2020. 2

    2020

  22. [31]

    Auto-encoding varia- tional bayes, 2022

    Diederik P Kingma and Max Welling. Auto-encoding varia- tional bayes, 2022. 12

    2022

  23. [32]

    Kingma, Tim Salimans, Ben Poole, and Jonathan Ho

    Diederik P. Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models, 2023. 2

    2023

  24. [33]

    Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick. Segment anything. arXiv:2304.02643, 2023. 7

    2023 arXiv

  25. [34]

    Adversarial exam- ples for generative models, 2017

    Jernej Kos, Ian Fischer, and Dawn Song. Adversarial exam- ples for generative models, 2017. 2

    2017

  26. [35]

    Multi-concept customization of text-to-image diffusion, 2023

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi-concept customization of text-to-image diffusion, 2023. 1, 2, 7, 26

    2023

  27. [36]

    Adver- sarial examples in the physical world, 2017

    Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adver- sarial examples in the physical world, 2017. 1, 2

    2017

  28. [37]

    Anti-dreambooth: Protect- ing users from personalized text-to-image synthesis, 2023

    Thanh Van Le, Hao Phung, Thuan Hoang Nguyen, Quan Dao, Ngoc Tran, and Anh Tran. Anti-dreambooth: Protect- ing users from personalized text-to-image synthesis, 2023. 2, 3, 26

    2023

  29. [38]

    Mist: Towards improved adversarial examples for diffusion models, 2023

    Chumeng Liang and Xiaoyu Wu. Mist: Towards improved adversarial examples for diffusion models, 2023. 3, 14, 15

    2023

  30. [39]

    Adversarial example does good: Preventing paint- ing imitation from diffusion models via adversarial exam- ples, 2023

    Chumeng Liang, Xiaoyu Wu, Yang Hua, Jiaru Zhang, Yim- ing Xue, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan. Adversarial example does good: Preventing paint- ing imitation from diffusion models via adversarial exam- ples, 2023. 1, 2, 3, 5, 7, 8, 14, 15, 21, 22, 23, 27, 28...

    2023

  31. [40]

    Common diffusion noise schedules and sample steps are flawed, 2024

    Shanchuan Lin, Bingchen Liu, Jiashi Li, and Xiao Yang. Common diffusion noise schedules and sample steps are flawed, 2024. 26

    2024

  32. [41]

    Towards understanding cross and self-attention in stable diffusion for text-guided image editing

    Bingyan Liu, Chengyu Wang, Tingfeng Cao, Kui Jia, and Jun Huang. Towards understanding cross and self-attention in stable diffusion for text-guided image editing. InProceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7817–7826, 2024. 4

    2024

  33. [42]

    Flow straight and fast: Learning to generate and transfer data with rectified flow, 2022

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow, 2022. 26

    2022

  34. [43]

    Instaflow: One step is enough for high-quality diffusion-based text-to-image generation, 2024

    Xingchao Liu, Xiwen Zhang, Jianzhu Ma, Jian Peng, and Qiang Liu. Instaflow: One step is enough for high-quality diffusion-based text-to-image generation, 2024. 26

    2024

  35. [44]

    Image watermarks are removable using controllable regeneration from clean noise,

    Yepeng Liu, Yiren Song, Hai Ci, Yu Zhang, Haofan Wang, Mike Zheng Shou, and Yuheng Bu. Image watermarks are removable using controllable regeneration from clean noise,

  36. [45]

    Image shortcut squeezing: Countering perturbative availability poi- sons with compression, 2023

    Zhuoran Liu, Zhengyu Zhao, and Martha Larson. Image shortcut squeezing: Countering perturbative availability poi- sons with compression, 2023. 3, 6

    2023

  37. [46]

    Tf-icon: Diffusion-based training-free cross-domain image composi- tion

    Shilin Lu, Yanzhu Liu, and Adams Wai-Kin Kong. Tf-icon: Diffusion-based training-free cross-domain image composi- tion. In Proceedings of the IEEE/CVF International Confer- ence on Computer Vision, pages 2294–2305, 2023. 4

    2023

  38. [47]

    Gradient descent on two-layer nets: Margin maximization and simplicity bias, 2021

    Kaifeng Lyu, Zhiyuan Li, Runzhe Wang, and Sanjeev Arora. Gradient descent on two-layer nets: Margin maximization and simplicity bias, 2021. 3

    2021

  39. [48]

    Ad- versarial attack for robust watermark protection against inpainting-based and blind watermark removers

    Mingzhi Lyu, Yi Huang, and Adams Wai-Kin Kong. Ad- versarial attack for robust watermark protection against inpainting-based and blind watermark removers. InProceed- ings of the 31st ACM International Conference on Multime- dia, page 8396–8405, New York, NY , USA, 2023. Associ...

    2023

  40. [49]

    Towards deep learning models resistant to adversarial attacks, 2019

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks, 2019. 1, 2

    2019

  41. [50]

    Sdedit: Guided image synthesis and editing with stochastic differential equa- tions, 2022

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jia- jun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equa- tions, 2022. 2

    2022

  42. [51]

    Negative-prompt inversion: Fast image inversion for editing with text-guided diffusion models, 2024

    Daiki Miyake, Akihiro Iohara, Yu Saito, and Toshiyuki Tanaka. Negative-prompt inversion: Fast image inversion for editing with text-guided diffusion models, 2024

    2024

  43. [52]

    Null-text inversion for editing real images using guided diffusion models, 2022

    Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models, 2022. 2

    2022

  44. [53]

    Lego: Learning to disentangle and invert personalized con- cepts beyond object appearance in text-to-image diffusion models, 2024

    Saman Motamed, Danda Pani Paudel, and Luc Van Gool. Lego: Learning to disentangle and invert personalized con- cepts beyond object appearance in text-to-image diffusion models, 2024. 2

    2024

  45. [54]

    Dinov2: Learning robust visual features with- out supervision, 2024

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mah- moud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michae...

    2024

  46. [55]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis, 2023. 26

    2023

  47. [56]

    Learning transferable visual models from natural language supervision, 2021

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. 7

    2021

  48. [57]

    Hierarchical text-conditional image gener- ation with clip latents, 2022

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents, 2022. 1

    2022

  49. [58]

    High-resolution image syn- thesis with latent diffusion models, 2022

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models, 2022. 1, 2, 3, 7, 12

    2022

  50. [59]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation, 2023

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation, 2023. 1, 2, 7, 26

    2023

  51. [60]

    Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion mod...

    2022

  52. [61]

    Raising the cost of malicious ai-powered image editing, 2023

    Hadi Salman, Alaa Khaddaj, Guillaume Leclerc, Andrew Ilyas, and Aleksander Madry. Raising the cost of malicious ai-powered image editing, 2023. 1, 2, 3, 7, 8, 14, 15, 21, 22, 23, 27, 28, 29, 30, 31, 32, 33

    2023

  53. [62]

    Laion-5b: An open large-scale dataset for training next generation image-text models, 2022

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. Lai...

    2022

  54. [63]

    Shawn Shan, Jenna Cryan, Emily Wenger, Haitao Zheng, Rana Hanocka, and Ben Y . Zhao. Glaze: Protecting artists from style mimicry by text-to-image models, 2023. 2, 3

    2023

  55. [64]

    Shawn Shan, Wenxin Ding, Josephine Passananti, Stanley Wu, Haitao Zheng, and Ben Y . Zhao. Nightshade: Prompt- specific poisoning attacks on text-to-image generative mod- els, 2024. 2, 3

    2024

  56. [65]

    Shidoto/adversecleaner: Remove adversarial noise from images, 2025

    Shidoto. Shidoto/adversecleaner: Remove adversarial noise from images, 2025. 3, 7, 8, 22, 23, 27, 28, 29, 30, 31, 32, 33

    2025

  57. [66]

    Denois- ing diffusion implicit models, 2022

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models, 2022. 2

    2022

  58. [67]

    Generative modeling by es- timating gradients of the data distribution, 2020

    Yang Song and Stefano Ermon. Generative modeling by es- timating gradients of the data distribution, 2020. 2

    2020

  59. [68]

    Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole

    Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions, 2021. 2, 3

    2021

  60. [69]

    Exploiting supervised poison vulnerability to strengthen self-supervised defense, 2024

    Jeremy Styborski, Mingzhi Lyu, Yi Huang, and Adams Kong. Exploiting supervised poison vulnerability to strengthen self-supervised defense, 2024. 3

    2024

  61. [70]

    In- triguing properties of neural networks, 2014

    Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. In- triguing properties of neural networks, 2014. 1, 2

    2014

  62. [71]

    Adversar- ial images for variational autoencoders, 2016

    Pedro Tabacof, Julia Tavares, and Eduardo Valle. Adversar- ial images for variational autoencoders, 2016. 2

    2016

  63. [72]

    Diffusers: State-of-the-art diffu- sion models

    Patrick von Platen, Suraj Patil, Anton Lozhkov, Pedro Cuenca, Nathan Lambert, Kashif Rasul, Mishig Davaadorj, Dhruv Nair, Sayak Paul, William Berman, Yiyi Xu, Steven Liu, and Thomas Wolf. Diffusers: State-of-the-art diffu- sion models. https://github.com/huggingface/ diffusers...

    2022

  64. [73]

    P+: Extended textual conditioning in text-to- image generation, 2023

    Andrey V oynov, Qinghao Chu, Daniel Cohen-Or, and Kfir Aberman. P+: Extended textual conditioning in text-to- image generation, 2023. 2

    2023

  65. [74]

    Elite: Encoding visual con- cepts into textual embeddings for customized text-to-image generation, 2023

    Yuxiang Wei, Yabo Zhang, Zhilong Ji, Jinfeng Bai, Lei Zhang, and Wangmeng Zuo. Elite: Encoding visual con- cepts into textual embeddings for customized text-to-image generation, 2023. 2

    2023

  66. [75]

    Multivariate normal distribution, 2025

    Wikipedia. Multivariate normal distribution, 2025. 12

    2025

  67. [76]

    Adversarial examples for se- mantic segmentation and object detection, 2017

    Cihang Xie, Jianyu Wang, Zhishuai Zhang, Yuyin Zhou, Lingxi Xie, and Alan Yuille. Adversarial examples for se- mantic segmentation and object detection, 2017. 2

    2017

  68. [77]

    Pixel is a barrier: Diffusion models are more adversarially robust than we think, 2024

    Haotian Xue and Yongxin Chen. Pixel is a barrier: Diffusion models are more adversarially robust than we think, 2024. 3, 7, 8, 22, 23, 27, 28, 29, 30, 31, 32, 33

    2024

  69. [78]

    Toward effective protection against diffusion-based mimicry through score distillation

    Haotian Xue, Chumeng Liang, Xiaoyu Wu, and Yongxin Chen. Toward effective protection against diffusion-based mimicry through score distillation. In The Twelfth Interna- tional Conference on Learning Representations, 2024. 1, 2, 3, 7, 8, 14, 15, 21, 22, 23, 26, 27, 28, 29, 30, ...

    2024

  70. [79]

    Which features are learnt by con- trastive learning? on the role of simplicity bias in class col- lapse and feature suppression, 2023

    Yihao Xue, Siddharth Joshi, Eric Gan, Pin-Yu Chen, and Baharan Mirzasoleiman. Which features are learnt by con- trastive learning? on the role of simplicity bias in class col- lapse and feature suppression, 2023. 3

    2023

  71. [80]

    Cutmix: Regu- larization strategy to train strong classifiers with localizable features, 2019

    Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regu- larization strategy to train strong classifiers with localizable features, 2019. 3

    2019

  72. [81]

    Dauphin, and David Lopez-Paz

    Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimiza- tion, 2018. 3

    2018

  73. [82]

    Adding conditional control to text-to-image diffusion models, 2023

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models, 2023. 1, 2

    2023

  74. [83]

    Compositional inversion for stable diffusion models, 2024

    Xulu Zhang, Xiao-Yong Wei, Jinlin Wu, Tianyi Zhang, Zhaoxiang Zhang, Zhen Lei, and Qing Li. Compositional inversion for stable diffusion models, 2024. 2

    2024

  75. [84]

    standardize

    Xuandong Zhao, Kexun Zhang, Zihao Su, Saastha Vasan, Ilya Grishchenko, Christopher Kruegel, Giovanni Vigna, Yu- Xiang Wang, and Lei Li. Invisible image watermarks are provably removable using generative ai, 2024. 3, 7, 8, 22, 23, 27, 28, 29, 30, 31, 32, 33 Figure 9. Comparison...

    2024

  76. [85]

    (14) ZM applies masking to the latent vector zt and is given by LZM (x, t, c, Mx) = ||ϵθ(zt ⊙ Mz, t, c) − ϵ||2

  77. [86]

    LM-D08”, “LM-D16

    (15) We evaluate masking types on NovelConcepts10 across all poisons. Tables 9, 10, and 11 demonstrate that LM out- performs all other forms of masking for poison defense, as it is the only method that fully preserves background informa- tion in the forward process. IM perform...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.