Pith. sign in

REVIEW 4 major objections 6 minor 68 references

Not All Parameters Matter: Masking Diffusion Models for Enhancing Generation Ability

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Zeroing selected U-Net weights at each denoising timestep improves image quality without updating the model.

desk verdict MaskUNet is a plausible parameter-efficient trick for frozen diffusion U-Nets, but the paper underspecifies how masking applies to convolutional layers and the reported gains are modest and thinly evidenced. read the letter →

arxiv 2505.03097 v1 pith:HLHRRBGM submitted 2025-05-06 cs.CV

Lei Wang , Senmao Li , Fei Yang , Jianye Wang , Ziheng Zhang , Yuhan Liu , Yaxing Wang , Jian Yang This is my paper · ORCID
classification cs.CV
keywords diffusionmodelstext-to-imagegenerationU-Netbinarymaskparameter-efficientfine-tuningzero-shotsemanticbindingreward-modeloptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MaskUNet claims that a frozen text-to-image diffusion U-Net can generate substantially better images if, at each denoising step and for each prompt, a learned binary mask zeros out some of its weights. The paper reports that this masking lowers FID, a standard image-quality metric, from 12.85 to 11.72 on COCO 2014 and from 23.39 to 21.88 on COCO 2017, outperforming LoRA while updating no U-Net parameters. The authors support this with the observation that random weight-masking already helps denoising, and that the beneficial mask varies by timestep and by sample. If true, the work shows a parameter-efficient route to improving pretrained diffusion models at inference, and a new axis for adapting them: weight selection rather than weight update.

What carries the argument

The load-bearing object is the timestep- and sample-dependent binary mask, produced by a lightweight generator: $z' = FC(t_{\text{emb}}) + \text{GAP}(z)$, then $\hat{z} = \text{MLP}(z')$, then $m = \sigma(\hat{z};\tau,\delta)$ where $\sigma$ is Gumbel-Sigmoid with temperature $\tau$ and threshold $\delta$. The mask is reshaped to $m' \in \mathbb{R}^{B \times C_{\text{out}} \times C_{\text{in}}}$ and applied to the frozen linear-layer weight $w$ by $\hat{w} = m' \odot w$, with outputs computed via batch matrix multiplication. The training-free variant replaces the generator with direct optimization of mask logits against a reward-model loss. This machinery is what lets the method sample parameters from a frozen U-Net on the fly.

What would settle it

Run MaskUNet on COCO 2014 with the mask generator replaced by a random binary mask at the same mask ratio, and compare FID; if random masking reproduces the gain, the learned sample- and timestep-dependent mask content is not what matters. A second check is to verify whether the released implementation masks convolutional kernels or only linear layers, since the paper's equations show masking only for a linear layer.

Watch

Extended reading notes

Core claim

The paper's central discovery is that a pre-trained diffusion U-Net contains parameters that actively hurt generation, and that zeroing them out—including some large-magnitude ones—can make the network denoise better. The beneficial set of parameters changes with the noise level and with the specific image being generated, so MaskUNet learns a binary mask conditioned on the timestep embedding and the global-average-pooled latent code. At inference, the frozen U-Net weights are multiplied element-wise by this mask; no U-Net parameter is updated. On COCO 2014 the method lowers FID from 12.85 (SD 1.5) and 12.82 (LoRA) to 11.72, and on COCO 2017 from 23.39 to 21.88, with CLIP scores unchanged.

Load-bearing premise

The load-bearing premise is that zeroing out individual U-Net weights produces a valid, better denoising network for every mask and timestep, and that a timestep embedding plus a global-average-pooled latent code carries enough sample-specific information to choose those masks.

Editorial extensions

If this is right

  • MaskUNet reports FID 11.72 on COCO 2014 and 21.88 on COCO 2017, beating both SD 1.5 and LoRA while keeping the U-Net frozen; if the result holds, weight selection alone is a viable alternative to low-rank adaptation.
  • Full fine-tuning on the same data worsens FID (14.06 vs 12.85 for SD 1.5), which the paper reads as overfitting; masking avoids this by preserving the original weights.
  • Downstream methods such as Textual Inversion, ReVersion, and Text2Video-Zero show improved outputs when the same masking is added, so the benefit transfers beyond plain text-to-image generation.
  • In the training-free mode, adding MaskUNet to SynGen raises the color-attribution score on GenEval from 0.05 to 0.26 and the overall score from 0.43 to 0.50, indicating the mask can be optimized per prompt from a reward model without training.
  • Ablations show both timestep embedding and sample conditioning contribute to the FID gain: removing either worsens FID from 21.88 to 22.30 or 22.14, so both dependencies are claimed to matter.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the initial motivation is that random masks already help, the method suggests pre-trained diffusion U-Nets contain many high-quality subnetworks; if so, other sparse-subnetwork search or pruning schemes could achieve similar gains, a comparison the paper does not run.
  • Editorial inference: the mask generator's conditioning on global-average-pooled latents is a coarse summary of the sample; conditioning on spatially localized or attention-derived features might yield larger gains, a testable variant of MaskUNet.
  • Editorial inference: timestep-dependent masks could double as an interpretability tool—reading which weights survive at early versus late steps might expose where structure versus texture is processed inside the U-Net, a byproduct the paper only visualizes indirectly.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes MaskUNet, a method that learns timestep- and sample-dependent binary masks over the weights of a frozen, pretrained Stable Diffusion U-Net, without updating any U-Net parameters. Two variants are introduced: a training-based approach in which a small MLP mask generator is trained with the standard diffusion denoising loss, and a training-free approach in which mask logits are optimized per image using ImageReward and HPSv2. Experiments report improved FID on COCO 2014 and COCO 2017 relative to SD 1.5 and LoRA, improved semantic binding on T2I-CompBench and GenEval, and qualitative gains on downstream tasks such as DreamBooth, Textual Inversion, ReVersion, and Text2Video-Zero. The central claim is that applying learned masks at inference time enhances generation quality and prompt alignment while preserving the pretrained model's generalization.

Significance. If the mechanism were fully specified and the results reproducible, the idea would be interesting and potentially useful: it offers parameter-efficient enhancement of a frozen diffusion backbone, includes a training-free variant, and appears orthogonal to attention-based semantic binding methods such as SynGen. The paper includes ablations, a user study, and downstream task demonstrations, which strengthen the empirical story. However, the masking operation is formally defined only for a linear layer in the main text, while the SD 1.5 U-Net is predominantly convolutional; as written, the causal source of the reported FID gains is underspecified and the central contribution cannot yet be fully evaluated. The paper's explicit limitation that dynamic masking does not enable learning of new knowledge is appropriately stated, though its implications for the downstream personalization claims deserve more discussion.

major comments (4)
  1. [Sec. 3.2, Eqs. (3)-(7)] The masking mechanism is defined only for a single linear layer weight w in R^{Cout x Cin}, with output o = BMM(h, w_hat). SD 1.5's U-Net contains predominantly convolutional weights, and the paper does not state which layers are masked, how the reshaped mask m' is applied to convolutional kernels (which have shape R^{Cout x Cin x Kh x Kw}), or what fraction of the U-Net parameters are actually masked. As written, the reported 1.13 FID improvement could come from masking a small subset of linear projections rather than from the claimed dynamic selection of U-Net parameters. Please specify the exact masked layers, describe the convolution masking operation, quantify the parameter coverage, and ideally ablate linear-only versus convolution masking.
  2. [Sec. 3.3 and Algorithm 1] In the training-free variant, the algorithm optimizes mask logits l, but the dimensionality, initialization, and parameterization of l are not specified. Applying theta' <- theta ⊙ m in line 6 requires m to have the same shape as all masked parameters; no description is given for how m is constructed for the full U-Net or how reward gradients flow to logits of different shapes. In addition, the computational cost is substantial but not discussed: with lambda=15 and 15 inference steps, the algorithm performs 225 reward-model evaluations and 225 image decodings per generated image. Please clarify the optimization setup, report the added inference cost, and discuss the practical implications.
  3. [Table 1] The COCO FID values are reported as single point estimates with no standard errors, confidence intervals, or significance tests. The claimed improvements are on the order of 1.1 FID units (e.g., 12.85 to 11.72 on COCO 2014), which may be within run-to-run variation for 30k generated samples. Please provide multiple-seed statistics, error bars, or a significance test so the reader can assess whether the reported differences are meaningful.
  4. [Table 3] The 'Single' row for SynGen+MaskUNet is reported as 0.10, yet the reported Overall=0.50 is consistent with a much higher value: averaging the other rows with 0.98 in place of 0.10 gives approximately 0.50, matching the printed Overall. As printed, the 0.10 entry is internally inconsistent with the reported mean and contradicts the qualitative claim of improved single-object generation. Please correct the table entry or, if 0.10 is actually correct, explain how the Overall is computed and reconcile the contradiction.
minor comments (6)
  1. [Eq. (7)] Eq. (7) writes o = BMM(z, w_hat), but the input feature map was defined as h in Eq. (6); please use a consistent symbol (h) in the batch matrix multiplication.
  2. [Algorithm 1 vs. Eq. (8)] Eq. (8) defines z_t = epsilon_theta(z_{t+1}, t+1, c), while Algorithm 1 line 7 writes 'Predict noisy latent z_{t-1} <- epsilon_{theta'}(z_t, t, c)'. The indexing is inconsistent and should be aligned (the loop appears to run from T down to 0).
  3. [Sec. 4.1 and references] The text cites both ImageReward and HPSv2 as [62]; according to the reference list, HPSv2 is reference [61] and ImageReward is reference [62]. Please fix the citation.
  4. [Throughout] There are several typos and inconsistencies: 'paramaters' in the introduction, 'GENEV AL' in Sec. 4.1, 'Overrall' in Table 3, and 'Tex2Video-Zero' versus 'Text2Video-Zero' in Sec. 4.2.2 and Figure 7. Please proofread.
  5. [Fig. 9] Figure 9(b) plots 'Mask ratio v.s. FID' but the axes are unlabeled; the horizontal axis appears to be checkpoint iterations, and the vertical axes for mask ratio and FID are not distinguished. Please add clear axis labels and a legend.
  6. [Limitations] The Limitations section says dynamic masking 'does not enable learning of new knowledge,' but the paper also claims improvements in personalization tasks such as DreamBooth and Textual Inversion. Please explain how masking improves these tasks despite not learning new knowledge, or clarify the intended scope of the limitation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MaskUNet's masks are trained with diffusion or reward objectives and evaluated on external benchmarks; the only author-overlap citation is a non-load-bearing related-work reference.

full rationale

The paper's central claim is that a learned, timestep- and sample-dependent binary mask over frozen U-Net weights improves generation quality. The training-based variant optimizes the mask generator with the standard diffusion denoising loss (Eq. 2) on a LAION-art subset, while the reported COCO FID and CLIP numbers come from zero-shot evaluation on COCO 2014/2017, which is not part of the training objective. The training-free variant optimizes masks with ImageReward and HPSv2 (Eq. 10) and is evaluated on T2I-CompBench BLIP-VQA and GenEval, which are different metrics. No equation in the paper defines the reported FID or alignment score in terms of the mask generator's training loss, so the results are not equivalent to the inputs by construction. The architectural equations (Eqs. 3-7) specify how a mask is produced and applied, but they do not assume the conclusion that masking improves FID. The motivating random-mask experiment is an empirical observation, not a derived prediction. The only reference with overlapping authors, [32] (Faster Diffusion), appears in the related-work sentence about structured pruning acceleration and is not used to justify MaskUNet's mechanism or evaluation. A genuine limitation is that the paper only formalizes the masking operation for a linear layer (Eqs. 6-7) although the SD 1.5 U-Net is mostly convolutional, and the GenEval Single=0.10 entry appears to be a typo given the Overall=0.50 average; however, these are implementation and consistency concerns, not circularity. The derivation chain is therefore self-contained against external benchmarks.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method rests on two trained components (mask generator MLP, or test-time mask logits) and several hand-set hyperparameters. No external entities are postulated.

free parameters (4)
  • Gumbel-Sigmoid temperature tau and threshold delta = tau=1.0, delta=0.5 (training-free); unspecified for training-based
    These control the discreteness of the mask and are set by hand; they directly affect which weights are zeroed.
  • Reward balance factors omega = omega1=1.0 (ImageReward), omega2=5.0 (HPSv2)
    Hand-chosen weighting of the two reward losses in Eq. 10; affects training-free mask optimization.
  • Mask generator MLP parameters = learned from 20.1k Laion-art pairs
    The MLP (Eq. 3-4) is trained on a small subset and its weights are a learned component the method depends on.
  • Optimization iterations lambda (training-free) = 15
    Number of reward-model optimization steps per timestep; affects quality and compute.
assumptions (4)
  • standard math Diffusion denoising objective L_diff (Eq. 2) is a valid training signal for mask generation.
    Assumes standard DDPM-style denoising score matching.
  • domain assumption U-Net parameters act differently across timesteps: early steps encode structure, later steps encode texture (motivation).
    Based on prior work [5,13]; not directly verified in this paper.
  • ad hoc to paper The mask generator architecture (FC + GAP + 4-layer MLP) is sufficiently expressive to output beneficial masks.
    No analysis of capacity or sufficiency is provided; this is an architectural choice.
  • domain assumption The reward models (ImageReward, HPSv2) provide gradients that improve prompt-image alignment when optimized at test time.
    Borrowed from ReNO [8]; not newly justified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Not All Parameters Matter: Masking Diffusion Models for Enhancing Generation Ability." pith.science (2026). https://pith.science/paper/HLHRRBGM

@misc{pith2026250503097,
  author       = {Pith},
  title        = {Pith review of: Not All Parameters Matter: Masking Diffusion Models for Enhancing Generation Ability},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HLHRRBGM}},
  note         = {Machine review of arXiv:2505.03097}
}
read the original abstract

The diffusion models, in early stages focus on constructing basic image structures, while the refined details, including local features and textures, are generated in later stages. Thus the same network layers are forced to learn both structural and textural information simultaneously, significantly differing from the traditional deep learning architectures (e.g., ResNet or GANs) which captures or generates the image semantic information at different layers. This difference inspires us to explore the time-wise diffusion models. We initially investigate the key contributions of the U-Net parameters to the denoising process and identify that properly zeroing out certain parameters (including large parameters) contributes to denoising, substantially improving the generation quality on the fly. Capitalizing on this discovery, we propose a simple yet effective method-termed ``MaskUNet''- that enhances generation quality with negligible parameter numbers. Our method fully leverages timestep- and sample-dependent effective U-Net parameters. To optimize MaskUNet, we offer two fine-tuning strategies: a training-based approach and a training-free approach, including tailored networks and optimization functions. In zero-shot inference on the COCO dataset, MaskUNet achieves the best FID score and further demonstrates its effectiveness in downstream task evaluations. Project page: https://gudaochangsheng.github.io/MaskUnet-Page/

Figures

Figures reproduced from arXiv: 2505.03097 by the authors.

Figure 1
Figure 1. The motivation of our method. using certain random masks enhances the denoising capa￾bility of the U-Net architecture, thereby contributing to a superior output in terms of both fidelity and detail preserva￾tion. Further, we also visualize the corresponding features at different timesteps (see [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The pipeline of the MaskUnet. G-Sig represents the Gumbel-Sigmoid activate function. GAP is global average pooling. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Quality results compared to other methods. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Quality results compared to other methods. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Quality results by Textual Inversion [10] with or without mask. ReVersion +Mask ReVersion cat <R> stone <R> = painted on ReVersion ReVersion +Mask rabbit <R> cup <R> = inside dog <R> jade <R> = carved by ReVersion ReVersion +Mask [PITH_FULL_IMAGE:figures/full_fig_p007…
Figure 6
Figure 6. Figure 6: Quality results by ReVersion [25] with or without mask. Version might place the rabbit on the surface of the cup or outside it, but with MaskUnet, sensitivity to the “inside” embedding is increased, resulting in images with the cor￾rect relational context. Additionally…
Figure 7
Figure 7. Figure 7: Quality results by Text2Video-Zero [29] with or without mask [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Quantitative results compared to other methods. [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: (a) Visualization of image distributions for different [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

68 extracted references · 31 canonical work pages

  1. [1]

    A-star: Test-time attention segregation and retention for text-to-image synthesis

    Aishwarya Agarwal, Srikrishna Karanam, KJ Joseph, Apoorv Saxena, Koustava Goswami, and Balaji Vasan Srini- vasan. A-star: Test-time attention segregation and retention for text-to-image synthesis. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 2283– 2293, 2023. 3

    work page 2023

  2. [2]

    Attend-and-excite: Attention-based se- mantic guidance for text-to-image diffusion models

    Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and-excite: Attention-based se- mantic guidance for text-to-image diffusion models. ACM Transactions on Graphics (TOG), 42(4):1–10, 2023. 1, 3, 5, 7

    work page 2023

  3. [3]

    A cat is a cat (not a dog!): Unraveling information mix- ups in text-to-image encoders through causal analysis and embedding optimization

    Chieh-Yun Chen, Chiang Tseng, Li-Wu Tsao, and Hong-Han Shuai. A cat is a cat (not a dog!): Unraveling information mix- ups in text-to-image encoders through causal analysis and embedding optimization. Advances in Neural Information Processing Systems, 2024. 3

    work page 2024

  4. [4]

    Pixart-{\delta}: Fast and controllable image generation with latent consistency models

    Junsong Chen, Yue Wu, Simian Luo, Enze Xie, Sayak Paul, Ping Luo, Hang Zhao, and Zhenguo Li. Pixart-{\delta}: Fast and controllable image generation with latent consistency models. arXiv preprint arXiv:2401.05252, 2024. 3

    arXiv 2024

  5. [5]

    Perception pri- oritized training of diffusion models

    Jooyoung Choi, Jungbeom Lee, Chaehun Shin, Sungwon Kim, Hyunwoo Kim, and Sungroh Yoon. Perception pri- oritized training of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11472–11481, 2022. 1

    work page 2022

  6. [6]

    Swiftbrush v2: Make your one-step diffusion model better than its teacher

    Trung Dao, Thuan Hoang Nguyen, Thanh Le, Duc Vu, Khoi Nguyen, Cuong Pham, and Anh Tran. Swiftbrush v2: Make your one-step diffusion model better than its teacher. In European Conference on Computer Vision , pages 176–192. Springer, 2025. 3

    work page 2025

  7. [7]

    Learning universal policies via text-guided video genera- tion

    Yilun Du, Sherry Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Josh Tenenbaum, Dale Schuurmans, and Pieter Abbeel. Learning universal policies via text-guided video genera- tion. Advances in Neural Information Processing Systems , 36, 2024. 1, 3

    work page 2024

  8. [8]

    Reno: Enhancing one-step text-to-image models through reward-based noise optimiza- tion

    Luca Eyring, Shyamgopal Karthik, Karsten Roth, Alexey Dosovitskiy, and Zeynep Akata. Reno: Enhancing one-step text-to-image models through reward-based noise optimiza- tion. arXiv preprint arXiv:2406.04312, 2024. 3, 4

    arXiv 2024

Show all 68 references
  1. [9]

    Training-free struc- tured diffusion guidance for compositional text-to-image syn- thesis

    Weixi Feng, Xuehai He, Tsu-Jui Fu, Varun Jampani, Ar- jun Reddy Akula, Pradyumna Narayana, Sugato Basu, Xin Eric Wang, and William Yang Wang. Training-free struc- tured diffusion guidance for compositional text-to-image syn- thesis. In The Eleventh International Conference on ...

    2023

  2. [10]

    An image is worth one word: Personalizing text-to-image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit Haim Bermano, Gal Chechik, and Daniel Cohen-or. An image is worth one word: Personalizing text-to-image generation using textual inversion. In The Eleventh Interna- tional Conference on Learning Representations, 2023. 3...

    2023

  3. [11]

    How does selective mechanism im- prove self-attention networks? In Proceedings of the 58th Annual Meeting of the Association for Computational Lin- guistics, pages 2986–2995, 2020

    Xinwei Geng, Longyue Wang, Xing Wang, Bing Qin, Ting Liu, and Zhaopeng Tu. How does selective mechanism im- prove self-attention networks? In Proceedings of the 58th Annual Meeting of the Association for Computational Lin- guistics, pages 2986–2995, 2020. 4

    2020

  4. [12]

    Geneval: An object-focused framework for evaluating text- to-image alignment

    Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text- to-image alignment. Advances in Neural Information Pro- cessing Systems, 36, 2024. 1, 5

    2024

  5. [13]

    Towards practical plug-and-play diffusion models

    Hyojun Go, Yunsung Lee, Jin-Young Kim, Seunghyun Lee, Myeongho Jeong, Hyun Seung Lee, and Seungtaek Choi. Towards practical plug-and-play diffusion models. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1962–1971, 2023. 1

    1962

  6. [14]

    Parameter- efficient transfer learning with diff pruning

    Demi Guo, Alexander M Rush, and Yoon Kim. Parameter- efficient transfer learning with diff pruning. arXiv preprint arXiv:2012.07463, 2020. 3

    2012 arXiv

  7. [15]

    I2v-adapter: A general image-to-video adapter for diffusion models

    Xun Guo, Mingwu Zheng, Liang Hou, Yuan Gao, Yufan Deng, Pengfei Wan, Di Zhang, Yufan Liu, Weiming Hu, Zhengjun Zha, et al. I2v-adapter: A general image-to-video adapter for diffusion models. InACM SIGGRAPH 2024 Con- ference Papers, pages 1–12, 2024. 3

    2024

  8. [16]

    Freestyle: Free lunch for text- guided style transfer using diffusion models

    Feihong He, Gang Li, Mengyuan Zhang, Leilei Yan, Lingyu Si, Fanzhang Li, and Li Shen. Freestyle: Free lunch for text- guided style transfer using diffusion models. arXiv preprint arXiv:2401.15636, 2024. 3

    2024 arXiv

  9. [17]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 1

    2016

  10. [18]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bern- hard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017. 5

    2017

  11. [19]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 5

    2022 arXiv

  12. [20]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 1, 3

    2020

  13. [21]

    Lora: Low- rank adaptation of large language models

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low- rank adaptation of large language models. In International Conference on Learning Representations, 2022. 3, 5, 6

    2022

  14. [22]

    Sara: High-efficient diffusion model fine-tuning with progressive sparse low-rank adap- tation

    Teng Hu, Jiangning Zhang, Ran Yi, Hongrui Huang, Yabiao Wang, and Lizhuang Ma. Sara: High-efficient diffusion model fine-tuning with progressive sparse low-rank adap- tation. arXiv preprint arXiv:2409.06633, 2024. 3

    2024 arXiv

  15. [23]

    Densely connected convolutional net- works

    Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kil- ian Q Weinberger. Densely connected convolutional net- works. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700–4708, 2017. 1

    2017

  16. [24]

    T2i-compbench: A comprehensive benchmark for open- world compositional text-to-image generation

    Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A comprehensive benchmark for open- world compositional text-to-image generation. Advances in Neural Information Processing Systems , 36:78723–78747,

  17. [25]

    Reversion: Diffusion-based relation inversion from images

    Ziqi Huang, Tianxing Wu, Yuming Jiang, Kelvin CK Chan, and Ziwei Liu. Reversion: Diffusion-based relation inversion from images. arXiv preprint arXiv:2303.13495, 2023. 1, 3, 5, 6, 7

    2023 arXiv

  18. [26]

    Scaling up visual and vision-language representation learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In International confer- ence on machine learning, pages 4904–491...

    2021

  19. [27]

    Progressive growing of GANs for improved quality, stabil- ity, and variation

    Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of GANs for improved quality, stabil- ity, and variation. In International Conference on Learning Representations, 2018. 1

    2018

  20. [28]

    Alias-free generative adversarial networks

    Tero Karras, Miika Aittala, Samuli Laine, Erik H ¨ark¨onen, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Alias-free generative adversarial networks. Advances in neural infor- mation processing systems, 34:852–863, 2021. 1

    2021

  21. [29]

    Text2video-zero: Text- to-image diffusion models are zero-shot video generators

    Levon Khachatryan, Andranik Movsisyan, Vahram Tade- vosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Text2video-zero: Text- to-image diffusion models are zero-shot video generators. In Proceedings of the IEEE/CVF International Conference on Comput...

    2023

  22. [30]

    Similarity of neural network represen- tations revisited

    Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network represen- tations revisited. In International conference on machine learning, pages 3519–3529. PMLR, 2019. 1

    2019

  23. [31]

    Multi-concept customization of text- to-image diffusion

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shecht- man, and Jun-Yan Zhu. Multi-concept customization of text- to-image diffusion. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 1931–1941, 2023. 3

    1931

  24. [32]

    Faster diffusion: Rethinking the role of unet encoder in dif- fusion models

    Senmao Li, Taihang Hu, Fahad Shahbaz Khan, Linxuan Li, Shiqi Yang, Yaxing Wang, Ming-Ming Cheng, and Jian Yang. Faster diffusion: Rethinking the role of unet encoder in dif- fusion models. arXiv e-prints, pages arXiv–2312, 2023. 3

    2023

  25. [33]

    Photomaker: Customizing realistic human photos via stacked id embedding

    Zhen Li, Mingdeng Cao, Xintao Wang, Zhongang Qi, Ming- Ming Cheng, and Ying Shan. Photomaker: Customizing realistic human photos via stacked id embedding. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8640–8650, 2024. 1, 3

    2024

  26. [34]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceeding...

    2014

  27. [35]

    Dora: Weight-decomposed low-rank adaptation

    Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. Dora: Weight-decomposed low-rank adaptation. arXiv preprint arXiv:2402.09353, 2024. 3

    2024 arXiv

  28. [36]

    Decoupled weight decay regularization

    I Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 5

    2017 arXiv

  29. [37]

    Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems , 35:5775–5787,

  30. [38]

    Latent consistency models: Synthesizing high- resolution images with few-step inference

    Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high- resolution images with few-step inference. arXiv preprint arXiv:2310.04378, 2023. 3

    2023 arXiv

  31. [39]

    The surprising effectiveness of skip-tuning in diffusion sam- pling

    Jiajun Ma, Shuchen Xue, Tianyang Hu, Wenjia Wang, Zhao- qiang Liu, Zhenguo Li, Zhi-Ming Ma, and Kenji Kawaguchi. The surprising effectiveness of skip-tuning in diffusion sam- pling. arXiv preprint arXiv:2402.15170, 2024. 3

    2024 arXiv

  32. [40]

    Deepcache: Accelerating diffusion models for free

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. Deepcache: Accelerating diffusion models for free. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15762–15772, 2024. 3

    2024

  33. [41]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 4296–4304, 2024. 1, 3

    2024

  34. [42]

    Swiftbrush: One- step text-to-image diffusion model with variational score dis- tillation

    Thuan Hoang Nguyen and Anh Tran. Swiftbrush: One- step text-to-image diffusion model with variational score dis- tillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7807–7816,

  35. [43]

    Not all noises are created equally: Diffusion noise selection and optimization

    Zipeng Qi, Lichen Bai, Haoyi Xiong, et al. Not all noises are created equally: Diffusion noise selection and optimization. arXiv preprint arXiv:2407.14041, 2024. 3

    2024 arXiv

  36. [44]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  37. [45]

    X-adapter: Adding universal compatibility of plugins for up- graded diffusion model

    Lingmin Ran, Xiaodong Cun, Jia-Wei Liu, Rui Zhao, Song Zijie, Xintao Wang, Jussi Keppo, and Mike Zheng Shou. X-adapter: Adding universal compatibility of plugins for up- graded diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...

    2024

  38. [46]

    Linguistic binding in dif- fusion models: Enhancing attribute correspondence through attention map alignment

    Royi Rassin, Eran Hirsch, Daniel Glickman, Shauli Ravfogel, Yoav Goldberg, and Gal Chechik. Linguistic binding in dif- fusion models: Enhancing attribute correspondence through attention map alignment. Advances in Neural Information Processing Systems, 36, 2024. 1, 3, 5, 7

    2024

  39. [47]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 3, 5, 7

    2022

  40. [48]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven gen- eration

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven gen- eration. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages ...

  41. [49]

    Laion-400m: Open dataset of clip-filtered 400 million image-text pairs

    Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. arXiv preprint arXiv:2111.02114, 2021. 3

    2021 arXiv

  42. [50]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural In- f...

    2022

  43. [51]

    Grad-cam: Visual explanations from deep networks via gradient-based localization

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE in- ternational conference on computer vision , pages 618–626,

  44. [52]

    Freeu: Free lunch in diffusion u-net

    Chenyang Si, Ziqi Huang, Yuming Jiang, and Ziwei Liu. Freeu: Free lunch in diffusion u-net. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4733–4743, 2024. 3

    2024

  45. [53]

    Very deep convolutional networks for large- scale image recognition

    Karen Simonyan. Very deep convolutional networks for large- scale image recognition. arXiv preprint arXiv:1409.1556 ,

    arXiv

  46. [54]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International confer- ence on machine learning, pages 2256–2265. PMLR, 2015. 1, 3

    2015

  47. [55]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021. 3

    2021

  48. [56]

    Generative modeling by estimating gradients of the data distribution

    Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019. 3

    2019

  49. [57]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020. 3

    2011 arXiv

  50. [58]

    Going deeper with convolutions

    Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1–9, 2015. 1

    2015

  51. [59]

    Multi-concept t2i- zero: Tweaking only the text embeddings and nothing else

    Hazarapet Tunanyan, Dejia Xu, Shant Navasardyan, Zhangyang Wang, and Humphrey Shi. Multi-concept t2i- zero: Tweaking only the text embeddings and nothing else. arXiv preprint arXiv:2310.07419, 2023. 3

    2023 arXiv

  52. [60]

    Visualizing data using t-sne

    Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. JMLR, 9(11), 2008. 8

    2008

  53. [61]

    Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341 ,

    arXiv

  54. [62]

    Imagereward: Learning and evaluating human preferences for text-to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Sys- tems, 36, 2024. 2, 5

    2024

  55. [63]

    Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models. arXiv preprint arXiv:2308.06721 ,

    arXiv

  56. [64]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. InPro- ceedings of the IEEE/CVF International Conference on Com- puter Vision, pages 3836–3847, 2023. 1, 3

    2023

  57. [65]

    gddim: Generalized denoising diffusion implicit models

    Qinsheng Zhang, Molei Tao, and Yongxin Chen. gddim: Generalized denoising diffusion implicit models. In The Eleventh International Conference on Learning Represen- tations, 2023. 3, 5

    2023

  58. [66]

    Real- world image variation by aligning diffusion inversion chain

    Yuechen Zhang, Jinbo Xing, Eric Lo, and Jiaya Jia. Real- world image variation by aligning diffusion inversion chain. Advances in Neural Information Processing Systems , 36,

  59. [67]

    Learning deep features for discrimi- native localization

    Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discrimi- native localization. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 2921–2929,

  60. [68]

    Storydiffusion: Consistent self-attention for long-range image and video generation

    Yupeng Zhou, Daquan Zhou, Ming-Ming Cheng, Jiashi Feng, and Qibin Hou. Storydiffusion: Consistent self-attention for long-range image and video generation. arXiv preprint arXiv:2405.01434, 2024. 1, 3

    2024 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.