RECORD
Fast Sampling Through The Reuse Of Attention Maps In Diffusion Models
Computer Vision and Pattern Recognition · Submitted 13 Dec 2023
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim We empirically compare our reuse strategies with few-step sampling procedures of comparable latency, finding that reuse generates images that are closer to those produced by the original high-latency diffusion model.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Text-to-image diffusion models have demonstrated unprecedented capabilities for flexible and realistic image synthesis. Nevertheless, these models rely on a time-consuming sampling procedure, which has motivated attempts to reduce their latency. When improving efficiency, researchers often use the original diffusion model to train an additional network designed specifically for fast image generation. In contrast, our approach seeks to reduce latency directly, without any retraining, fine-tuning, or knowledge distillation. In particular, we find the repeated calculation of attention maps to be costly yet redundant, and instead suggest reusing them during sampling. Our specific reuse strategies are based on ODE theory, which implies that the later a map is reused, the smaller the distortion in the final image. We empirically compare our reuse strategies with few-step sampling procedures of comparable latency, finding that reuse generates images that are closer to those produced by the original high-latency diffusion model.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Text-to-image diffusion models have demonstrated unprecedented capabilities for flexible and realistic image synthesis. | background 1.00 | 0.04 | 0.02 | asserted 1.00 |
| s1 | Nevertheless, these models rely on a time-consuming sampling procedure, which has motivated attempts to reduce their latency. | background 0.94 | 0.06 | 0.03 | asserted 1.00 |
| s2 | When improving efficiency, researchers often use the original diffusion model to train an additional network designed specifically for fast image generation. | background 1.00 | 0.04 | 0.03 | asserted 0.99 |
| s3 | In contrast, our approach seeks to reduce latency directly, without any retraining, fine-tuning, or knowledge distillation. | contribution 0.94 | 0.65 | 0.14 | asserted 0.98 |
| s4 | In particular, we find the repeated calculation of attention maps to be costly yet redundant, and instead suggest reusing them during sampling. | method 0.50 | 0.94 | 0.29 | asserted 0.97 |
| s5 | Our specific reuse strategies are based on ODE theory, which implies that the later a map is reused, the smaller the distortion in the final image. | method 0.74 | 0.70 | 0.14 | asserted 0.98 |
| s6 | We empirically compare our reuse strategies with few-step sampling procedures of comparable latency, finding that reuse generates images that are closer to those produced by the original high-latency diffusion model. | result 1.00 | 0.96 | 0.66 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v4
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Text-to-image diffusion models have demonstrated unprecedented capabilities for flexible and realistic image synthesis. Nevertheless, these models rely on a time-consuming sampling procedure, which has motivated attempts to reduce their latency. When improving efficiency, researchers often use the original diffusion model to train an additional network designed specifically for fast image generation. In contrast, our approach seeks to reduce latency directly, without any retraining, fine-tuning, or knowledge distillation. In particular, we find the repeated calculation of attention maps to be costly yet redundant, and instead suggest reusing them during sampling. Our specific reuse strategies are based on ODE theory, which implies that the later a map is reused, the smaller the distortion in the final image. We empirically compare our reuse strategies with few-step sampling procedures of comparable latency, finding that reuse generates images that are closer to those produced by the original high-latency diffusion model.
Human review
References
How the literature uses it
3 arXiv papers cite this in 3 sentences
Papers citing it · 2 sentences without a confident label
Cites it as background 1 paper
-
[ref] proposes to reuse the attention map.
Learning-to-Cache: Accelerating Diffusion Transformer via Layer Caching · Related Work · 0.84
What it cites · 6 sentences without a confident label
Uses the method or tool of 1 paper
-
All of the figures presented in this section are run using Stable Diffusion v1.5 with 20-step DDIM [ref] as the (base) sampling proceduresampler.
Denoising Diffusion Implicit Models · Analysis · 0.94
Cites as background 6 papers
-
Several researchers [ref] have introduced a notion of consistency, producing single-step solvers that generate (or sample from) any point along the whole diffusion trajectory.
TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation · Related Work · 0.95
-
Several researchers [ref] have introduced a notion of consistency, producing single-step solvers that generate (or sample from) any point along the whole diffusion trajectory.
BOOT: Data-free Distillation of Denoising Diffusion Models with Bootstrapping · Related Work · 0.95
-
Alternatively, Liu et al. [ref] reduced the curvature of the score function, which made the U-Net more accurate for large step-sizes.
InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation · Related Work · 0.88
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.