REVIEW 6 cited by
Cache Me if You Can: Accelerating Diffusion Models through Block Caching
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Diffusion models have recently revolutionized the field of image synthesis due to their ability to generate photorealistic images. However, one of the major drawbacks of diffusion models is that the image generation process is costly. A large image-to-image network has to be applied many times to iteratively refine an image from random noise. While many recent works propose techniques to reduce the number of required steps, they generally treat the underlying denoising network as a black box. In this work, we investigate the behavior of the layers within the network and find that 1) the layers' output changes smoothly over time, 2) the layers show distinct patterns of change, and 3) the change from step to step is often very small. We hypothesize that many layer computations in the denoising network are redundant. Leveraging this, we introduce block caching, in which we reuse outputs from layer blocks of previous steps to speed up inference. Furthermore, we propose a technique to automatically determine caching schedules based on each block's changes over timesteps. In our experiments, we show through FID, human evaluation and qualitative analysis that Block Caching allows to generate images with higher visual quality at the same computational cost. We demonstrate this for different state-of-the-art models (LDM and EMU) and solvers (DDIM and DPM).
Forward citations
Cited by 6 Pith papers
-
OmniCache: A Trajectory-Oriented Global Perspective on Training-Free Cache Reuse for Diffusion Transformer Models
A training-free cache-reuse scheme that spreads computation across the full diffusion trajectory and subtracts estimated noise, accelerating DiT sampling with claimed competitive quality.
-
Forecast then Calibrate: Feature Caching as ODE for Efficient Diffusion Transformers
A training-free predictor-corrector method that accelerates Diffusion Transformers by solving a feature-ODE, achieving large compute reductions with modest quality loss.
-
Chipmunk: Training-Free Acceleration of Diffusion Transformers with Dynamic Column-Sparse Deltas
Chipmunk speeds up diffusion transformer inference by recomputing, at each step, only the top percent of attention and MLP activation columns that change most between steps, caching the rest in column-sparse GPU kernels.
-
dKV-Cache: The Cache for Diffusion Language Models
dKV-Cache reuses cached key and value states of decoded tokens during diffusion LM denoising, delivering 2-10x faster inference with near-lossless quality on several benchmarks.
-
Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compression
ScaleKV cuts KV cache memory for Visual Autoregressive text-to-image generation to 10% by classifying layers as drafters or refiners per scale and pruning low-attention tokens while keeping benchmark scores nearly unchanged.
-
Predictive Feature Caching for Training-free Acceleration of Molecular Geometry Generation
Predictive feature caching, borrowed from image diffusion, speeds up molecular flow-matching generation by 2-3x at near-matched quality by forecasting hidden features instead of recomputing them.
Discussion (0). Sign in to comment.