REVIEW 24 cited by
From Reusing to Forecasting: Accelerating Diffusion Models with TaylorSeers
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
From Reusing to Forecasting: Accelerating Diffusion Models with TaylorSeers
abstract
Diffusion Transformers (DiT) have revolutionized high-fidelity image and video synthesis, yet their computational demands remain prohibitive for real-time applications. To solve this problem, feature caching has been proposed to accelerate diffusion models by caching the features in the previous timesteps and then reusing them in the following timesteps. However, at timesteps with significant intervals, the feature similarity in diffusion models decreases substantially, leading to a pronounced increase in errors introduced by feature caching, significantly harming the generation quality. To solve this problem, we propose TaylorSeer, which firstly shows that features of diffusion models at future timesteps can be predicted based on their values at previous timesteps. Based on the fact that features change slowly and continuously across timesteps, TaylorSeer employs a differential method to approximate the higher-order derivatives of features and predict features in future timesteps with Taylor series expansion. Extensive experiments demonstrate its significant effectiveness in both image and video synthesis, especially in high acceleration ratios. For instance, it achieves an almost lossless acceleration of 4.99$\times$ on FLUX and 5.00$\times$ on HunyuanVideo without additional training. On DiT, it achieves $3.41$ lower FID compared with previous SOTA at $4.53$$\times$ acceleration. %Our code is provided in the supplementary materials and will be made publicly available on GitHub. Our codes have been released in Github:https://github.com/Shenyi-Z/TaylorSeer
Forward citations
Cited by 24 Pith papers
-
JAGG: Jacobian-Aggregated Group Gradient for Efficient GRPO Training of Diffusion Models
JAGG replaces per-step gradient backpropagation in diffusion GRPO with two endpoint backward passes joined by timestep-weighted interpolation, giving ~2x backward-pass savings at modest quality cost.
-
Efficient Video Diffusion Models: Advancements and Challenges
A survey that groups efficient video diffusion methods into four paradigms—step distillation, efficient attention, model compression, and cache/trajectory optimization—and outlines open challenges for practical use.
-
LayerCache: Exploiting Layer-wise Velocity Heterogeneity for Efficient Flow Matching Inference
LayerCache enables per-layer-group caching in flow matching models via adaptive JVP span selection and greedy 3D scheduling, delivering 1.37x speedup with PSNR 37.46 dB, SSIM 0.9834, and LPIPS 0.0178 on Qwen-Image.
-
DRIFT: Harnessing Inherent Fault Tolerance for Efficient and Reliable Diffusion Model Inference
DRIFT uses resilience analysis, targeted DVFS, and adaptive rollback ABFT to deliver 36% average energy savings or 1.7x speedup in diffusion model inference while preserving generation quality.
-
Beyond Few-Step Inference: Accelerating Video Diffusion Transformer Model Serving with Inter-Request Caching Reuse
Chorus accelerates video DiT serving up to 45% via inter-request caching reuse in a three-stage denoising strategy with token-guided attention amplification.
-
DiTango: Cost-Effective Parallel Diffusion Generation with Selective Attention State Reuse
Selective reuse of composed attention states across denoising steps lets DiTango skip both remote KV communication and attention compute for low-contribution sequence partitions, cutting multi-GPU diffusion latency by...
-
CODA: Algorithm-Hardware Co-design for Edge Video Diffusion via NMP-Enabled Compute-Cache Operator Disaggregation
Disaggregating cache operators from compute and overlapping them across the two classifier-free-guidance branches turns cross-timestep caching into up to 1.80x real end-to-end speedup on edge GPUs when the cache overf...
-
LearniBridge: Learnable Calibration of Feature Caching for Diffusion Models Acceleration
LearniBridge learns shared low-rank calibration updates via LoRA to enable high-ratio feature caching in DiTs using only 3-5 samples, delivering up to 5.87x acceleration on FLUX with quality gains on video models.
-
Budget-Constrained Step-Level Diffusion Caching
BudCache optimizes step cache policies for a fixed inference budget in diffusion models via combinatorial search, outperforming threshold heuristics in quality on FLUX.1-dev and Wan2.1.
-
RT-Lynx: Putting the GEMM Sparsity In a Right Way for Diffusion Models
RT-Lynx shifts DiT sparsity from weights to activations, reports up to 1.55x linear-layer speedup while preserving generation quality across multiple diffusion models.
-
Motion-Aware Caching for Efficient Autoregressive Video Generation
MotionCache accelerates autoregressive video generation up to 6.28x by motion-weighted cache reuse based on inter-frame differences, with negligible quality loss on SkyReels-V2 and MAGI-1.
-
CoCoDiff: Optimizing Collective Communications for Distributed Diffusion Transformer Inference Under Ulysses Sequence Parallelism
CoCoDiff achieves 3.6x average and 8.4x peak speedup for distributed DiT inference on up to 96 GPU tiles via tile-aware all-to-all, V-first scheduling, and selective V communication.
-
DiffSparse: Accelerating Diffusion Transformers with Learned Token Sparsity
A learnable cost predictor plus dynamic programming allocates layer-wise token sparsity for diffusion transformers, removing forced full steps and cutting ~54% compute on PixArt-α without quality loss.
-
World Action Models are Zero-shot Policies
DreamZero uses a 14B video diffusion model as a World Action Model to achieve over 2x better zero-shot generalization on real robots than state-of-the-art VLAs, real-time 7Hz closed-loop control, and cross-embodiment ...
-
FlashEdit: Decoupling Speed, Structure, and Semantics for Precise Image Editing
FlashEdit delivers real-time localized text-guided image editing under 0.2 seconds via cycle-consistent one-step inversion, background shield, and sparsified spatial cross-attention, achieving over 150x speedup on PIE-Bench.
-
OmniCache: Multidimensional Hierarchical Feature Caching For Diffusion Models
Training-free hierarchical feature caching across token, frame, block, and layer axes cuts diffusion inference latency up to 35% while preserving quality better than averaging-based token merging.
-
ACID: Adaptive Caching for vIDeo generation
ACID dynamically switches caching thresholds based on drift-signal rate of change, recovering near-conservative quality at substantially higher video-diffusion inference speeds.
-
ACID: Adaptive Caching for vIDeo generation
Adaptive threshold switching on the drift signal's rate of change expands the quality-vs-speed Pareto frontier of TeaCache, EasyCache, and DiCache across HunyuanVideo, Wan 2.1, and CogVideoX.
-
OTCache: Optimal Transport for Geometry-Aware Caching in Diffusion Models
OTCache uses optimal transport to interpolate caching schedules between a graph-based reference and an Optuna-optimized anchor, delivering 3.66x-4.7x speedups on FLUX.1, Qwen-Image and HunyuanVideo with improved fidelity.
-
ResilPhase: Plug-and-Play Phase Mapping and Noise-Resilient Macro-Trajectory Extrapolation for Diffusion Acceleration
ResilPhase accelerates DiT inference via stable ODE macro-trajectory extrapolation with derivative-free barycentric Lagrange extrapolator and bounded phase mapping, claiming SOTA fidelity at high acceleration ratios o...
-
Dynamic Video Generation: Shaping Video Generation Across Time and Space
DVG dynamically selects content-aware spatio-temporal acceleration strategies for diffusion-based video generation, delivering up to 7x speedup with near-lossless quality on models like HunyuanVideo.
-
Motion-Aware Caching for Efficient Autoregressive Video Generation
MotionCache speeds up autoregressive video generation by 6.28x on SkyReels-V2 and 1.64x on MAGI-1 via motion-weighted cache reuse based on inter-frame differences, with negligible quality loss on VBench.
-
Training-free, Perceptually Consistent Low-Resolution Previews with High-Resolution Image for Efficient Workflows of Diffusion Models
A commutator-zero condition enables training-free generation of perceptually consistent low-resolution previews for high-resolution diffusion model outputs, achieving up to 33% computation reduction.
-
Fast-SAM3D: 3Dfy Anything in Images but Faster
A training-free framework accelerates SAM3D single-view 3D reconstruction by up to 2.67× per object and 2.01× per scene while keeping geometric fidelity roughly unchanged.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.