Pith. sign in

REVIEW 45 cited by

$\Delta$-DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformers

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.01125 v1 pith:FNX3XV22 submitted 2024-06-03 cs.CV

Pengtao Chen , Mingzhu Shen , Peng Ye , Jianjian Cao , Chongjun Tu , Christos-Savvas Bouganis , Yiren Zhao , Tao Chen This is my paper · ORCID
classification cs.CV
keywords accelerationblocksdeltagenerationcachediffusionframeworkfront
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
abstract

Diffusion models are widely recognized for generating high-quality and diverse images, but their poor real-time performance has led to numerous acceleration works, primarily focusing on UNet-based structures. With the more successful results achieved by diffusion transformers (DiT), there is still a lack of exploration regarding the impact of DiT structure on generation, as well as the absence of an acceleration framework tailored to the DiT architecture. To tackle these challenges, we conduct an investigation into the correlation between DiT blocks and image generation. Our findings reveal that the front blocks of DiT are associated with the outline of the generated images, while the rear blocks are linked to the details. Based on this insight, we propose an overall training-free inference acceleration framework $\Delta$-DiT: using a designed cache mechanism to accelerate the rear DiT blocks in the early sampling stages and the front DiT blocks in the later stages. Specifically, a DiT-specific cache mechanism called $\Delta$-Cache is proposed, which considers the inputs of the previous sampling image and reduces the bias in the inference. Extensive experiments on PIXART-$\alpha$ and DiT-XL demonstrate that the $\Delta$-DiT can achieve a $1.6\times$ speedup on the 20-step generation and even improves performance in most cases. In the scenario of 4-step consistent model generation and the more challenging $1.12\times$ acceleration, our method significantly outperforms existing methods. Our code will be publicly available.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 45 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. GeoCache: Training-Free Acceleration of Multi-View Texture Diffusion via Geometric Delta Transport

    cs.CV 2026-08 conditional novelty 7.0 of 10

    A training-free plugin that speeds up multi-view texture diffusion by transporting anchor views' per-step clean-signal updates to non-anchor views via geometry, preserving fidelity better than temporal caches at over ...

  2. JAGG: Jacobian-Aggregated Group Gradient for Efficient GRPO Training of Diffusion Models

    cs.LG 2026-07 conditional novelty 7.0 of 10

    JAGG replaces per-step gradient backpropagation in diffusion GRPO with two endpoint backward passes joined by timestep-weighted interpolation, giving ~2x backward-pass savings at modest quality cost.

  3. LMEB: Long-horizon Memory Embedding Benchmark

    cs.CL 2026-03 unverdicted novelty 7.0 of 10

    LMEB is a new benchmark that evaluates embedding models on long-horizon memory retrieval and shows this skill is largely orthogonal to traditional passage-retrieval performance.

  4. Fast Video Generation with Sliding Tile Attention

    cs.CV 2025-02 conditional novelty 7.0 of 10

    Sliding tile attention (STA) replaces full 3D attention in video diffusion transformers with dense tile-local windows, achieving 1.89x training-free and up to 3.53x fine-tuned end-to-end speedups on HunyuanVideo with ...

  5. From Local Mismatch to Global Impact: Optimizing Cache Reuse Policy for Efficient Diffusion

    cs.AI 2026-08 conditional novelty 6.0 of 10

    GCache learns a global-impact-aware cache reuse policy for diffusion models, reporting lower LPIPS at matched speedups than prior caching baselines.

  6. BAG: Budget-Aware Gating for Diffusion Caching

    cs.CV 2026-08 conditional novelty 6.0 of 10

    A single learned gate, trained by copying offline-searched cache schedules, decides per step how to spend a diffusion sampling budget and beats static and heuristic caching baselines at matched compute.

  7. Deltoris: Enabling Real-time VLA Inference in Embodied AI via Bit-level Sparsity and Speculative Inference

    cs.AR 2026-08 conditional novelty 6.0 of 10

    Temporal-aware bit-level sparsity plus batched speculative verification lets a co-designed accelerator run diffusion-based vision-language-action models up to 34x faster than a mobile GPU.

  8. FeatFix: Reuse What You Verify through Local Exact-Feature Correction for Faster Cached Diffusion Inference

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Forwarding the already-computed exact feature at verification sites—instead of discarding it—improves diffusion generation quality at matched compute and reaches up to 6.70× speedup when wrapped around aggressive caches.

  9. Importance-Aware OBS Pruning for Diffusion Models

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Injecting spatial importance maps (e.g., CFG delta) into the OBS Hessian improves subject preservation in pruned diffusion models at high sparsity, but gains over the baseline are small and without error bars.

  10. DiTango: Cost-Effective Parallel Diffusion Generation with Selective Attention State Reuse

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Selective reuse of composed attention states across denoising steps lets DiTango skip both remote KV communication and attention compute for low-contribution sequence partitions, cutting multi-GPU diffusion latency by...

  11. Kaleido: Algorithm-Hardware Co-Design for Video Diffusion Transformers by Exploiting Latent Space Correlations

    cs.AR 2026-07 conditional novelty 6.0 of 10

    A channel-wise reuse algorithm plus a reconfigurable systolic accelerator skips redundant vDiT attention and MLP computation, achieving up to 5.9x speedup and 16x energy savings.

  12. Flash-BoN: Instant Drafts for Inference-Time Scaling in Diffusion Models

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Under wall-clock budgets, cheap multi-knob drafts plus multi-stage verification outperform guided intermediate search for diffusion T2I inference-time scaling.

  13. RFM-Editing 2: Text-Guided Audio Editing with Rectified Flow Matching and Coarse-to-Fine Diffusion Transformers

    cs.SD 2026-06 unverdicted novelty 6.0 of 10

    Hybrid two-stage diffusion transformer architecture for instruction-guided audio editing via rectified flow that performs joint attention at low resolution then alternates joint and cross-attention at high resolution ...

  14. DiffSparse: Accelerating Diffusion Transformers with Learned Token Sparsity

    cs.CV 2026-04 conditional novelty 6.0 of 10

    A learnable cost predictor plus dynamic programming allocates layer-wise token sparsity for diffusion transformers, removing forced full steps and cutting ~54% compute on PixArt-α without quality loss.

  15. LESA: Learnable Stage-Aware Predictors for Diffusion Model Acceleration

    cs.CV 2026-02 conditional novelty 6.0 of 10

    A learned, stage-segmented KAN predictor for feature caching accelerates diffusion transformers by 5-6.25x while preserving more image/video fidelity than prior training-free forecasters.

  16. Phase-Aligned RoPE for Mixed-Resolution Diffusion Transformer

    cs.CV 2025-11 conditional novelty 6.0 of 10

    Expressing all RoPE positions on the query's grid ('one attention, one scale') plus a small boundary content-exchange step restores mixed-resolution diffusion generation that naive position interpolation destroys.

  17. Timeripple: Accelerating vDiTs by Understanding the Spatio-Temporal Correlations in Latent Space

    cs.AR 2025-11 conditional novelty 6.0 of 10

    Timeripple cuts vDiT self-attention compute by up to 85% by reusing partial attention scores of spatially and temporally correlated tokens across channels, with VBench quality essentially unchanged.

  18. Compute Only 16 Tokens in One Timestep: Accelerating Diffusion Transformers with Cluster-Driven Feature Caching

    cs.CV 2025-09 conditional novelty 6.0 of 10

    ClusCa accelerates diffusion transformers by clustering tokens and computing only one token per cluster, achieving 4.96x FLOPs speedup on FLUX with ImageReward preserved.

  19. OmniCache: A Trajectory-Oriented Global Perspective on Training-Free Cache Reuse for Diffusion Transformer Models

    cs.CV 2025-08 unverdicted novelty 6.0 of 10

    A training-free cache-reuse scheme that spreads computation across the full diffusion trajectory and subtracts estimated noise, accelerating DiT sampling with claimed competitive quality.

  20. LaVieID: Local Autoregressive Diffusion Transformers for Identity-Preserving Video Creation

    cs.CV 2025-08 conditional novelty 6.0 of 10

    LaVieID improves identity-preserving text-to-video by routing local facial parts into early DiT blocks and autoregressively refining denoised video tokens in temporal chunks.

  21. SADA: Stability-guided Adaptive Diffusion Acceleration

    cs.LG 2025-07 conditional novelty 6.0 of 10

    SADA accelerates ODE-based generative model sampling by adaptively combining step skipping and token pruning through a stability criterion, giving about 1.8 times speedup with minor fidelity loss.

  22. Less is Enough: Training-Free Video Diffusion Acceleration via Runtime-Adaptive Caching

    cs.CV 2025-07 conditional novelty 6.0 of 10

    EasyCache is a training-free, runtime-adaptive caching method that accelerates DiT video generation by reusing transformation vectors during stable denoising phases.

  23. DDiT: Dynamic Resource Allocation for Diffusion Transformer Model Serving

    cs.DC 2025-06 conditional novelty 6.0 of 10

    DDiT cuts text-to-video serving latency by up to 1.44x via DiT-VAE phase decoupling and step-level, starvation-aware GPU reassignment.

  24. Chipmunk: Training-Free Acceleration of Diffusion Transformers with Dynamic Column-Sparse Deltas

    cs.CV 2025-06 conditional novelty 6.0 of 10

    Chipmunk speeds up diffusion transformer inference by recomputing, at each step, only the top percent of attention and MLP activation columns that change most between steps, caching the rest in column-sparse GPU kernels.

  25. Attend to Not Attended: Structure-then-Detail Token Merging for Post-training DiT Acceleration

    cs.CV 2025-05 conditional novelty 6.0 of 10

    A stage-aware token merging scheme, SDTM, speeds up DiT text-to-image generation 1.55x without fine-tuning and with negligible FID change.

  26. Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity

    cs.CV 2025-02 conditional novelty 6.0 of 10

    Sparse VideoGen accelerates video diffusion transformers by about 2.3x with only small quality loss by classifying attention heads into spatial and temporal sparse patterns and using hardware-friendly layouts.

  27. Token Pruning for Caching Better: 9 Times Acceleration on Stable Diffusion for Free

    cs.CV 2024-12 conditional novelty 6.0 of 10

    DaTo combines dynamics-aware token pruning with feature caching and a searched per-step schedule, reaching 9x faster Stable Diffusion inference without FID loss.

  28. AsymRnR: Video Diffusion Transformers Acceleration with Asymmetric Reduction and Restoration

    cs.CV 2024-12 conditional novelty 6.0 of 10

    AsymRnR selectively reduces query and key/value tokens in video DiT attention to cut FLOPs and latency by 10 to 30 percent with minor or no VBench score change.

  29. Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model

    cs.CV 2024-11 conditional novelty 6.0 of 10

    TeaCache speeds up video diffusion models by caching model outputs when the difference between timestep-embedding-modulated inputs is small, after polynomial rescaling calibrated on a small prompt set.

  30. Importance-Based Token Merging for Efficient Image and Video Generation

    cs.CV 2024-11 conditional novelty 6.0 of 10

    A token-merging method that anchors computation on high-CFG-importance tokens improves generation quality at fixed inference speedups.

  31. SmoothCache: A Universal Inference Acceleration Technique for Diffusion Transformers

    cs.LG 2024-11 conditional novelty 6.0 of 10

    SmoothCache uses calibration-measured layer error thresholds to skip redundant attention and feed-forward computations in Diffusion Transformers, achieving 8-71% speedup across image, video, and audio tasks.

  32. OmniCache: Multidimensional Hierarchical Feature Caching For Diffusion Models

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Training-free hierarchical feature caching across token, frame, block, and layer axes cuts diffusion inference latency up to 35% while preserving quality better than averaging-based token merging.

  33. Vera: Identity-Faithful Human Subject-to-Video Generation

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Vera improves identity consistency in human subject-to-video generation using cross-clip identity-aligned data, face-weighted masked loss, and layer-aware reference attention.

  34. ACID: Adaptive Caching for vIDeo generation

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Adaptive threshold switching on the drift signal's rate of change expands the quality-vs-speed Pareto frontier of TeaCache, EasyCache, and DiCache across HunyuanVideo, Wan 2.1, and CogVideoX.

  35. PromptTea: Let Prompts Tell TeaCache the Optimal Threshold

    cs.CV 2025-07 conditional novelty 5.0 of 10

    PromptTea tunes diffusion-model cache reuse thresholds using prompt-derived complexity, achieving up to 2.79x speedup on Wan2.1 with PSNR 23.0 dB.

  36. FullDiT2: Efficient In-Context Conditioning for Video Diffusion Transformers

    cs.CV 2025-06 conditional novelty 5.0 of 10

    FullDiT2 accelerates FullDiT-style in-context conditioning for video by dynamic token selection and selective context caching, cutting per-step time by 2-3x with minimal quality loss.

  37. Sparse-vDiT: Unleashing the Power of Sparse Attention to Accelerate Video Diffusion Transformers

    cs.CV 2025-06 conditional novelty 5.0 of 10

    Sparse-vDiT replaces dense attention with fixed per-head sparse patterns chosen offline, achieving 1.58-1.85x end-to-end speedups on CogVideoX1.5, HunyuanVideo, and Wan2.1 with minimal quality loss.

  38. One-Way Ticket:Time-Independent Unified Encoder for Distilling Text-to-Image Diffusion Models

    cs.CV 2025-05 conditional novelty 5.0 of 10

    TiUE shares one encoder pass across parallel decoder time steps in a distilled text-to-image student, improving one-step generation diversity while removing the iterative denoising loop.

  39. RainFusion: Adaptive Video Generation Acceleration via Multi-Dimensional Visual Redundancy

    cs.CV 2025-05 conditional novelty 5.0 of 10

    Training-free sparse attention that classifies each head as spatial, temporal, or textural and applies a matched mask or token reduction, giving about 1.9x attention speedup with small VBench losses.

  40. Accelerating Diffusion Transformer via Error-Optimized Cache

    cs.CV 2025-01 conditional novelty 5.0 of 10

    A precomputed trend correction injected into cached diffusion transformer blocks reduces FID by up to 28.8% at heavy caching with negligible added compute.

  41. Unveiling Redundancy in Diffusion Transformers (DiTs): A Systematic Study

    cs.CV 2024-11 conditional novelty 5.0 of 10

    Redundancy between consecutive diffusion steps varies widely across DiT models but is stable within each model across prompts, step counts, and schedulers, so caching strategies must be model-specific.

  42. Predictive Feature Caching for Training-free Acceleration of Molecular Geometry Generation

    cs.LG 2025-10 conditional novelty 4.0 of 10

    Predictive feature caching, borrowed from image diffusion, speeds up molecular flow-matching generation by 2-3x at near-matched quality by forecasting hidden features instead of recomputing them.

  43. UniCP: A Unified Caching and Pruning Framework for Efficient Video Generation

    cs.CV 2025-02 reject novelty 4.0 of 10

    UniCP combines error-aware caching and PCA-based pruning to speed up diffusion-transformer video generation by about 1.6x.

  44. A Survey on Diffusion Language Models

    cs.CL 2025-08 unverdicted novelty 3.0 of 10

    A comprehensive survey of diffusion language models covering taxonomy, training and inference techniques, and comparisons with autoregressive models.

  45. Aquarius: A Family of Industry-Level Video Generation Models for Marketing Scenarios

    cs.CV 2025-05 reject novelty 3.0 of 10

    A technical report describing 2B and 13.4B video diffusion models, a distributed data processing pipeline, large-scale training infrastructure, inference acceleration, and marketing applications, without releasing mod...

Pith tools