Pith. sign in

REVIEW 12 cited by

Dynamic Diffusion Transformer

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.03456 v2 pith:3VJZVPEA submitted 2024-10-04 cs.CV

Wangbo Zhao , Yizeng Han , Jiasheng Tang , Kai Wang , Yibing Song , Gao Huang , Fan Wang , Yang You This is my paper · ORCID
classification cs.CV
keywords diffusiondynamicgenerationcomputationspatialtransformercostsdydit
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Diffusion Transformer (DiT), an emerging diffusion model for image generation, has demonstrated superior performance but suffers from substantial computational costs. Our investigations reveal that these costs stem from the static inference paradigm, which inevitably introduces redundant computation in certain diffusion timesteps and spatial regions. To address this inefficiency, we propose Dynamic Diffusion Transformer (DyDiT), an architecture that dynamically adjusts its computation along both timestep and spatial dimensions during generation. Specifically, we introduce a Timestep-wise Dynamic Width (TDW) approach that adapts model width conditioned on the generation timesteps. In addition, we design a Spatial-wise Dynamic Token (SDT) strategy to avoid redundant computation at unnecessary spatial locations. Extensive experiments on various datasets and different-sized models verify the superiority of DyDiT. Notably, with <3% additional fine-tuning iterations, our method reduces the FLOPs of DiT-XL by 51%, accelerates generation by 1.73, and achieves a competitive FID score of 2.07 on ImageNet. The code is publicly available at https://github.com/NUS-HPC-AI-Lab/ Dynamic-Diffusion-Transformer.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 12 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Dynamic-in-Few-Step: Unifying Dynamic Computation and Few-Step Distillation for Efficient Video Generation

    cs.CV 2026-07 conditional novelty 6.5 of 10

    Joint few-step distillation and step-specific structural pruning turns a video diffusion model into a compact Mixture-of-Models that cuts 24% extra FLOPs per step and reaches 30× speedup on Wan-14B.

  2. Importance-Aware OBS Pruning for Diffusion Models

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Injecting spatial importance maps (e.g., CFG delta) into the OBS Hessian improves subject preservation in pruned diffusion models at high sparsity, but gains over the baseline are small and without error bars.

  3. ELSA3D: Elastic Semantic Anchoring for Unified 3D Understanding and Generation

    cs.CV 2026-07 unverdicted novelty 6.0 of 10

    ELSA3D introduces elastic semantic anchoring via sparse anchor tokens and a scale-aware octree tokenizer to unify 3D generation and captioning at reduced computational cost.

  4. Self-transcendence: Is External Feature Guidance Indispensable for Accelerating Diffusion Transformer Training?

    cs.CV 2026-01 conditional novelty 6.0 of 10

    A self-supervised two-stage training method—VAE-latent feature alignment then feature-level classifier-free guidance—lets DiT models match or beat DINO-guided REPA training without any external feature extractor.

  5. LSSGen: Leveraging Latent Space Scaling in Flow and Diffusion for Efficient Text to Image Generation

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A latent-space scaling framework that replaces pixel-space upscaling with a trainable latent upsampler and noise compensation, yielding faster high-resolution text-to-image generation.

  6. Revisiting Diffusion Models: From Generative Pre-training to One-Step Generation

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Fine-tuning a pretrained diffusion model with a GAN objective and most weights frozen yields a one-step generator that matches or beats prior distillation methods on several datasets.

  7. Attend to Not Attended: Structure-then-Detail Token Merging for Post-training DiT Acceleration

    cs.CV 2025-05 conditional novelty 6.0 of 10

    A stage-aware token merging scheme, SDTM, speeds up DiT text-to-image generation 1.55x without fine-tuning and with negligible FID change.

  8. Layer- and Timestep-Adaptive Differentiable Token Compression Ratios for Efficient Diffusion Transformers

    cs.CV 2024-12 conditional novelty 6.0 of 10

    DiffCR learns per-layer and per-timestep token compression ratios for diffusion transformers, improving FID at similar latency relative to uniform token pruning.

  9. A Stitch in Time Saves Nine: Small VLM is a Precise Guidance for Accelerating Large VLMs

    cs.CV 2024-12 conditional novelty 6.0 of 10

    A small VLM's aggregated attention map can guide aggressive visual token pruning in a much larger VLM, preserving accuracy at 9% token retention and enabling early exit.

  10. Importance-Based Token Merging for Efficient Image and Video Generation

    cs.CV 2024-11 conditional novelty 6.0 of 10

    A token-merging method that anchors computation on high-CFG-importance tokens improves generation quality at fixed inference speedups.

  11. Fast3D: Accelerating 3D Multi-modal Large Language Models for Efficient 3D Scene Understanding

    cs.CV 2025-07 conditional novelty 5.0 of 10

    Fast3D prunes up to 90% of object-centric visual tokens in 3D MLLMs while preserving about 96.8% of original benchmark performance, using a trained attention predictor and adaptive layer-wise pruning.

  12. Change of Thought: Adaptive Test-Time Computation

    cs.LG 2025-07 reject novelty 4.0 of 10

    A transformer layer that iteratively refines its attention matrix to a fixed point is claimed to improve accuracy with no extra parameters, but the benchmark evidence is not reproducible.

Pith tools