Pith. sign in

REVIEW 16 cited by

DDT: Decoupled Diffusion Transformer

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2504.05741 v2 pith:5XPMCECX submitted 2025-04-08 cs.CV cs.AI

Shuai Wang , Zhi Tian , Weilin Huang , Limin Wang This is my paper · ORCID
classification cs.CVcs.AI
keywords colordiffusiontextbfdecoupledperformancesemantictransformersachieves
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

Diffusion transformers have demonstrated remarkable generation quality, albeit requiring longer training iterations and numerous inference steps. In each denoising step, diffusion transformers encode the noisy inputs to extract the lower-frequency semantic component and then decode the higher frequency with identical modules. This scheme creates an inherent optimization dilemma: encoding low-frequency semantics necessitates reducing high-frequency components, creating tension between semantic encoding and high-frequency decoding. To resolve this challenge, we propose a new \textbf{\color{ddt}D}ecoupled \textbf{\color{ddt}D}iffusion \textbf{\color{ddt}T}ransformer~(\textbf{\color{ddt}DDT}), with a decoupled design of a dedicated condition encoder for semantic extraction alongside a specialized velocity decoder. Our experiments reveal that a more substantial encoder yields performance improvements as model size increases. For ImageNet $256\times256$, Our DDT-XL/2 achieves a new state-of-the-art performance of {1.31 FID}~(nearly $4\times$ faster training convergence compared to previous diffusion transformers). For ImageNet $512\times512$, Our DDT-XL/2 achieves a new state-of-the-art FID of 1.28. Additionally, as a beneficial by-product, our decoupled architecture enhances inference speed by enabling the sharing self-condition between adjacent denoising steps. To minimize performance degradation, we propose a novel statistical dynamic programming approach to identify optimal sharing strategies.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 16 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SPAE: Spectrally Guided Autoencoder for Pretrained Visual Latents

    cs.CV 2026-08 conditional novelty 7.0 of 10

    SPAE compresses pretrained vision latents into a 64-channel bottleneck with channel-wise masking, improving image generation while keeping reconstruction and semantic understanding strong.

  2. ReGen: Hierarchical Multi-Prompt Representation Generation for Efficient Waveform Diffusion Models

    cs.SD 2026-07 conditional novelty 6.5 of 10

    Hierarchical multi-prompt representation generation plus generalized flow matching yields high-quality single-stage waveform diffusion from 12.5 Hz latents and efficient LDM TTS.

  3. EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Data

    cs.RO 2026-07 accept novelty 6.5 of 10

    World Action Model co-training with DINO or 3D-flow targets scales human-to-robot transfer on bimanual tasks far better than behavior cloning, while pixel prediction transfers weakly.

  4. Energy-Guided Flow Matching

    cs.CV 2026-08 conditional novelty 6.0 of 10

    Energy-Guided Flow Matching replaces the fixed clean endpoint with a sample-adaptive heat-kernel-filtered endpoint, yielding coarse-to-fine generation, better ImageNet FID, and faster convergence.

  5. WaiT for the Signal: Simple Frequency-Aware Flow-Matching

    cs.CV 2026-07 conditional novelty 6.0 of 10

    WaiT delays high-frequency wavelet bands in flow-matching image generation until coarse structure emerges, improving quality and cutting compute, with a reported SOTA FID of 1.30 on ImageNet 512.

  6. WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Driving

    cs.CV 2026-07 conditional novelty 6.0 of 10

    WCog-VLA couples Game-CoT semantic reasoning with an aligned decoupled diffusion transformer to generate joint multi-agent trajectories and reaches 92.9 PDMS on NAVSIM.

  7. Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Aura combines VLM meta-queries, T5-teacher alignment, subject-aware RoPE shifts, memory tokens, and a large AIGC-curated dataset to claim SOTA multi-element subject-to-video generation under OpenS2V-Eval Total score.

  8. SRC-Flow: Compact Semantic Representations Enable Normalizing Flows for Image Generation

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    SRC-Flow compresses RAE features via a Semantic Representation Compressor into a low-dimensional space, enabling normalizing flows to reach gFID 1.65 on ImageNet 256x256 and 2.07 on 512x512 while retaining exact likelihoods.

  9. RAE-NWM: Navigation World Model in Dense Visual Representation Space

    cs.CV 2026-03 conditional novelty 6.0 of 10

    Navigation world models trained in dense DINOv2 space with flow-matching CDiT-DH and time-gated action injection improve structural stability and planning over VAE baselines.

  10. Transition Matching Distillation for Fast Video Generation

    cs.CV 2026-01 conditional novelty 6.0 of 10

    Splitting a video diffusion model into a fixed feature extractor and a small recurrent flow head lets TMD generate videos in one to two effective steps with better VBench scores than prior distilled models.

  11. Self-transcendence: Is External Feature Guidance Indispensable for Accelerating Diffusion Transformer Training?

    cs.CV 2026-01 conditional novelty 6.0 of 10

    A self-supervised two-stage training method—VAE-latent feature alignment then feature-level classifier-free guidance—lets DiT models match or beat DINO-guided REPA training without any external feature extractor.

  12. MixFlow Training: Alleviating Exposure Bias with Slowed Interpolation Mixture

    cs.CV 2025-12 conditional novelty 6.0 of 10

    Training diffusion models on a mixture of higher-noise interpolations (MixFlow) improves generation FID across SiT, REPA, RAE and SD3.5, reaching ImageNet 256 gFID 1.43 after post-training.

  13. REGLUE Your Latents with Global and Local Semantics for Entangled Diffusion

    cs.CV 2025-12 conditional novelty 6.0 of 10

    Nonlinear multi-layer compression of frozen VFM patch semantics, jointly denoised with VAE latents, improves ImageNet 256x256 FID (12.9 vs 15.2 for REG at SiT-B/2, 400K) and accelerates convergence over REPA/ReDi/REG.

  14. PixNerd: Pixel Neural Field Diffusion

    cs.CV 2025-07 conditional novelty 6.0 of 10

    PixNerd is a single-stage pixel-space diffusion transformer that uses predicted neural field weights to decode large patches, reaching 2.15 FID on ImageNet 256 without a VAE.

  15. CMuon: Accelerating and Stabilizing Diffusion Transformer Training via Chunked Momentum Orthogonalization

    cs.AI 2026-08 conditional novelty 5.0 of 10

    Chunked Muon, which orthogonalizes fused DiT weight matrices per functional block instead of jointly, reaches FID 1.18 on ImageNet 256 in 200 epochs, about 2× faster than AdamW.

  16. STEREOFLOW: Progressive Stereo Matching with StereoDiT and Transition Flow Matching

    cs.CV 2026-07 conditional novelty 5.0 of 10

    A hybrid stereo-matching model uses a cascade matching network to propose disparities and a diffusion transformer to refine ambiguous regions; it claims state-of-the-art benchmark results.

Pith tools