Pith. sign in

REVIEW 4 major objections 5 minor 56 references

A diffusion transformer can accelerate its own training, without any external feature extractor, by first aligning with VAE latents for structure and then sharpening its deep features with classifier-free guidance.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 10:59 UTC pith:FUG4337L

load-bearing objection Solid self-contained DiT acceleration recipe that nearly matches REPA at half the epochs — worth a careful referee, but the abstract overstates 'surpass' and the cost table omits teacher training. the 4 major comments →

arxiv 2601.07773 v3 pith:FUG4337L submitted 2026-01-12 cs.CV

Self-transcendence: Is External Feature Guidance Indispensable for Accelerating Diffusion Transformer Training?

Lingchen Sun , Rongyuan Wu , Zhengqiang Zhang , Ruibin Li , Yujing Sun , Shuaizheng Liu , Lei Zhang This is my paper
classification cs.CV
keywords diffusion transformerstraining accelerationfeature guidanceclassifier-free guidanceVAE latent alignmentrepresentation learningself-supervised learningimage generation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that external semantic features like DINO are not indispensable for accelerating diffusion transformer training. Instead, a DiT can generate its own supervision: first, a short-trained model's intermediate features are aligned with the clean VAE latents of the latent-diffusion framework to build structured representations; second, classifier-free guidance is applied at the feature level to sharpen semantics. These enriched internal features, used as a fixed teacher, guide a fresh DiT from scratch to match or beat REPA's FID in fewer epochs—7.51 vs 7.90 at 80 epochs on SiT-XL/2, and 1.44 vs 1.42 at 400 vs 800 epochs with CFG. The claim matters because it removes the dependency on pretrained external encoders, making fast DiT training fully self-contained.

Core claim

On its own terms, the paper discovers that the two requirements for effective guidance—structural cleanliness to separate noise from signal and semantic discriminability—can both be met inside the DiT itself. Stage one aligns shallow features with VAE latents, which are cheap, native, and clean. Stage two freezes that model and feeds its deep-layer features through the classifier-free guidance formula f_g = f_u + ω(f_c − f_u), which extrapolates from conditional to unconditional features to expose class or text semantics. That enriched f_g, matched by an MLP to the shallow layers of a new model, transfers both structure and semantics. The paper shows that this internal teacher generalizes ac

What carries the argument

The central mechanism is the two-stage self-teaching pipeline: (1) VAE structure guidance—an L2 loss aligning an MLP-projected intermediate feature with the clean VAE latent z; (2) self-guided representation alignment—feature-level classifier-free guidance f_g = f_u + ω(f_c − f_u) applied to the deeper layer of the frozen, warmed-up teacher, used as a regression target for the guided layer of a new model, with early stopping after the first 10–20 epochs. The identity doing the semantic lifting is the CFG extrapolation in feature space: it amplifies the conditional deviation so the guidance target carries more class or text signal than the raw deep feature.

Load-bearing premise

The whole acceleration rests on the empirical assumption that applying classifier-free guidance to the deep features of a model trained only 40 epochs with VAE alignment yields a target f_g that genuinely encodes class or text semantics; if that extrapolated feature is semantically empty, the guidance loss just injects noise and the FID gains would vanish.

What would settle it

Train a linear classifier on the guiding features f_g from the warmed-up teacher (layer 2n/3, t=0.6) and measure class accuracy on ImageNet; near-chance accuracy would mean the guidance target is semantically empty and the claimed mechanism fails. As a control, repeat the warm-up stage with random class-condition labels; if training still accelerates, the benefit is not semantic and the CFG extrapolation is not doing the claimed work.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • REPA-style external encoders are not required; a two-stage self-guided recipe can match or beat them, so future DiT acceleration can drop the pretrained-teacher dependency.
  • The guidance signals are architecture-matched to the student (same backbone), which stabilizes knowledge transfer and enables early stopping of the guidance loss without the degradation seen in REPA.
  • The method scales: gains grow with model size (43% FID reduction on SiT-B/2 vs 57% on SiT-XL/2), transfers to 512×512 resolution, and to MMDiT text-to-image, where it beats REPA (FID 4.56 vs 4.90).
  • The concept generalizes across latent spaces (SD-VAE and VAVAE) and backbones (SiT and LightningDiT), suggesting the recipe is tied to latent diffusion structure, not a specific architecture.
  • The full pipeline is compute-competitive: 60.45h vs 58.87h for vanilla and 65.44h for REPA on SiT-B/2, with teacher training at 200K steps costing 6.39h—far cheaper than pretraining DINO.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The feature-level CFG extrapolation may be a general bootstrapping mechanism: any conditional generative model could sharpen its own intermediate representations by contrasting conditional and unconditional activations; this suggests testing the recipe on video or 3D generation, which the paper does not explore.
  • The teacher-training-length sweet spot (200K steps best; 300K worse) implies representation drift limits self-guidance; an adaptive schedule that stops when teacher feature separability plateaus would likely make the method less brittle across tasks.
  • Because the warm-up model already shows class-clustered features, the same pipeline could serve as a cheap self-supervised feature extractor, potentially substituting for DINO in other tasks—a claim the paper does not make.
  • The current validation relies entirely on FID-style metrics on ImageNet and MS-COCO; whether f_g encodes genuine semantics or only geometry remains untested, so a linear-probe comparison of f_g versus f_c would directly test the semantic mechanism.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Self-Transcendence, a two-stage training strategy for diffusion transformers that replaces the external DINO-v2 features used in REPA with features extracted from the model itself. Stage 1 trains a teacher model with an auxiliary L2 loss aligning an intermediate layer (through an MLP) to the VAE latent, providing 'structural' guidance. Stage 2 freezes the teacher and combines its conditional and unconditional intermediate features with classifier-free guidance in feature space, f_g = f_u + ω(f_c − f_u), to produce a semantic target that guides a new model's shallow layers. Results on ImageNet 256×256 and 512×512 with SiT and LightningDiT, and on a text-to-image task, report FID values close to or better than REPA at a fraction of the training epochs. The paper also includes ablations over ω, layer choices, teacher training length, λ_guide, early stopping, and a training-cost comparison.

Significance. If substantiated, the paper would be a significant contribution: it would show that the external semantic encoder in REPA is not indispensable, simplifying training pipelines and removing pretrained-teacher dependencies. Concrete strengths include evaluation on two DiT backbones, two VAE latent spaces, two resolutions, and a text-to-image task; a training-cost table; early-stop analysis; a one-stage versus two-stage comparison; and a public code link. The central limitation is that the mechanism behind the 'semantic' teacher features is only qualitatively supported, and the headline claim of surpassing REPA in generation quality is not supported by the paper's own Table 2. The current evidence supports the weaker, still valuable claim that self-contained guidance reaches approximately REPA-level quality with considerably fewer epochs.

major comments (4)
  1. [Abstract; Sec. 4.2, Table 2] The abstract claims the method 'can even surpass REPA ... in both generation quality and convergence speed' for class-to-image tasks. Table 2 does not support the quality part: for SiT-XL/2, Ours at 400 epochs gives FID 1.44 and sFID 4.85, while REPA at 800 epochs gives FID 1.42 and sFID 4.70. Only IS is higher (311.3 vs 305.7). The claim should be revised to 'comparable or better quality at substantially fewer epochs' unless additional metrics or runs change the picture.
  2. [Sec. 3.3, Eq. (3)] The semantic validity of f_g is the load-bearing assumption of the method, but it is asserted rather than measured. The evidence is limited to t-SNE (Fig. 2d), PCA, and final FID. t-SNE is not a quantitative measure of class separability, and FID improvements only show that matching this target helps training, not that the help comes from semantic content. Since f_c is class-conditioned by construction, the gap used in Eq. (3) may reflect any label-dependent feature statistic. Please add a quantitative probe of the teacher features—e.g., linear or k-NN class accuracy of f_g versus f_c and f_u on held-out classes, or a class-separability measure—and report whether the proposed mechanism is supported.
  3. [Sec. 4.1 and Table 4] The key hyperparameters (ω=30, layer depth 2n/3→n/2, teacher trained 200K iterations, λ_guide=0.5, early-stop schedule) are selected by FID on the same benchmark used for the headline results, with no held-out validation and no repeated seeds. Several adjacent configurations are within 0.1–0.2 FID of the chosen one (ω=45: 23.01; 300K teacher: 23.05 vs 22.91), and the headline Table 1 gap (7.51 vs 7.90 over REPA) is small enough to be within seed noise. Please report mean±std over at least three seeds for the main comparisons, or select hyperparameters on a validation split.
  4. [Tables 1 and 3] There is an internal inconsistency: Table 3 reports a SiT-B/2 80-epoch baseline FID of 38.60, while Table 1 reports 36.14 for the same model and epoch count. The captions state different evaluation sample sizes (10k vs 50k), but the direction is opposite to the usual FID bias: FID computed on fewer samples is typically lower, not higher. If these are different runs, the ablation effect sizes in Table 3 are not directly comparable to the main experiments. Please clarify the protocol or align the tables.
minor comments (5)
  1. [Sec. 3.3, Eq. (3)] The displayed equation appears as f_g = f_u + ω·f_c − f_u, which is algebraically equal to (1−ω)f_u + ω f_c only if parentheses are intended. Insert parentheses: f_g = f_u + ω·(f_c − f_u).
  2. [Sec. 3.2 and Table 5] The text says VAE structure guidance is applied 'without adding extra computational resources,' but Table 5 shows a per-card memory increase (12.34 GB vs 9.03 GB) and reduced speed (6.69 iters/s vs 9.28 iters/s) during the first 50K iterations. Rephrase to indicate that the overhead is temporary rather than nonexistent.
  3. [Sec. 3.2; Table 1] The claim that VAE structure guidance alone 'can already ... obtain even better performance than the existing self-contained methods' is only true for SiT-B/2. In Table 1, for SiT-XL/2 the VAE-guided model reaches 12.25 FID at 80 epochs, which is worse than LayerSync's 8.80 at 200 epochs. Please qualify the claim by model size.
  4. [Sec. 4.1 and Table 5] The early-stop schedule is described as '20 epochs for base models, 10 epochs for larger models,' while Table 5 says the loss is used only for the first 50K iterations of a 400-epoch SiT-B/2 run. Please clarify the epoch/iteration conversion used for each model.
  5. [References] References [55] and [56] are duplicates (same title and author list for SD-DiT). Also, the name of the VAVAE tokenizer is written inconsistently as both 'VAVAE' and 'VA-VAE'.

Circularity Check

0 steps flagged

No significant circularity: the central claim is benchmarked externally against REPA/DINO baselines, and the teacher-student scheme is a self-distillation design rather than a definitional identity.

full rationale

The paper's derivation chain is not circular in the sense of equating a conclusion with an input. Stage 1 (L_VAE-guide = ||MLP(f_n) - z||_2^2) aligns internal features to VAE latents, which are native components of the latent-diffusion pipeline and are not defined in terms of the claimed acceleration. Stage 2 defines a feature-space CFG target f_g = f_u + ω(f_c − f_u) and trains a fresh student with L_guide = ||MLP(f_m) − f_g||_2^2; this is a self-distillation/self-guidance mechanism, but its success is evaluated by external FID/sFID/IS/precision/recall on ImageNet against the REPA/DINO baseline. The headline REPA-surpassing results are therefore externally falsifiable rather than forced by construction. The only self-referential element is that the guiding model is produced by the same VAE-alignment method the paper proposes, and the hyperparameters (ω, layer choice, teacher length, early stopping) were selected by ablations on the same benchmark; these are correctness/overfitting risks, not circularity of the kind where a quantity is defined in terms of the target result. The self-citation to SRA ([16], which shares author Lei Zhang) is used only as related work and is independently corroborated by the paper's own comparisons, so it is not load-bearing. The paper's stated limitation that internal guidance is 'upper-bounded by the model's own capacity' and that additional hyperparameters may require tuning across tasks confirms that the contribution is empirical rather than definitional.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 0 invented entities

The paper introduces no new entities. The central empirical claims rest on a set of hyperparameters chosen by FID ablations and on domain assumptions about VAE latents, feature-space CFG, and layer roles; these assumptions are untested beyond the reported benchmarks.

free parameters (7)
  • guidance scale ω = 30.0 (SiT), 10.0 (LightningDiT)
    Feature-space CFG blending strength; ablated over 1.0–60.0 in Table 4, with 30 best for SiT-B/2.
  • guidance loss weight λguide = 0.5
    Ablated over 0.1–1.0 in Table 4; 0.5 chosen as best FID.
  • guided layer index = n/2 (e.g., layer 6 for SiT-B/2)
    Layer supervised in student; ablated layer pairs 8→4, 8→6, 8→8, 6→6, 10→6 in Table 4.
  • guiding layer index = 2n/3 (e.g., layer 8)
    Layer extracted from teacher; ablated in Table 4.
  • teacher training iterations = 200K (~40 epochs)
    Ablated 50K/100K/200K/300K in Table 4; 200K best.
  • early stop step for self-guided loss = 50K iterations for SiT-B/2; 20 epochs base / 10 epochs larger
    Chosen 'from our experimental study' (Sec. 4.1); no systematic ablation table for all model sizes.
  • VAE structure guidance phase length = 40 epochs
    Teacher is VAE-guided for 40 epochs; no dedicated ablation for this value beyond teacher iteration counts.
axioms (5)
  • domain assumption Shallow DiT layers are the convergence bottleneck and can be trained against intermediate supervision without destabilizing deep layers.
    Sec. 3.1 motivation and Sec. 4.1 early-stop strategy; if false, the guidance losses would not accelerate training.
  • domain assumption The VAE latent z is a 'clean structural' target suitable as an L2 alignment objective for intermediate features at all noise levels.
    Used in Eq. 1; supported only by ablations, not by an information-theoretic argument.
  • ad hoc to paper Feature-space CFG f_g = f_u + ω(f_c − f_u) makes deep features more semantically discriminative and remains a valid supervision target for shallow layers.
    Eq. 3; essential to Stage 2, validated only by FID/IS ablations in Table 4.
  • domain assumption L2 alignment through a lightweight MLP transfers structure/semantics from teacher to student features.
    Eqs. 1 and 4; standard distillation assumption, not proven here.
  • domain assumption Comparing against REPA/SRA/LayerSync under each baseline's own training and inference settings is fair even though the guiding model and early stopping differ.
    Sec. 4.1; if the comparison is unfair, the headline 'surpass REPA' is unsupported.

pith-pipeline@v1.3.0-alltime-deepseek · 16956 in / 15437 out tokens · 141355 ms · 2026-08-03T10:59:10.382196+00:00 · methodology

0 comments
read the original abstract

Recent works such as REPA have shown that guiding diffusion models with external semantic features (e.g., DINO) can significantly accelerate the training of diffusion transformers (DiTs). However, the use of pretrained external features as guidance signals introduces additional dependencies. We argue that DiTs actually have the power to guide the training of themselves, and propose SelfTranscendence, an effective method that achieves fast convergence using internal feature supervision only. The desired internal guidance features should meet two requirements: structurally clean to help shallow blocks separate noise from signal, and semantically discriminative to help shallow layers learn effective representations. With this consideration, we first align the DiT features with the clean VAE latent features, a native component of latent diffusion, for a short training phase (e.g., 40 epochs) to improve their structural representations, then apply the classifier-free guidance to the intermediate features, enhancing their discriminative capability and semantic expressiveness. These enriched internal features, learned entirely within the model, are used as supervision signals to guide a new DiT training from scratch. Compared to existing self-contained methods, our approach achieves a significant performance boost. It can even surpass REPA, which uses the external DINO features as guidance, in both generation quality and convergence speed for both class-to-image and text-to-image generation tasks. The source code of our method can be found at https://github.com/csslc/Self-Transcendence.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

56 extracted references · 16 linked inside Pith

  1. [1]

    Williams

    Hervé Abdi and Lynne J. Williams. Principal component analysis.Wiley Interdisciplinary Reviews: Computational Statistics, 2(4):433–459, 2010. doi: 10.1002/wics.101

    doi:10.1002/wics.101 2010

  2. [2]

    Dense2moe: Unifying pruning and upcycling for efficient large language models

    Anonymous. Dense2moe: Unifying pruning and upcycling for efficient large language models. InSubmitted to The Fourteenth International Conference on Learning Representations, 2025. URLhttps://openreview.net/forum? id=hYGPetyGSr. under review

    2025

  3. [3]

    All are worth words: a vit backbone for score-based diffusion models

    Fan Bao, Chongxuan Li, Yue Cao, and Jun Zhu. All are worth words: a vit backbone for score-based diffusion models. InNeurIPS 2022 Workshop on Score-Based Methods, 2022. URLhttps://openreview.net/forum?id= WfkBiPO5dsG

    2022

  4. [4]

    Masked autoencoders are effective tokenizers for diffusion models

    Hao Chen, Yujin Han, Fangyi Chen, Xiang Li, Yidong Wang, Jindong Wang, Ze Wang, Zicheng Liu, Difan Zou, and Bhiksha Raj. Masked autoencoders are effective tokenizers for diffusion models. InInternational Conference on Learning Representations, year=2025,

    2025

  5. [5]

    On the efficacy of knowledge distillation

    Jang Hyun Cho and Bharath Hariharan. On the efficacy of knowledge distillation. In2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 4793–4801, 2019. doi: 10.1109/ICCV.2019.00489

    arXiv 2019

  6. [6]

    Text-to-image diffusion models are zero shot classifiers

    Kevin Clark and Priyank Jaini. Text-to-image diffusion models are zero shot classifiers. InThirty-seventh Confer- ence on Neural Information Processing Systems, 2023. URLhttps://openreview.net/forum?id=fxNQJVMwK2

    2023

  7. [7]

    InstructBLIP: Towards general-purpose vision-language models with instruction tuning

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. InstructBLIP: Towards general-purpose vision-language models with instruction tuning. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URLhttps://openreview.net/ forum?id=vvoWPYqZJA

    2023

  8. [8]

    Emerging properties in unified multimodal pretraining, 2025

    Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, Guang Shi, and Haoqi Fan. Emerging properties in unified multimodal pretraining, 2025. URL https://arxiv.org/abs/2505.14683

    Pith/arXiv arXiv 2025

  9. [9]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. InInternational Conference on Learning Representations, 2021. URLhttps:/...

    2021

  10. [10]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis. InForty-first International Conference on Machine Learning, 2024. URLhttps:/...

    2024

  11. [11]

    Masked diffusion transformer is a strong image synthesizer, 2023

    Shanghua Gao, Pan Zhou, Ming-Ming Cheng, and Shuicheng Yan. Masked diffusion transformer is a strong image synthesizer, 2023

    2023

  12. [12]

    Mdtv2: Masked diffusion transformer is a strong image synthesizer, 2024

    Shanghua Gao, Pan Zhou, Ming-Ming Cheng, and Shuicheng Yan. Mdtv2: Masked diffusion transformer is a strong image synthesizer, 2024. URLhttps://arxiv.org/abs/2303.14389

    Pith/arXiv arXiv 2024

  13. [13]

    Layersync: Self-aligning intermediate layers, 2025

    Yasaman Haghighi, Bastien van Delft, Mariam Hassan, and Alexandre Alahi. Layersync: Self-aligning intermediate layers, 2025. URLhttps://arxiv.org/abs/2510.12581

    arXiv 2025

  14. [14]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017

    2017

  15. [15]

    Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    Pith/arXiv arXiv 2022

  16. [16]

    No other representation component is needed: Diffusion transformers can provide representation guidance by themselves.arXiv preprint arXiv:2505.02831, 2025

    Dengyang Jiang, Mengmeng Wang, Liuzhuozheng Li, Lei Zhang, Haoyu Wang, Wei Wei, Guang Dai, Yanning Zhang, and Jingdong Wang. No other representation component is needed: Diffusion transformers can provide representation guidance by themselves.arXiv preprint arXiv:2505.02831, 2025

    arXiv 2025

  17. [17]

    Tread: Token routing for efficient architecture-agnostic diffusion training.arXiv preprint arXiv:2501.04765, 2025

    Felix Krause, Timy Phan, Ming Gui, Stefan Andreas Baumann, Vincent Tao Hu, and Björn Ommer. Tread: Token routing for efficient architecture-agnostic diffusion training.arXiv preprint arXiv:2501.04765, 2025. 17

    arXiv 2025

  18. [18]

    Improved precision and recall metric for assessing generative models.Advances in neural information processing systems, 32, 2019

    Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models.Advances in neural information processing systems, 32, 2019

    2019

  19. [19]

    Black Forest Labs. Flux. https://github.com/black-forest-labs/flux, 2024. Official inference repo for FLUX.1 models

    2024

  20. [20]

    Exploiting diffusion prior for generalizable dense prediction

    Hsin-Ying Lee, Hung-Yu Tseng, Hsin-Ying Lee, and Ming-Hsuan Yang. Exploiting diffusion prior for generalizable dense prediction. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  21. [21]

    Repa-e: Unlocking vae for end-to-end tuning with latent diffusion transformers

    Xingjian Leng, Jaskirat Singh, Yunzhong Hou, Zhenchang Xing, Saining Xie, and Liang Zheng. Repa-e: Unlocking vae for end-to-end tuning with latent diffusion transformers. 2025

    2025

  22. [22]

    Your diffusion model is secretly a zero-shot classifier

    Alexander Cong Li, Mihir Prabhudesai, Shivam Duggal, Ellis Langham Brown, and Deepak Pathak. Your diffusion model is secretly a zero-shot classifier. InICML 2023 Workshop on Structured Probabilistic Inference & Generative Modeling, 2023. URLhttps://openreview.net/forum?id=Ck3yXRdQXD

    2023

  23. [23]

    LLaVA-onevision: Easy visual task transfer.Transactions on Machine Learning Research, 2025

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. LLaVA-onevision: Easy visual task transfer.Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URLhttps://openreview.net/forum?id=zKv8qULV6n

    2025

  24. [24]

    Return of unconditional generation: A self-supervised representation generation method.Advances in Neural Information Processing Systems, 37:125441–125468, 2024

    Tianhong Li, Dina Katabi, and Kaiming He. Return of unconditional generation: A self-supervised representation generation method.Advances in Neural Information Processing Systems, 37:125441–125468, 2024

    2024

  25. [25]

    Step-video-t2v technical report: The practice, challenges, and future of video foundation model, 2025

    Guoqing Ma, Haoyang Huang, Kun Yan, Liangyu Chen, Nan Duan, Shengming Yin, Changyi Wan, Ranchen Ming, Xiaoniu Song, Xing Chen, Yu Zhou, Deshan Sun, Deyu Zhou, Jian Zhou, Kaijun Tan, Kang An, Mei Chen, Wei Ji, Qiling Wu, Wen Sun, Xin Han, Yanan Wei, Zheng Ge, Aojie Li, Bin Wang, Bizhu Huang, Bo Wang, Brian Li, Changxing Miao, Chen Xu, Chenfei Wu, Chenguang...

    Pith/arXiv arXiv 2025

  26. [26]

    Albergo, Nicholas M

    Nanye Ma, Mark Goldstein, Michael S. Albergo, Nicholas M. Boffi, Eric Vanden-Eijnden, and Saining Xie. Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers, 2024. URL https://arxiv.org/abs/2401.08740

    Pith/arXiv arXiv 2024

  27. [27]

    Not all diffusion model activations have been evaluated as discriminative features.Advances in Neural Information Processing Systems, 37:55141–55177, 2024

    Benyuan Meng, Qianqian Xu, Zitai Wang, Xiaochun Cao, and Qingming Huang. Not all diffusion model activations have been evaluated as discriminative features.Advances in Neural Information Processing Systems, 37:55141–55177, 2024

    2024

  28. [28]

    Improved knowledge distillation via teacher assistant

    Seyed Iman Mirzadeh, Mehrdad Farajtabar, Ang Li, Nir Levine, Akihiro Matsukawa, and Hassan Ghasemzadeh. Improved knowledge distillation via teacher assistant. InProceedings of the AAAI conference on artificial intelligence, volume 34, pages 5191–5198, 2020

    2020

  29. [29]

    Generating images with sparse representa- tions.arXiv preprint arXiv:2103.03841, 2021

    Charlie Nash, Jacob Menick, Sander Dieleman, and Peter W Battaglia. Generating images with sparse representa- tions.arXiv preprint arXiv:2103.03841, 2021

    Pith/arXiv arXiv 2021

  30. [30]

    Video generation models as world simulators

    OpenAI. Video generation models as world simulators. https://openai.com/index/ video-generation-models-as-world-simulators/, 2024

    2024

  31. [31]

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Herve Jegou, Julien Mairal, Patrick Lab...

    2024

  32. [32]

    Scalable diffusion models with transformers, 2023

    William Peebles and Saining Xie. Scalable diffusion models with transformers, 2023. URLhttps://arxiv.org/ abs/2212.09748

    Pith/arXiv arXiv 2023

  33. [33]

    Qwen2.5 technical report, 2025

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li,...

    Pith/arXiv arXiv 2025

  34. [34]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684–10695, June 2022

    2022

  35. [35]

    Improved techniques for training gans.Advances in neural information processing systems, 29, 2016

    Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans.Advances in neural information processing systems, 29, 2016

    2016

  36. [36]

    Glu variants improve transformer, 2020

    Noam Shazeer. Glu variants improve transformer, 2020. URLhttps://arxiv.org/abs/2002.05202

    Pith/arXiv arXiv 2020

  37. [37]

    What matters for representation alignment: Global information or spatial structure?arXiv preprint arXiv:2512.10794, 2025

    Jaskirat Singh, Xingjian Leng, Zongze Wu, Liang Zheng, Richard Zhang, Eli Shechtman, and Saining Xie. What matters for representation alignment: Global information or spatial structure?arXiv preprint arXiv:2512.10794, 2025

    arXiv 2025

  38. [38]

    Roformer: Enhanced transformer with rotary position embedding, 2021

    Jianlin Su, Yu Lu, Shengfeng Pan, Bo Wen, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding, 2021

    2021

  39. [39]

    U-dits: Downsample tokens in u-shaped diffusion transformers

    Yuchuan Tian, Zhijun Tu, Hanting Chen, Jie Hu, Chao Xu, and Yunhe Wang. U-dits: Downsample tokens in u-shaped diffusion transformers. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URLhttps://openreview.net/forum?id=SRWs2wxNs7

    2024

  40. [40]

    U-repa: Aligning diffusion u-nets to vits, 2025

    Yuchuan Tian, Hanting Chen, Mengyu Zheng, Yuchen Liang, Chao Xu, and Yunhe Wang. U-repa: Aligning diffusion u-nets to vits, 2025. URLhttps://arxiv.org/abs/2503.18414

    arXiv 2025

  41. [41]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors,Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URLhttp...

    2017

  42. [42]

    Wan: Open and advanced large-scale video generative models, 2025

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng, Shiwei Zhang, Siyang Sun, Tao Fang, T...

    Pith/arXiv arXiv 2025

  43. [43]

    Learning diffusion models with flexible representation guidance.arXiv preprint arXiv:2507.08980, 2025

    Chenyu Wang, Cai Zhou, Sharut Gupta, Zongyu Lin, Stefanie Jegelka, Stephen Bates, and Tommi Jaakkola. Learning diffusion models with flexible representation guidance.arXiv preprint arXiv:2507.08980, 2025

    arXiv 2025

  44. [44]

    Diffuse and disperse: Image generation with representation regularization, 2025

    Runqian Wang and Kaiming He. Diffuse and disperse: Image generation with representation regularization, 2025. URLhttps://arxiv.org/abs/2506.09027

    Pith/arXiv arXiv 2025

  45. [45]

    Ddt: Decoupled diffusion transformer.arXiv preprint arXiv:2504.05741, 2025

    Shuai Wang, Zhi Tian, Weilin Huang, and Limin Wang. Ddt: Decoupled diffusion transformer.arXiv preprint arXiv:2504.05741, 2025

    Pith/arXiv arXiv 2025

  46. [46]

    Repaworksuntilitdoesn’t: Early-stopped, holisticalignmentsupercharges 19 diffusion training.arXiv preprint arXiv:2505.16792, 2025

    Ziqiao Wang, Wangbo Zhao, Yuhao Zhou, Zekai Li, Zhiyuan Liang, Mingjia Shi, Xuanlei Zhao, Pengfei Zhou, KaipengZhang, ZhangyangWang, etal. Repaworksuntilitdoesn’t: Early-stopped, holisticalignmentsupercharges 19 diffusion training.arXiv preprint arXiv:2505.16792, 2025

    Pith/arXiv arXiv 2025

  47. [47]

    Representation entanglement for generation: Training diffusion transformers is much easier than you think, 2025

    Ge Wu, Shen Zhang, Ruijing Shi, Shanghua Gao, Zhenyuan Chen, Lei Wang, Zhaowei Chen, Hongcheng Gao, Yao Tang, Jian Yang, Ming-Ming Cheng, and Xiang Li. Representation entanglement for generation: Training diffusion transformers is much easier than you think, 2025. URLhttps://arxiv.org/abs/2507.01467

    arXiv 2025

  48. [48]

    Sana: Efficient high-resolution image synthesis with linear diffusion transformer,

    Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, and Song Han. Sana: Efficient high-resolution image synthesis with linear diffusion transformer,

  49. [49]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, Da Yin, Yuxuan.Zhang, Weihan Wang, Yean Cheng, Bin Xu, Xiaotao Gu, Yuxiao Dong, and Jie Tang. Cogvideox: Text-to-video diffusion models with an expert transformer. InThe Thirteenth International Conference on Learning Represen...

    2025

  50. [50]

    Reconstruction vs

    Jingfeng Yao, Bin Yang, and Xinggang Wang. Reconstruction vs. generation: Taming optimization dilemma in latent diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025

    2025

  51. [51]

    Representation alignment for generation: Training diffusion transformers is easier than you think

    Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. InInternational Conference on Learning Representations, 2025

    2025

  52. [52]

    Root Mean Square Layer Normalization

    Biao Zhang and Rico Sennrich. Root Mean Square Layer Normalization. InAdvances in Neural Information Pro- cessing Systems 32, Vancouver, Canada, 2019. URLhttps://openreview.net/references/pdf?id=S1qBAf6rr

    2019

  53. [53]

    Dynamic diffusion transformer.arXiv preprint arXiv:2410.03456, 2024

    Wangbo Zhao, Yizeng Han, Jiasheng Tang, Kai Wang, Yibing Song, Gao Huang, Fan Wang, and Yang You. Dynamic diffusion transformer.arXiv preprint arXiv:2410.03456, 2024

    Pith/arXiv arXiv 2024

  54. [54]

    Fast training of diffusion models with masked transformers.Transactions on Machine Learning Research, 2024

    Hongkai Zheng, Weili Nie, Arash Vahdat, and Anima Anandkumar. Fast training of diffusion models with masked transformers.Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https: //openreview.net/forum?id=vTBjBtGioE

    2024

  55. [56]

    Sd-dit: Unleashing the power of self-supervised discrimination in diffusion transformer, 2024

    Rui Zhu, Yingwei Pan, Yehao Li, Ting Yao, Zhenglong Sun, Tao Mei, and Chang Wen Chen. Sd-dit: Unleashing the power of self-supervised discrimination in diffusion transformer, 2024. URLhttps://arxiv.org/abs/2403.17004. 20

    Pith/arXiv arXiv 2024

  56. [2024]

    URLhttps://arxiv.org/abs/2410.10629

    Pith/arXiv arXiv