Pith. sign in

REVIEW 4 major objections 4 minor 2 cited by

DMVAE explicitly matches an autoencoder's latent distribution to any target prior, turning tokenizer design into a choice of reference distribution.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

DMVAE matches a tokenizer's aggregate latent distribution to an arbitrary reference via diffusion score matching; SSL-derived (DINO) targets yield gFID 3.22 on ImageNet-256 with fast generator convergence.

T0 review reviewed 2026-08-03 challenge →

load-bearing objection A genuinely new way to shape VAE latents, but the paper's headline comparison is rigged by its own initialization; the method deserves a serious look, the headline result doesn't. the 4 major comments →

arxiv 2512.07778 v2 pith:C26QXQ2O submitted 2025-12-08 cs.CV

Distribution Matching Variational AutoEncoder

Sen Ye , Jianning Pei , Mengde Xu , Shuyang Gu , Chunyu Wang , Liwei Wang , Han Hu This is my paper
classification cs.CV
keywords distribution matchingvariational autoencoderlatent priorself-supervised learningdiffusion modelsscore matchingimage generationlatent diffusion
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces DMVAE, a variational autoencoder that, instead of softly forcing latents toward a Gaussian, explicitly matches the entire distribution of encoded images to an arbitrary target distribution using scores from diffusion models. This turns autoencoder design into a choice of reference distribution, and the authors use it to compare Gaussian, text, supervised, self-supervised, and diffusion-noise priors on ImageNet. They find that self-supervised (DINO) features give the best trade-off between reconstruction fidelity and ease of generative modeling, reaching gFID 3.22 after only 64 training epochs and 1.64 at 800 epochs. If right, the result says the structure of the latent distribution matters more than the tokenizer architecture, and SSL-shaped latents are the sweet spot.

Core claim

DMVAE imposes a distributional constraint on the encoder's aggregate posterior q(z) by requiring its time-dependent score function to match that of a pre-trained reference distribution p_r(z). A 'fake' diffusion model tracks the score of the evolving q(z), and the encoder is updated with the difference of the scores, s_fake - s_real, which pulls latents toward the reference's high-density regions while preserving coverage. The authors report that this explicit distribution-level alignment generalizes VAE priors beyond Gaussians and, in a head-to-head comparison of six reference distributions, SSL-derived features yield the best balance: semantically clustered enough to be trivial to model, i

What carries the argument

The mechanism is Distribution Matching Distillation (DMD) repurposed for autoencoders: a 'real' score model is trained once on the chosen reference distribution, and a 'fake' score model is trained jointly to track the encoder's aggregate posterior. The encoder's gradient is the difference between the two score functions evaluated on noisy latents, which acts as a difference vector field that pushes q(z) toward p_r(z) without mode collapse. A learnable projection head adapts the latent dimension to the reference, and the encoder is initialized from a pretrained tokenizer to stabilize distant matching.

Load-bearing premise

The systematic comparison assumes that all reference distributions are matched with comparable fidelity, so that differences in gFID reflect the intrinsic merits of the target distribution rather than how hard it was to match it in the first place.

What would settle it

Train DMVAE with each reference distribution starting from a randomly initialized encoder (or from the same pretrained encoder for all targets), and measure the actual KL/JS divergence between the learned aggregate q(z) and each p_r(z). If DINO's gFID advantage disappears or shrinks once matching difficulty is equalized, the paper's claim that SSL distributions are intrinsically better priors would be falsified. Alternatively, show that a well-matched Gaussian prior (matched to the same fidelity as DINO) achieves comparable gFID would also undercut the claim.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Tokenizer design reduces to prior selection: to get a generative-friendly latent space, pick a reference distribution and match it, rather than hand-crafting per-sample losses.
  • SSL-derived latents, such as DINO features, enable fast diffusion training: gFID 3.22 in 64 epochs, 1.82 in 400 epochs, outperforming previous tokenizers with far fewer steps.
  • Distributional alignment (global score matching) is qualitatively better than per-sample alignment (e.g., VAVAE and AlignTok) because it prevents the aggregate posterior from degenerating into disconnected mixtures.
  • The framework generalizes beyond images: the same distribution-matching constraint applies to audio, video, and 3D latents.
  • The choice of reference distribution has a larger effect on generation quality than the tokenizer architecture, since even a standard DiT reaches state-of-the-art gFID with the right prior.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The headline comparison may be confounded: since the DMVAE encoder is initialized from a frozen DINO encoder and the DINO reference is extracted from that same encoder, q(z) starts nearly equal to p_r(z) for the SSL target; other targets start farther away, so part of DINO's advantage could be a head start rather than intrinsic superiority of the distribution. A fair test would initialize all runs
  • The method's reliance on low-dimensional latents (e.g., 32-dim) and a projection head suggests that distribution matching in high-dimensional spaces is still hard; the benefit may shrink as latent dimensionality grows unless the score models are scaled up.
  • The score-difference objective could be reused for other distributional alignment tasks, e.g., aligning embeddings across modalities or distilling a target distribution into a lightweight encoder in a single pass.
  • The 'regularizer rather than complete matching' admission implies that the final latent distribution is a blend of the reconstruction manifold and the reference; future work might measure how far q(z) actually is from p_r(z) per reference to separate matching difficulty from prior quality.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes Distribution Matching VAE (DMVAE), which uses a distribution-matching loss based on diffusion score models (following DMD) to align the aggregate posterior q(z) of a VAE-style tokenizer with an arbitrary reference distribution p_r(z). The authors evaluate several reference distributions (DINOv2 features, ResNet features, SigLIP text features, diffusion noise states, sub-sampled DINO, Gaussian, GMM) and report in Table 1 that DINO-derived SSL features give the best balance of reconstruction and generation quality. They further report a gFID of 3.22 on ImageNet-256 with a 64-epoch Lightning-DiT training, and claim that SSL-derived latent distributions are intrinsically better for generative modeling. The method is presented as the first systematic study of latent priors via distribution-level alignment. The appendix describes a multi-stage pipeline: tokenizer pretraining with a frozen DINOv2 encoder (A.1), teacher/reference score model pretraining on latents extracted from that same encoder (A.2), joint DMVAE training (A.3), decoder fine-tuning (A.4), and final diffusion model training (A.5).

Significance. If the central comparison were properly controlled, the idea of using score-based distribution matching to shape the latent distribution of a tokenizer is a timely and potentially valuable contribution. The paper is clearly written, provides detailed implementation details, and releases code. However, the main empirical claim—that SSL-derived distributions are superior latent priors—is not supported by the current experiment because the comparison is confounded. The DINO reference distribution is extracted from the same frozen encoder that initializes the DMVAE tokenizer, so the DINO condition starts at zero matching distance, while other references start far away; Section 7 admits that matching distant distributions remains a challenge. The use of different matching weights across reference families further prevents a controlled comparison. The efficiency claim is also misleading because the '64 training epochs' counts only the final generative model, not the multi-stage tokenizer pipeline. Thus, the paper's central contribution—a systematic study of latent priors—is currently unsubstantiated, although the method itself is plausible and could be vindicated by a revised experiment.

major comments (4)
  1. [App. A.1–A.2 / Sec. 4] The DINO reference distribution p_r(z) is defined by extracting latents from a frozen DINOv2 encoder + MLP projector (App. A.2), and the DMVAE tokenizer is initialized from that exact same encoder+projector (App. A.1, A.3). At the start of joint training, q(z) is essentially equal to p_r(z) for the DINO condition, whereas all other references start far away. Since Sec. 7 admits "matching distributions that are initially far apart remains a challenge" and that the reference acts "more like a regularizer than a complete matching", the ranking in Table 1 (DINO best) can be fully explained by initialization and matching difficulty, not by intrinsic properties of the latent distribution. The claim of a "systematic study" (Sec. 4) is unsupported without a common initialization for all references and a report of achieved matching distance for each condition.
  2. [Sec. 4] The reference-family comparison uses different distribution-matching weights: λ_DM=10 for data-derived priors and λ_DM=1 for synthetic priors. This means the degree of alignment is not held constant across conditions. The superior gFID for DINO over Gaussian/GMM could be due to the larger matching weight rather than the nature of the prior. A fair comparison requires either the same λ_DM for all references or a full ablation of λ_DM for each reference family.
  3. [Abstract / Sec. 5.2 / App. A] The claim "gFID equals 3.2 on ImageNet with only 64 training epochs" is misleading. The 64 epochs in Table 3 refer only to the final Lightning-DiT training. The full pipeline includes tokenizer pretraining (A.1: 8 epochs), teacher/reference score model pretraining (A.2: 400 epochs), DMVAE joint training (A.3: ~70 epochs at batch 256), and decoder fine-tuning (A.4: ~50 epochs), before the final generative model is trained. If the teacher training is counted, the total is even larger. The authors should clearly state whether 64 epochs is only the generative stage and report end-to-end training cost, which is central to the paper's efficiency contribution.
  4. [Sec. 5.1 / App. A.3] The main text states the default configuration does not adopt CFG (weight 1.0) for the real/fake score models and evaluation, but Appendix A.3 says "we apply a CFG scale of w=5 during the computation of the DM loss." Table 2 shows that changing CFG from 1.0 to 5.0 changes gFID-5k from 13.1 to 11.5, so the discrepancy is material. The authors must reconcile these statements and specify which configuration was used for Table 1 and Table 3. As written, the reported results are not reproducible from the described setup.
minor comments (4)
  1. [Throughout] Typos and grammar issues: "avaliable" in the abstract, "Guassian" in Sec. 2, "adopts several stablizing" in Sec. 3.2. Please proofread.
  2. [Sec. 4 / App. A.3] Sec. 4 states all reference models are trained for 300k iterations with batch size 256, while App. A.3 states the DMVAE joint training runs for 350K steps. These numbers should be reconciled.
  3. [Fig. 4] The t-SNE visualization is qualitative. To support the claim that the DINO condition preserves semantic clustering better than others, the authors should provide quantitative distribution-alignment metrics (e.g., MMD, FID between q(z) and p_r(z)) for each reference.
  4. [Table 2] The default row is labeled "CFG=1.0" but App. A.3 describes CFG=5; this inconsistency should be fixed in the table caption or main text.

Circularity Check

1 steps flagged

The favorable DINO-prior result is confounded by construction: the DINO reference distribution is the aggregate posterior of the same frozen encoder that initializes the tokenizer, so the 'systematic comparison' re-imports the initialization choice rather than testing intrinsic distribution quality.

specific steps
  1. fitted input called prediction [Appendix A.1–A.3; Section 7; Section 4/Table 1]
    "Appendix A.1: 'We employ the pre-trained DINO-v2-large model... we freeze the parameters of the DINO-v2 encoder.' A.2: 'We first extract the latent representations for the entire ImageNet dataset using the frozen encoder and projector from the previous stage trained tokenizer.' A.3: 'We initialize the AE from the tokenizer pretraining stage checkpoint...' Section 7: 'matching distributions that are initially far apart remains a challenge... more like a regularizer than a complete matching.'"

    The DINO reference p_r(z) is extracted from the same frozen DINO encoder + MLP projector that defines the initial AE (A.1/A.2), and A.3 initializes DMVAE from that AE. So for DINO, q(z) starts essentially equal to p_r(z) and the DM loss is already nearly zero. Other references start far away, and Section 7 admits distant matching is unsolved and acts only as a regularizer. Thus Table 1's ranking that SSL features are best is not a controlled test of intrinsic distribution quality; it largely restates the initialization. The comparison additionally uses different DM weights (λ=10 vs λ=1), so alignment strength is not held constant.

full rationale

The central empirical claim—that SSL/DINO-derived latent distributions are the best prior and enable fast training (Abstract; §4; Tab. 1; §5.2)—is not circular in the mathematical sense, but it is not an independent finding either. The paper's own appendix shows the DINO reference distribution is the aggregate posterior of the frozen DINO-v2 encoder used to initialize the tokenizer, making the DINO condition start at zero matching distance. Section 7 explicitly concedes that far-apart references are only partially matched, so the other conditions in Table 1 are not aligned to the same degree. Thus the ranking conflates intrinsic distribution quality with matching difficulty and initialization. I do not find load-bearing self-citation: the DMD/score-matching technique is cited from external work ([38], [37]) and the reconstruction pipeline is standard. The 64-epoch efficiency headline also omits the A.1–A.4 pretraining cost, which is a reporting concern but not circularity. Because the favorable prior result is substantially manufactured by the initialization/reference construction, my circularity score is 7; the method itself (distribution matching as a tokenizer regularizer) still has independent content and could be validated by a controlled comparison with matched initialization and equal λ_DM.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The central empirical claim rests on several free choices: the per-category DM weight, the CFG scale used during DM loss, the low latent dimension, and the alternating update schedule. No new physical entities are introduced. Two of the axioms are standard but the last two are paper-specific and directly affect the validity of the 'SSL is best' conclusion.

free parameters (4)
  • DM weight λ_DM = 10 for data-derived priors, 1 for synthetic priors
    Hand-selected per reference-distribution category; the larger value for the favored data-derived priors confounds the comparison (Sec. 4).
  • CFG weight w during DM loss = 5
    Applied to the score models when computing the DM loss (Appendix A.3), while evaluation uses CFG=1; this changes the effective target distribution and is not ablated.
  • Latent dimension d = 32
    All latent spaces and priors are reduced to d=32; the paper states this is critical to reduce sparsity in distribution matching (Sec. 3.2).
  • Alternating update ratio = VAE updated once per 5 student steps
    Stabilization choice (Appendix A.3) with no sensitivity analysis reported.
axioms (4)
  • standard math A distribution is uniquely characterized by its time-dependent score function, and diffusion models can serve as universal distribution estimators.
    Invoked in Sec. 3 to justify matching q(z) and p_r(z) through learned score functions; from Song et al. (2020).
  • domain assumption The gradient approximation in Eq. (9), ∇ L_DM ≈ E[w_t (s_fake − s_real) dE/dθ], is a valid proxy for minimizing D_KL(q||p_r) and remains stable during training.
    Inherited from DMD (Yin et al. 2024). The paper itself notes instability for distant distributions and that the constraint acts only as a regularizer in practice (Sec. 7).
  • ad hoc to paper The pretrained DINOv2 aggregate posterior on ImageNet is a representative and fair 'SSL-derived distribution' to serve as the reference target.
    The choice of DINO features is central to the main conclusion, but the reference is extracted from the same frozen encoder used to initialize the tokenizer (App. A.1/A.2), making the comparison asymmetric.
  • ad hoc to paper The heuristic weighting of λ_DM by reference-distribution category yields a fair comparison.
    Sec. 4 states the weighting is based on presumed distance to the data manifold; if this does not compensate correctly, the ranking of priors is an artifact of hyperparameters.

reviewed 2026-08-03 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Distribution Matching Variational AutoEncoder." pith.science (2026). https://pith.science/paper/C26QXQ2O

@misc{pith2026251207778,
  author       = {Pith},
  title        = {Pith review of: Distribution Matching Variational AutoEncoder},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C26QXQ2O}},
  note         = {Machine review of arXiv:2512.07778}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Most visual generative models compress images into a latent space before applying diffusion or autoregressive modelling. Yet, existing approaches such as VAEs and foundation model aligned encoders implicitly constrain the latent space without explicitly shaping its distribution, making it unclear which types of distributions are optimal for modeling. We introduce \textbf{Distribution-Matching VAE} (\textbf{DMVAE}), which explicitly aligns the encoder's latent distribution with an arbitrary reference distribution via a distribution matching constraint. This generalizes beyond the Gaussian prior of conventional VAEs, enabling alignment with distributions derived from self-supervised features, diffusion noise, or other prior distributions. With DMVAE, we can systematically investigate which latent distributions are more conducive to modeling, and we find that SSL-derived distributions provide an excellent balance between reconstruction fidelity and modeling efficiency, reaching gFID equals 3.2 on ImageNet with only 64 training epochs. Our results suggest that choosing a suitable latent distribution structure (achieved via distribution-level alignment), rather than relying on fixed priors, is key to bridging the gap between easy-to-model latents and high-fidelity image synthesis. Code is avaliable at https://github.com/sen-ye/dmvae.

Figures

Figures reproduced from arXiv: 2512.07778 by Chunyu Wang, Han Hu, Jianning Pei, Liwei Wang, Mengde Xu, Sen Ye, Shuyang Gu.

Figure 1
Figure 1. Figure 1: Illustrution of VAE [13], RAE [41], pointwise matching encoder [3, 36], and Distribution Matching VAE. where z0 is StopGradient[Eθ(x)]. Distribution Matching Loss (Trains Eθ): It penalizes the encoder Eθ by comparing the scores of the student and teacher models. Following [38], we compute the gradient directly without back-propagating through the fake score network: ∇θLDM ≃ Et,x[wt(sfake(zt, t) − sreal(zt,… view at source ↗
Figure 2
Figure 2. Figure 2: The training pipeline of Distribution Matching VAE. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Analysis of different distribution matching objectives on [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Illustration of t-SNE on different distributions. (a-d) [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Different CFG scale represents different distributions. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Comparison of the convergence speed. ing and better final generative quality. • Classifier-Free Guidance (CFG): While our default set￾ting disables CFG (guidance weight of 1.0), we find that applying a small CFG weight (e.g., 3.0) during the score matching process (Eq. (9)) can slightly improve genera￾tive quality, albeit at a minor cost to reconstruction. We visualize the how the real diffusion models the… view at source ↗
Figure 7
Figure 7. Figure 7: DMD update direction visualized at different timesteps in a 2D toy setting. [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Quanlitative comparison of convergence speed on ImageNet 256×256. We compare DMVAE with VAVAE and report conditional results without CFG. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Diffusing in the Right Space: A Systematic Study of Latent Diffusability

    cs.CV 2026-06 unverdicted novelty 7.0

    A large-scale empirical study across tokenizers and diffusion backbones identifies Velocity Irreducible Variance (VIV) as one of the most stable predictors of latent diffusion generation quality.

  2. DecQ: Detail-Condensing Queries for Enhanced Reconstruction and Generation in Representation Autoencoders

    cs.CV 2026-05 unverdicted novelty 7.0

    DecQ uses detail-condensing queries on shallow and deep VFM features to improve both reconstruction PSNR and generative convergence/FID in RAEs without fine-tuning the encoder.

Reference graph

Works this paper leans on

42 extracted references · 14 linked inside Pith · cited by 2 Pith papers

  1. [1]

    End-to- end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to- end object detection with transformers. InEuropean confer- ence on computer vision, pages 213–229. Springer, 2020. 4

    2020

  2. [2]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. InPro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 1, 2

    2021

  3. [3]

    Aligning visual foundation encoders to tokenizers for diffusion models.arXiv preprint arXiv:2509.25162, 2025

    Bowei Chen, Sai Bi, Hao Tan, He Zhang, Tianyuan Zhang, Zhengqi Li, Yuanjun Xiong, Jianming Zhang, and Kai Zhang. Aligning visual foundation encoders to tokenizers for diffusion models.arXiv preprint arXiv:2509.25162, 2025. 1, 2, 3, 4, 8, 9, 12

    arXiv 2025

  4. [4]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021. 1, 9, 12

    2021

  5. [5]

    Generative adversarial networks.Commu- nications of the ACM, 63(11):139–144, 2020

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks.Commu- nications of the ACM, 63(11):139–144, 2020. 9

    2020

  6. [6]

    Vec- tor quantized diffusion model for text-to-image synthesis

    Shuyang Gu, Dong Chen, Jianmin Bao, Fang Wen, Bo Zhang, Dongdong Chen, Lu Yuan, and Baining Guo. Vec- tor quantized diffusion model for text-to-image synthesis. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 10696–10706, 2022. 1

    2022

  7. [7]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 2, 6

    2016

  8. [8]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000– 16009, 2022. 9

    2022

  9. [9]

    beta-vae: Learning basic visual con- cepts with a constrained variational framework

    Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual con- cepts with a constrained variational framework. InInterna- tional conference on learning representations, 2017. 9

    2017

  10. [10]

    Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022. 7, 8

    Pith/arXiv arXiv 2022

  11. [11]

    Denoising dif- fusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020. 1

    2020

  12. [12]

    Self forcing: Bridging the train- test gap in autoregressive video diffusion.arXiv preprint arXiv:2506.08009, 2025

    Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train- test gap in autoregressive video diffusion.arXiv preprint arXiv:2506.08009, 2025. 5

    Pith/arXiv arXiv 2025

  13. [13]

    Auto-encoding vari- ational bayes.arXiv preprint arXiv:1312.6114, 2013

    Diederik P Kingma and Max Welling. Auto-encoding vari- ational bayes.arXiv preprint arXiv:1312.6114, 2013. 2, 4, 9

    Pith/arXiv arXiv 2013

  14. [14]

    Flux.https://github.com/ black-forest-labs/flux, 2024

    Black Forest Labs. Flux.https://github.com/ black-forest-labs/flux, 2024. 12

    2024

  15. [15]

    Repa-e: Unlocking vae for end-to-end tuning with latent diffusion transformers

    Xingjian Leng, Jaskirat Singh, Yunzhong Hou, Zhenchang Xing, Saining Xie, and Liang Zheng. Repa-e: Unlocking vae for end-to-end tuning with latent diffusion transformers. arXiv preprint arXiv:2504.10483, 2025. 6

    arXiv 2025

  16. [16]

    Autoregressive image generation without vec- tor quantization.Advances in Neural Information Processing Systems, 37:56424–56445, 2024

    Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vec- tor quantization.Advances in Neural Information Processing Systems, 37:56424–56445, 2024. 8

    2024

  17. [17]

    Aligning latent spaces with flow priors.arXiv preprint arXiv:2506.05240, 2025

    Yizhuo Li, Yuying Ge, Yixiao Ge, Ying Shan, and Ping Luo. Aligning latent spaces with flow priors.arXiv preprint arXiv:2506.05240, 2025. 2, 5

    arXiv 2025

  18. [18]

    Flow matching for generative mod- eling.arXiv preprint arXiv:2210.02747, 2022

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximil- ian Nickel, and Matt Le. Flow matching for generative mod- eling.arXiv preprint arXiv:2210.02747, 2022. 3

    Pith/arXiv arXiv 2022

  19. [19]

    Sit: Explor- ing flow and diffusion-based generative models with scalable interpolant transformers

    Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijnden, and Saining Xie. Sit: Explor- ing flow and diffusion-based generative models with scalable interpolant transformers. InEuropean Conference on Com- puter Vision, pages 23–40. Springer, 2024. 3, 8

    2024

  20. [20]

    Adversarial autoencoders

    Alireza Makhzani, Jonathon Shlens, Navdeep Jaitly, Ian Goodfellow, and Brendan Frey. Adversarial autoencoders. arXiv preprint arXiv:1511.05644, 2015. 3, 9 10

    Pith/arXiv arXiv 2015

  21. [21]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 1, 2, 6, 9

    Pith/arXiv arXiv 2023

  22. [22]

    Dreamfusion: Text-to-3d using 2d diffusion.arXiv preprint arXiv:2209.14988, 2022

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion.arXiv preprint arXiv:2209.14988, 2022. 3

    Pith/arXiv arXiv 2022

  23. [23]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, pages 8748–8763. PmLR, 2021. 9

    2021

  24. [24]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. InInternational confer- ence on machine learning, pages 8821–8831. Pmlr, 2021. 1

    2021

  25. [25]

    Hierarchical text-conditional image gener- ation with clip latents.arXiv preprint arXiv:2204.06125, 1 (2):3, 2022

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents.arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 1

    Pith/arXiv arXiv 2022

  26. [26]

    Preventing posterior collapse with delta-vaes.arXiv preprint arXiv:1901.03416, 2019

    Ali Razavi, A ¨aron van den Oord, Ben Poole, and Oriol Vinyals. Preventing posterior collapse with delta-vaes.arXiv preprint arXiv:1901.03416, 2019. 3, 9

    Pith/arXiv arXiv 1901

  27. [27]

    Taming vaes

    Danilo Jimenez Rezende and Fabio Viola. Taming vaes. arXiv preprint arXiv:1810.00597, 2018. 3, 9

    Pith/arXiv arXiv 2018

  28. [28]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 2, 3, 7, 9

    2022

  29. [29]

    Photorealistic text-to-image diffusion models with deep language understanding.Advances in neural information processing systems, 35:36479–36494, 2022

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding.Advances in neural information processing systems, 35:36479–36494, 2022. 1

    2022

  30. [30]

    Latent diffusion model without variational autoen- coder.arXiv preprint arXiv:2510.15301, 2025

    Minglei Shi, Haolin Wang, Wenzhao Zheng, Ziyang Yuan, Xiaoshi Wu, Xintao Wang, Pengfei Wan, Jie Zhou, and Ji- wen Lu. Latent diffusion model without variational autoen- coder.arXiv preprint arXiv:2510.15301, 2025. 1

    arXiv 2025

  31. [31]

    Score-based generative modeling through stochastic differential equa- tions.arXiv preprint arXiv:2011.13456, 2020

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions.arXiv preprint arXiv:2011.13456, 2020. 1, 3, 5

    Pith/arXiv arXiv 2011

  32. [32]

    Autoregressive model beats diffusion: Llama for scalable image generation.arXiv preprint arXiv:2406.06525, 2024

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation.arXiv preprint arXiv:2406.06525, 2024. 8

    Pith/arXiv arXiv 2024

  33. [33]

    Neural discrete representation learning.Advances in neural information pro- cessing systems, 30, 2017

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning.Advances in neural information pro- cessing systems, 30, 2017. 1, 9

    2017

  34. [34]

    Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion.Advances in neural information processing systems, 36: 8406–8441, 2023

    Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion.Advances in neural information processing systems, 36: 8406–8441, 2023. 9

    2023

  35. [35]

    Fasterdit: Towards faster diffusion transformers train- ing without architecture modification.Advances in Neural Information Processing Systems, 37:56166–56189, 2024

    Jingfeng Yao, Cheng Wang, Wenyu Liu, and Xinggang Wang. Fasterdit: Towards faster diffusion transformers train- ing without architecture modification.Advances in Neural Information Processing Systems, 37:56166–56189, 2024. 8

    2024

  36. [36]

    Reconstruc- tion vs

    Jingfeng Yao, Bin Yang, and Xinggang Wang. Reconstruc- tion vs. generation: Taming optimization dilemma in latent diffusion models. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 15703–15712, 2025. 1, 2, 3, 4, 7, 8, 9, 12

    2025

  37. [37]

    Im- proved distribution matching distillation for fast image syn- thesis.Advances in neural information processing systems, 37:47455–47487, 2024

    Tianwei Yin, Micha ¨el Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and Bill Freeman. Im- proved distribution matching distillation for fast image syn- thesis.Advances in neural information processing systems, 37:47455–47487, 2024. 9, 12

    2024

  38. [38]

    One-step diffusion with distribution matching distillation

    Tianwei Yin, Micha ¨el Gharbi, Richard Zhang, Eli Shecht- man, Fredo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 6613–6623, 2024. 2, 3, 4, 5, 9

    2024

  39. [39]

    Language model beats diffusion–tokenizer is key to visual generation.arXiv preprint arXiv:2310.05737, 2023

    Lijun Yu, Jos ´e Lezama, Nitesh B Gundavarapu, Luca Ver- sari, Kihyuk Sohn, David Minnen, Yong Cheng, Vighnesh Birodkar, Agrim Gupta, Xiuye Gu, et al. Language model beats diffusion–tokenizer is key to visual generation.arXiv preprint arXiv:2310.05737, 2023. 9

    Pith/arXiv arXiv 2023

  40. [40]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. InProceedings of the IEEE/CVF international conference on computer vision, pages 11975–11986, 2023. 2, 6, 9

    2023

  41. [41]

    Diffusion transformers with representation autoen- coders.arXiv preprint arXiv:2510.11690, 2025

    Boyang Zheng, Nanye Ma, Shengbang Tong, and Saining Xie. Diffusion transformers with representation autoen- coders.arXiv preprint arXiv:2510.11690, 2025. 1, 2, 4, 8, 9

    Pith/arXiv arXiv 2025

  42. [42]

    tennis ball

    Mingyuan Zhou, Huangjie Zheng, Zhendong Wang, Mingzhang Yin, and Hai Huang. Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation. InForty-first International Confer- ence on Machine Learning, 2024. 9 11 Appendix A. Implementation Details Our methodology involves a multi-stage training pipeline...

    2024

This paper was first reviewed by deepseek-v4-flash on August 3, 2026.