MIMFlow: Integrating Masked Image Modeling with Normalizing Flows for End-to-End Image Generation

Limin Wang; Qiushi Guo; Shuai Wang; Tiezheng Ge; Xiaowei Xu; Xinwen Zhang; Yang Chen

REVIEW 2 major objections 6 minor 57 references

MIMFlow: Integrating Masked Image Modeling with Normalizing Flows for End-to-End Image Generation

T0 review · 2 major / 6 minor · reviewed 2026-07-12 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Masking forces normalizing flows to model semantics instead of pixel noise, cutting FID 32.8% with half the tokens.

desk verdict Solid systems paper: masked learnable-token bottleneck + end-to-end NF prior halves tokens and cuts same-scale FID by ~33%, with ablations that mostly back the causal story. read the letter →

arxiv 2606.26016 v3 pith:G3SKPAV6 submitted 2026-06-24 cs.CV

Yang Chen , Xiaowei Xu , Shuai Wang , Xinwen Zhang , Qiushi Guo , Tiezheng Ge , Limin Wang This is my paper · ORCID

classification cs.CV

keywords normalizingflowsmaskedimagemodelingend-to-endgenerationlatentsemanticbottleneckNetexactdensityestimationVAE

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Normalizing flows can do exact density estimation and sampling in one network, but invertibility forces them to spend capacity on low-level pixel details and leaves high-level structure under-modeled. MIMFlow trains a VAE encoder on masked images so that a short set of learnable tokens must recover global context; a normalizing flow then models only that compact latent, while a decoder reconstructs texture. On ImageNet 256×256 the resulting model reaches FID 2.50 and 71.3% linear-probe accuracy with only 128 tokens—half the usual budget—and improves a similar-scale flow baseline by 32.8%. The claim is that the mask-induced bottleneck is what finally frees flows from redundant noise and makes them competitive generative models.

What carries the argument

The learnable-token masked bottleneck: K fixed query tokens (K=128) aggregate information from a randomly masked image via a bidirectional Transformer encoder; the flow models only those tokens while the decoder synthesizes pixels.

What would settle it

Train the identical end-to-end architecture, auxiliary losses and GAN schedule with the mask ratio fixed at zero (or with a non-semantic random bottleneck of the same length) and check whether FID remains near 2.50 or reverts toward the unmasked baseline.

Watch

Extended reading notes

Core claim

Jointly training a normalizing flow on fixed-length semantic latents extracted from masked images, together with a pixel decoder, decouples low-frequency structure from high-frequency synthesis and thereby removes the capacity bottleneck that has limited flow-based image generation.

Load-bearing premise

That the large FID gain is mainly caused by the mask-induced semantic bottleneck rather than by the simultaneous package of end-to-end training, auxiliary feature losses, GAN fine-tuning and architectural upgrades already present in the authors’ improved baseline.

Editorial extensions

If this is right

Normalizing-flow generators can operate on half the usual token count without losing fidelity once the latent is forced to be semantic.
Linear-probe accuracy above 70% becomes a practical side-product of the same training run that produces competitive FID.
Training memory and sampling latency drop substantially because the flow never sees a dense 256-token grid.
Exact-likelihood models regain competitiveness with diffusion and autoregressive systems on class-conditional ImageNet.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same masked-bottleneck recipe may transfer to other invertible or continuous-time generators that currently overfit high-frequency detail.
If the causal factor is truly the information bottleneck rather than the particular MIM objective, any sufficiently strong semantic compressor could replace masking.
The progressive ablation already shows that unmasked end-to-end training alone yields a large jump; future work can quantify how much of the remaining gap is uniquely attributable to masking.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The useful takeaway is simple: they put a fixed-length learnable-token bottleneck on a masked ViT encoder, train a STARFlow-style prior jointly with reconstruction and DINO/CLIP aux losses, then GAN-finetune the decoder. Result on ImageNet 256 is FID 2.50 with 128 tokens (half the usual grid), 71.3% linear probe, and a clear win over their own ~480M SimFlow-scale baseline (3.72 → 2.50).

What is actually new is the concrete joint package, not the individual pieces. End-to-end VAE+flow already exists (SimFlow, REPA-E); MIM tokenizers already exist (MAETok, DeTok). The combination—mask-conditioned fixed-K queries feeding an NF prior, trained end-to-end—is cleanly executed and ships code. They do the right engineering: progressive baseline upgrades (Table 1), mask-ratio / K / σ / aux ablations (Table 5), progressive component stack (Table 8), UMAP, and Jacobian singular-value checks. Table 5a is the load-bearing control: under identical e2e training, no-mask collapses both gFID (12.82→29) and probe accuracy (71.3%→56.6%). Table 8 shows masking as the largest single step once the improved STARFlow backbone is in place. Efficiency numbers (memory, throughput, sampling time) are real and useful for anyone still working with latent NFs.

Soft spot, in proportion: the narrative slightly over-credits “principled semantic decoupling” for the full 3.72→2.50 jump. That jump is the whole package (architecture + e2e + aux + GAN + mask). The paper does not hide this, but the abstract and intro lean hard on the decoupling story. Residual risk is only attribution, not validity of the reported system. Math is standard ELBO + change-of-variables; no circularity; free parameters are ordinary hyperparameters. Citations are fair to the immediate prior art.

This is for people who still care about exact-likelihood latent flows and want a practical recipe that closes some of the gap to diffusion while cutting sequence length. It will not reorganize the field, but it is a real, reproducible step inside that niche. I would send it to referees without hesitation; the evidence is strong enough that the remaining debate is about how much of the gain is mask vs. the rest of the stack, not whether the numbers hold.

Referee Report

2 major / 6 minor

Summary. MIMFlow proposes an end-to-end generative framework that couples Masked Image Modeling with a latent Normalizing Flow. A ViT encoder with learnable query tokens extracts a fixed-length latent from randomly masked images; an NF models the density of that latent as a VAE prior; and a generative decoder reconstructs pixels (with optional DINO/CLIP auxiliary supervision and a short GAN fine-tuning stage). The central claim is that the mask-induced bottleneck forces the latent toward global structure, so the NF models a simpler semantic manifold while the decoder handles high-frequency detail. On ImageNet 256 imes256, MIMFlow-L reports FID 2.50 (CFG), 71.3% linear-probe accuracy, and a 32.8% FID reduction versus a similar-scale SimFlow-L baseline, using only 128 tokens.

Significance. If the results hold, the paper is a solid, practical advance for latent normalizing flows rather than a paradigm shift relative to leading diffusion/AR systems. Its main value is showing that MIM-style bottlenecks can be integrated into joint NF–VAE training (not only modular tokenizers), with clear efficiency gains from 128-token latents (memory, throughput, sampling time in Table 4) and supporting analyses (linear probe, UMAP, Jacobian spectra, progressive ablations). Code release and a carefully built same-scale baseline strengthen reproducibility. The contribution is most significant within the NF literature, where capacity bottlenecks and token efficiency remain open issues.

major comments (2)

Section 4.2 and Abstract claim a 32.8% FID gain (3.72→2.50) over similar-scale NF baselines and attribute it primarily to mask-induced semantic/low-frequency decoupling. Table 1 already moves an improved STARFlow-L e2e baseline to ~3.70, and progressive Table 8 shows multiple stacked changes (learnable tokens, aux, masking, GAN FT). Table 5a does isolate masking under matched e2e training (gFID 29.0→12.82; Acc. 56.6%→71.3%), which supports the mechanism, but the abstract/intro should state more carefully that the headline 32.8% is for the full system, with masking validated as a major controlled factor rather than the sole cause of the entire jump.
Sections 1 and 3 repeatedly describe the NF as modeling a 'low-frequency semantic manifold,' yet the evidence is semantic (linear probe, UMAP in Fig. 4) and numerical (Jacobian spectra in Fig. 5), not a frequency decomposition of latents or reconstructions. The decoupling story is plausible and consistent with mask-ratio/token-count ablations (Tables 5a, 5c), but the 'low-frequency' wording should be either supported by a simple spectral/frequency analysis of z or reconstructions, or tempered to 'semantic / structure-biased' to avoid overclaiming the frequency content.

minor comments (6)

Table 3 mixes models with very different token budgets (128 vs 256 vs 1024) and parameter scales; a short note in the caption clarifying that MIMFlow’s efficiency claim is partly from token compression (cross-ref Table 4/10) would help readers interpret the NF-category comparison.
Section 3.2, Eq. (2)–(6): the ELBO presentation is standard, but it would help to state explicitly whether β is annealed and how the fixed-variance posterior interacts with the additive noise σ in Eq. (1) (both appear as free knobs in the ablations).
Table 5 ablations use 50 epochs and 10K samples while Table 3 uses full training and the ADM suite; please mark this consistently in every ablation caption (already noted in 4.3) so readers do not compare absolute gFID numbers across tables.
Fig. 3 samples are useful; adding a brief failure-mode or diversity note (or more classes) would balance the qualitative presentation.
Minor formatting/spacing artifacts appear in the Related Work and elsewhere in the compiled text (e.g., concatenated words around MIM headings); a proofreading pass would improve readability.
Limitation section correctly flags text-to-image and other generative families; a one-sentence comment on whether the masked encoder is used only at train time (sampling is pure NF→decoder) would make the inference path fully explicit for practitioners.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: architectural proposal and joint losses evaluated on independent held-out ImageNet metrics (FID, linear probe).

full rationale

MIMFlow is an empirical systems paper. Its central claims (mask-induced semantic bottleneck lets the NF model a simpler manifold, yielding FID 2.50 and 71.3% linear-probe accuracy with 128 tokens) rest on a standard ELBO-style joint objective (reconstruction + NF NLL + optional auxiliary DINO/CLIP feature prediction) plus ablations (Tables 5a, 8) and external benchmarks. Linear-probe accuracy and FID/IS are computed on held-out ImageNet data with frozen external classifiers/feature extractors; they are not algebraic rearrangements of the training losses or of any fitted free parameter. The progressive baseline (Table 1) and progressive ablation (Table 8) isolate components without defining any reported metric in terms of itself. Self-citations (SimFlow, STARFlow, FlowBack) appear only as related work or architectural starting points and are not load-bearing uniqueness theorems. No equation, definition, or prediction reduces by construction to its own inputs. Score 0 is therefore the correct, non-manufactured finding.

Assumptions & free parameters 6 free parameters · 3 assumptions · 2 invented entities

The work is empirical; its claims rest on standard deep-learning practice plus a handful of hand-chosen architectural and loss hyper-parameters that are ablated but not derived. No new physical entities are postulated; the “learnable token bottleneck” is an architectural device whose utility is measured by the same metrics used to claim success.

free parameters (6)

mask_ratio_range = 0.4–0.6
Default 0.4–0.6 chosen by ablation; performance collapses outside this band.
latent_token_count_K = 128
Set to 128 after ablation; 64 and 192 both degrade FID.
latent_noise_scale_sigma = 0.3
Additive Gaussian noise variance for continuous density modeling; optimal at 0.3.
flow_loss_weight_beta = 1.0
Balances NLL against reconstruction; fixed at 1.0 without extensive search reported.
aux_loss_weight_gamma
Weight on DINO/CLIP feature prediction; present in final model but not exhaustively swept.
GAN_loss_weight_alpha = 0.05
Adversarial fine-tuning weight set to 0.05.

assumptions (3)

domain assumption A fixed-variance Gaussian posterior around the encoder output yields a valid ELBO when the prior is an invertible flow.
Section 3.2; standard VAE-with-flow construction used by SimFlow.
ad hoc to paper Masking the encoder input forces the latent to discard high-frequency pixel noise and retain global structure.
Core design hypothesis of Sections 1 and 3.1; supported by linear-probe and UMAP but not proven.
domain assumption Classifier-free guidance applied only inside the final deep flow block is a valid sampling procedure.
Inherited from TARFlow and used at inference.

invented entities (2)

Learnable Token Bottleneck
purpose: Produce a fixed-length, mask-invariant latent sequence that the flow can model stably.
Architectural device introduced in Section 3.1; utility measured only by the paper’s own metrics.
MIMFlow joint objective
purpose: Simultaneously optimize reconstruction, flow NLL and auxiliary semantic prediction under masking.
Composite loss of Section 3.2–3.3; no external falsifiable prediction beyond ImageNet FID.

how reviews work

0 comments

Cite this review

Pith. "Pith review of MIMFlow: Integrating Masked Image Modeling with Normalizing Flows for End-to-End Image Generation." pith.science (2026). https://pith.science/paper/G3SKPAV6

@misc{pith2026260626016,
  author       = {Pith},
  title        = {Pith review of: MIMFlow: Integrating Masked Image Modeling with Normalizing Flows for End-to-End Image Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G3SKPAV6}},
  note         = {Machine review of arXiv:2606.26016}
}

abstract

Normalizing Flows (NFs) are powerful generative models capable of exact density estimation and sampling. However, their strict invertibility often forces the model to exhaust its capacity on low-level pixel details, hindering the capture of high-level semantic structures. While Masked Image Modeling (MIM) has excelled in representation learning, its integration into generative pipelines has remained largely modular and disjointed. In this paper, we propose MIMFlow, a unified end-to-end framework that jointly optimizes latent semantics, pixel reconstruction, and generative flow. By employing a VAE encoder to infer semantic latent from masked images, MIMFlow achieves a principled decoupling of the generative task: the Normalizing Flow focuses on modeling a simplified, low-frequency semantic manifold, while a specialized decoder handles high-frequency synthesis. This design effectively resolves the inherent capacity bottleneck of NFs, allowing the model to prioritize global structural coherence over redundant noise. Empirical results on ImageNet 256$\times$256 show that MIMFlow-L reaches 71.3\% linear probing accuracy and an FID of 2.50. Despite using only 128 tokens (50\% fewer than standard models), it yields a 32.8\% performance gain over similar-scale NF baselines. Our code is available at https://github.com/MCG-NJU/MIMFlow.

Figures

Figures reproduced from arXiv: 2606.26016 by the authors.

**Figure 1.** MIM in Different Paradigms. (a) Self-Supervised Learning: Employs highratio masking as a self-supervised proxy task for representation learning. (b) Generative Tokenizers: A two-stage approach where the latent space is pre-trained with MIM before training a separate generative model. (c) MIMFlow (Ours): A unified framework that jointly optimizes latent semantics, pixel reconstruction, and generative flow in an end… view at source ↗

**Figure 2.** Structure of MIMFlow. N is the number of image patches, K is the number of learnable latent query tokens, m is the binary mask, and e denotes learnable decoder embeddings. MAE [17] and SimMIM [43], present inherent difficulties for density estimation. MAE only processes visible patches, resulting in a latent sequence whose length and positional context vary with the random mask pattern, which imposes an intractable … view at source ↗

**Figure 3.** Selected Samples on ImageNet 256 × 256 from MIMFlow-L. We use classifierfree guidance equal to 2.0. flow model to learn a more efficient and structured semantic manifold, extracting higher generative value from the same parameter budget. Efficiency via Token Compression. A key highlight of MIMFlow is its token efficiency. While most latent models (e.g., DiT, LDM, SimFlow) operate on a 16 × 16 = 256 token grid, and … view at source ↗

Figures from the paper (5 more)

**Figure 3.** Figure 3: Selected Samples on ImageNet 256 × 256 from MIMFlow-L. We use classifierfree guidance equal to 2.0. We iteratively construct our baseline starting from a STARFlow-L model trained on a fixed VAE latent space. The refinement process involves: (1) removing the softplus …

**Figure 4.** Figure 4: UMAP visualization on ImageNet of the learned latent space from (a) SDVAE; (b) MIMFlow. Colors indicate different classes. MIMFlow presents a more discriminative latent space. flow model, thereby violating the principled decoupling and hindering the NF’s ability to m…

**Figure 5.** Figure 5: Jacobian Spectral Analysis of STARFlow and MIMFlow. The three panels report, from left to right, the empirical distributions of the largest singular value σmax(J), the smallest singular value σmin(J), and the log-condition number log10 κ(J) (with κ(J) = σmax(J)/σmin(J…

**Figure 6.** Figure 6: Linear Probe Accuracy vs Depth under Different Mask Ratios. C.3 Efficiency Analysis A key advantage of our MIMFlow is its high efficiency, achieved through a significantly reduced token budget. While existing methods typically rely on 256 or even 1024 tokens to repres…

**Figure 6.** Figure 6: Linear Probe Accuracy vs Depth under Different Mask Ratios. References 1. Chen, H., Han, Y., Chen, F., Li, X., Wang, Y., Wang, J., Wang, Z., Liu, Z., Zou, D., Raj, B.: Masked autoencoders are effective tokenizers for diffusion models (2025) 2, 4, 7, 9 2. Chen, R.T.Q., …

Review history (3 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 17 linked inside Pith

[1]

Chen, H., Han, Y., Chen, F., Li, X., Wang, Y., Wang, J., Wang, Z., Liu, Z., Zou, D., Raj, B.: Masked autoencoders are effective tokenizers for diffusion models (2025) 2, 4, 7, 9

2025
[2]

Chen, R.T.Q., Rubanova, Y., Bettencourt, J., Duvenaud, D.: Neural ordinary dif- ferential equations (2019) 1

2019
[3]

arXiv preprint arXiv:2504.07963 (2025) 10

Chen, S., Ge, C., Zhang, S., Sun, P., Luo, P.: Pixelflow: Pixel-space generative models with flow. arXiv preprint arXiv:2504.07963 (2025) 10

arXiv 2025
[4]

In: Proceedings of the AAAI Conference on Artificial Intelligence

Chen, Y., Xu, X., Wang, S., Zhu, C., Wen, R., Li, X., Ge, T., Wang, L.: Flowing backwards: Improving normalizing flows via reverse representation alignment. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 40, pp. 3074– 3082 (2026) 1, 3, 10

2026
[5]

IEEE Conference on Computer Vision and Pattern Recognition pp

Deng,J.,Dong,W.,Socher,R.,Li,L.J.,Li,K.,Fei-Fei,L.:ImageNet:ALarge-scale Hierarchical Image Database. IEEE Conference on Computer Vision and Pattern Recognition pp. 248–255 (2009) 8

2009
[6]

Advances in Neural Information Processing Systems34, 8780–8794 (2021) 8, 10, 16

Dhariwal, P., Nichol, A.: Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems34, 8780–8794 (2021) 8, 10, 16

2021
[7]

arXiv preprint arXiv:1410.8516 (2014) 3

Dinh, L., Krueger, D., Bengio, Y.: Nice: Non-linear independent components esti- mation. arXiv preprint arXiv:1410.8516 (2014) 3

arXiv 2014
[8]

arXiv preprint arXiv:1605.08803 (2016) 1, 3

Dinh, L., Sohl-Dickstein, J., Bengio, S.: Density estimation using real nvp. arXiv preprint arXiv:1605.08803 (2016) 1, 3

arXiv 2016

Show all 57 references

[9]

In: International Conference on Artificial Intelligence and Statistics

Draxler, F., Sorrenson, P., Zimmermann, L., Rousselot, A., Köthe, U.: Free-form flows: Make any architecture a normalizing flow. In: International Conference on Artificial Intelligence and Statistics. pp. 2197–2205. PMLR (2024) 3

2024
[10]

arXiv preprint arXiv:2402.06578 (2024) 3 MIMFlow 21

Draxler, F., Wahl, S., Schnörr, C., Köthe, U.: On the universality of volume- preserving and coupling-based normalizing flows. arXiv preprint arXiv:2402.06578 (2024) 3 MIMFlow 21

2024 arXiv
[11]

In: Proceedings of the IEEE/CVF International Conference on Computer Vision

Gao, S., Zhou, P., Cheng, M.M., Yan, S.: Masked diffusion transformer is a strong image synthesizer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 23164–23173 (2023) 10

2023
[12]

Gao, Y., Chen, C., Chen, T., Gu, J.: One layer is enough: Adapting pretrained visual encoders for image generation (2025) 10

2025
[13]

Advances in Neu- ral Information Processing Systems33, 22104–22117 (2020) 3

Giaquinto, R., Banerjee, A.: Gradient boosted normalizing flows. Advances in Neu- ral Information Processing Systems33, 22104–22117 (2020) 3

2020
[14]

arXiv preprint arXiv:2506.06276 (2025) 3, 8, 10

Gu,J.,Chen,T.,Berthelot,D.,Zheng,H.,Wang,Y.,Zhang,R.,Dinh,L.,Bautista, M.A., Susskind, J., Zhai, S.: Starflow: Scaling latent normalizing flows for high- resolution image synthesis. arXiv preprint arXiv:2506.06276 (2025) 3, 8, 10

2025 arXiv
[15]

Gu, J., Chen, T., Shen, Y., Berthelot, D., Zhai, S., Susskind, J.: Normalizing trajectory models (2026) 1

2026
[16]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

Gu, J., Shen, Y., Chen, T., Dinh, L., Wang, Y., Bautista, M.A., Berthelot, D., Susskind, J., Zhai, S.: Starflow-v: End-to-end video generative modeling with au- toregressive normalizing flows. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...

2026
[17]

Machine Intelligence Research pp

Han, C., Fan, J., Wu, N., Dai, J., Bao, H., Lu, X.: Object-centric Video Prediction with Mask-guided Spatiotemporal Diffusion. Machine Intelligence Research pp. 1– 11 (2026) 3

2026
[18]

He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.: Masked autoencoders are scalable vision learners (2021) 2, 4, 5, 18

2021
[19]

Advances in neural information processing systems30(2017) 8

Heusel,M.,Ramsauer,H.,Unterthiner,T.,Nessler,B.,Hochreiter,S.:Ganstrained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems30(2017) 8

2017
[20]

arXiv preprint arXiv:2410.19324 (2024) 10

Hoogeboom, E., Mensink, T., Heek, J., Lamerigts, K., Gao, R., Salimans, T.: Simpler diffusion (sid2): 1.5 fid on imagenet512 with pixel-space diffusion. arXiv preprint arXiv:2410.19324 (2024) 10

2024 arXiv
[21]

Machine Intelligence Research21(2), 272–282 (2024) 3

Hu,J.,Wu,L.,Chen,Y.,Hu,P.,Zaki,M.J.:GraphFlow+:ExploitingConversation Flow in Conversational Machine Comprehension with Graph Neural Networks. Machine Intelligence Research21(2), 272–282 (2024) 3

2024
[22]

arXiv preprint arXiv:2212.11972 (2022) 10

Jabri, A., Fleet, D., Chen, T.: Scalable adaptive computation for iterative genera- tion. arXiv preprint arXiv:2212.11972 (2022) 10

2022 arXiv
[23]

Kingma, D.P., Dhariwal, P.: Glow: Generative flow with invertible 1x1 convolutions (2018) 1, 3

2018
[24]

IEEE transactions on pattern analysis and machine intelligence43(11), 3964–3979 (2020) 3

Kobyzev, I., Prince, S.J., Brubaker, M.A.: Normalizing flows: An introduction and review of current methods. IEEE transactions on pattern analysis and machine intelligence43(11), 3964–3979 (2020) 3

2020
[25]

Advances in Neural Information Processing Systems32(2019) 8

Kynkäänniemi, T., Karras, T., Laine, S., Lehtinen, J., Aila, T.: Improved precision and recall metric for assessing generative models. Advances in Neural Information Processing Systems32(2019) 8

2019
[26]

In: arXiv preprint arXiv:2405.18373 (2024) 2, 4, 10

Lee, S.H., Park, S., Kim, G.M.: REPA-E: End-to-end training of latent-diffusion models via representation alignment. In: arXiv preprint arXiv:2405.18373 (2024) 2, 4, 10

2024 arXiv
[27]

Li, C., Tang, H., Zhu, Y., Yamanishi, Y.: A reinforcement learning-driven trans- formerganformoleculargeneration.MachineIntelligenceResearchpp.1–22(2026) 1

2026
[28]

Advances in Neural Information Processing Systems37, 56424–56445 (2024) 10 22 Y

Li, T., Tian, Y., Li, H., Deng, M., He, K.: Autoregressive image generation with- out vector quantization. Advances in Neural Information Processing Systems37, 56424–56445 (2024) 10 22 Y. Chen et al

2024
[29]

arXiv preprint arXiv:2401.08740 (2024) 10

Ma, N., Goldstein, M., Albergo, M.S., Boffi, N.M., Vanden-Eijnden, E., Xie, S.: Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers. arXiv preprint arXiv:2401.08740 (2024) 10

2024 arXiv
[30]

arXiv preprint arXiv:2304.07193 (2023) 7

Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al.: Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 (2023) 7

2023 arXiv
[31]

Journal of Machine Learning Research22(57), 1–64 (2021) 3

Papamakarios, G., Nalisnick, E., Rezende, D.J., Mohamed, S., Lakshminarayanan, B.: Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research22(57), 1–64 (2021) 3

2021
[32]

In: Proceedings of the IEEE/CVF International Conference on Computer Vision

Peebles, W., Xie, S.: Scalable diffusion models with transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 4195–4205 (2023) 10

2023
[33]

Qiu, Z., Wang, Z., Zheng, B., Huang, Z., Wen, K., Yang, S., Men, R., Yu, L., Huang, F., Huang, S., Liu, D., Zhou, J., Lin, J.: Gated attention for large language models: Non-linearity, sparsity, and attention-sink-free (2025) 9

2025
[34]

arXiv preprint arXiv:2103.00020 (2021) 7

Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. arXiv preprint arXiv:2103.00020 (2021) 7

2021 arXiv
[35]

arXiv preprint arXiv:2502.20388 (2025) 10

Ren, S., Yu, Q., He, J., Shen, X., Yuille, A., Chen, L.C.: Beyond next-token: Next- x prediction for autoregressive visual generation. arXiv preprint arXiv:2502.20388 (2025) 10

2025 arXiv
[36]

In: Bach, F., Blei, D

Rezende, D., Mohamed, S.: Variational inference with normalizing flows. In: Bach, F., Blei, D. (eds.) Proceedings of the 32nd International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 37, pp. 1530–1538. PMLR, Lille, France (07–09 Jul 2015) 3

2015
[37]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 10684–10695 (2022) 9

2022
[38]

Advances in neural information processing systems29(2016) 8

Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X.: Improved techniques for training gans. Advances in neural information processing systems29(2016) 8

2016
[39]

Shen, Y., Chen, T., Gao, Y., Zhang, Y., Wang, Y., Ángel Bautista, M., Zhai, S., Susskind, J.M., Gu, J.: Starflow2: Bridging language models and normalizing flows for unified multimodal generation (2026) 1

2026
[40]

Singh, J., Zheng, B., Wu, Z., Zhang, R., Shechtman, E., Xie, S.: Improved baselines with representation autoencoders (2026) 2

2026
[41]

Tian, K., Jiang, Y., Yuan, Z., Peng, B., Wang, L.: Visual autoregressive modeling: Scalableimagegenerationvianext-scaleprediction.Advancesinneuralinformation processing systems37, 84839–84865 (2024) 10

2024
[42]

arXiv preprint arXiv:2411.19722 (2024) 1, 3, 10

Tschannen, M., Pinto, A.S., Kolesnikov, A.: Jetformer: An autoregressive genera- tive model of raw images and text. arXiv preprint arXiv:2411.19722 (2024) 1, 3, 10

2024 arXiv
[43]

Tu, G., Fu, X., Yu, S., Tang, Y., Kang, H., Qin, L., Zhang, Y., Gu, J.: Latent reasoning with normalizing flows (2026) 1

2026
[44]

Wang, S., Gao, Z., Zhu, C., Huang, W., Wang, L.: Pixnerd: Pixel neural field diffusion (2025) 10

2025
[45]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Wang, S., Tian, Z., Huang, W., Wang, L.: Ddt: Decoupled diffusion transformer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 40633–40642 (June 2026) 10

2026
[46]

Xie, Z., Zhang, Z., Cao, Y., Lin, Y., Bao, J., Yao, Z., Dai, Q., Hu, H.: Simmim: A simple framework for masked image modeling (2022) 2, 4, 5 MIMFlow 23

2022
[47]

arXiv preprint arXiv:2604.28190 (2026) 15

Yang, J., Geng, Z., Ju, X., Tian, Y., Wang, Y.: Representation fréchet loss for visual generation. arXiv preprint arXiv:2604.28190 (2026) 15

2026 arXiv
[48]

Yang, J., Li, T., Fan, L., Tian, Y., Wang, Y.: Latent denoising makes good tok- enizers (2026) 2, 4

2026
[49]

Yao, J., Song, Y., Zhou, Y., Wang, X.: Towards scalable pre-training of visual tokenizers for generation (2025) 2

2025
[50]

generation: Taming optimization dilemma in latent diffusion models (2025) 10

Yao, J., Yang, B., Wang, X.: Reconstruction vs. generation: Taming optimization dilemma in latent diffusion models (2025) 10

2025
[51]

arXiv preprint arXiv:2410.06940 (2024) 2, 4, 8, 10

Yu, S., Kwak, S., Jang, H., Jeong, J., Huang, J., Shin, J., Xie, S.: Representation alignment for generation: Training diffusion transformers is easier than you think. arXiv preprint arXiv:2410.06940 (2024) 2, 4, 8, 10

2024 arXiv
[52]

arXiv preprint arXiv:2412.06329 (2024) 1, 3, 10, 16

Zhai, S., Zhang, R., Nakkiran, P., Berthelot, D., Gu, J., Zheng, H., Chen, T., Bautista, M.A., Jaitly, N., Susskind, J.: Normalizing flows are capable generative models. arXiv preprint arXiv:2412.06329 (2024) 1, 3, 10, 16

2024 arXiv
[53]

Zhao, Q., Zheng, G., Yang, T., Zhu, R., Leng, X., Gould, S., Zheng, L.: Simflow: Simplified and end-to-end training of latent normalizing flows (2025) 1, 3, 4, 5, 10

2025
[54]

Zheng, B., Ma, N., Tong, S., Xie, S.: Diffusion transformers with representation autoencoders (2025) 2, 10, 16

2025
[55]

Zheng, G., Zhao, Q., Yang, T., Xiao, F., Lin, Z., Wu, J., Deng, J., Zhang, Y., Zhu, R.: Farmer: Flow autoregressive transformer over pixels (2025) 1, 10

2025
[56]

In: Transactions on Machine Learning Research (TMLR) (2024) 10

Zheng, H., Nie, W., Vahdat, A., Anandkumar, A.: Fast training of diffusion mod- els with masked transformers. In: Transactions on Machine Learning Research (TMLR) (2024) 10

2024
[57]

In: arXiv preprint arXiv:2405.15438 (2024) 2, 4

Zheng, Y., Tian, Y., Li, S., Wu, Z., Liu, B., Li, J., Ye, B., Zhou, J.R.: LightningDiT: A vision-foundation-model-aligned VAE for fast and high-quality generation. In: arXiv preprint arXiv:2405.15438 (2024) 2, 4

2024 arXiv

Pith tools

Reviewed July 12, 2026 · model on record in the stance chip above.

[1] [1]

Chen, H., Han, Y., Chen, F., Li, X., Wang, Y., Wang, J., Wang, Z., Liu, Z., Zou, D., Raj, B.: Masked autoencoders are effective tokenizers for diffusion models (2025) 2, 4, 7, 9

2025

[2] [2]

Chen, R.T.Q., Rubanova, Y., Bettencourt, J., Duvenaud, D.: Neural ordinary dif- ferential equations (2019) 1

2019

[3] [3]

arXiv preprint arXiv:2504.07963 (2025) 10

Chen, S., Ge, C., Zhang, S., Sun, P., Luo, P.: Pixelflow: Pixel-space generative models with flow. arXiv preprint arXiv:2504.07963 (2025) 10

arXiv 2025

[4] [4]

In: Proceedings of the AAAI Conference on Artificial Intelligence

Chen, Y., Xu, X., Wang, S., Zhu, C., Wen, R., Li, X., Ge, T., Wang, L.: Flowing backwards: Improving normalizing flows via reverse representation alignment. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 40, pp. 3074– 3082 (2026) 1, 3, 10

2026

[5] [5]

IEEE Conference on Computer Vision and Pattern Recognition pp

Deng,J.,Dong,W.,Socher,R.,Li,L.J.,Li,K.,Fei-Fei,L.:ImageNet:ALarge-scale Hierarchical Image Database. IEEE Conference on Computer Vision and Pattern Recognition pp. 248–255 (2009) 8

2009

[6] [6]

Advances in Neural Information Processing Systems34, 8780–8794 (2021) 8, 10, 16

Dhariwal, P., Nichol, A.: Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems34, 8780–8794 (2021) 8, 10, 16

2021

[7] [7]

arXiv preprint arXiv:1410.8516 (2014) 3

Dinh, L., Krueger, D., Bengio, Y.: Nice: Non-linear independent components esti- mation. arXiv preprint arXiv:1410.8516 (2014) 3

arXiv 2014

[8] [8]

arXiv preprint arXiv:1605.08803 (2016) 1, 3

Dinh, L., Sohl-Dickstein, J., Bengio, S.: Density estimation using real nvp. arXiv preprint arXiv:1605.08803 (2016) 1, 3

arXiv 2016