Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

DC-AR: Efficient Masked Autoregressive Image Generation with Deep Compression Hybrid Tokenizer

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DC-AR claims that a masked autoregressive model can match leading diffusion text-to-image quality at 12 sampling steps and several times higher throughput, using a 32x-compressed hybrid tokenizer that splits structure from detail.

desk verdict A useful systems paper: the 32x 2D hybrid tokenizer and three-stage training recipe are solid engineering, but the claim that residual tokens only refine structure is asserted, not shown, and deserves the main referee push. read the letter →

arxiv 2507.04947 v1 pith:VFSIMW2D submitted 2025-07-07 cs.CV cs.AI

Yecheng Wu , Junyu Chen , Zhuoyang Zhang , Enze Xie , Jincheng Yu , Junsong Chen , Jinyi Hu , Yao Lu
show 2 more authors
Song Han Han Cai
This is my paper
classification cs.CVcs.AI
keywords maskedautoregressivegenerationtext-to-imagehybridtokenizervectorquantizationresidualtokens32xcompressionefficientinferenceresolutiongeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that a masked autoregressive (mask-prediction, BERT-style) text-to-image model can match leading diffusion models in quality while running much faster, provided the image tokenizer compresses 32x and splits each image into discrete structural tokens plus continuous residual tokens. To make that split work at high compression, the authors introduce a three-stage training recipe for the tokenizer: continuous warm-up, discrete learning, and alternating fine-tuning, improving reconstruction FID from 1.92 to 1.60. The resulting system, DC-AR, generates all discrete tokens through progressive unmasking in 12 steps, then regresses residual tokens with a lightweight diffusion head; the paper reports gFID 5.49 on MJHQ-30K, GenEval 0.69, and 1.5-7.9x higher throughput versus leading baselines. A sympathetic reader would care because this is evidence that discrete-token autoregressive pipelines need not trade quality for efficiency.

What carries the argument

The central object is the deep-compression hybrid tokenizer DC-HT with 32x spatial compression (a 256x256 image becomes 64 tokens), built on the DC-AE-f32c32 encoder-decoder and a vector-quantization codebook of size 16384. The load-bearing identity is the residual decomposition Zr = Z − Zq, with the decoder trained to accept either Zq alone or Z = Zq + Zr. At generation, a 634M-parameter transformer with cross-attention predicts masked discrete tokens via cross-entropy while its hidden states condition a 37M-parameter six-layer MLP diffusion head that denoises residual tokens; discrete and residual tokens are summed and then decoded. A three-stage tokenizer training schedule (continuous warm-up, discrete learning, alternating fine-tuning with the encoder and quantizer frozen) is what makes the 32x discrete path reconstruct well, improving discrete-rFID from 6.18 to 5.13.

What would settle it

Take a trained DC-AR, keep the 12-step discrete tokens fixed, and replace the diffusion head's residuals with residuals sampled from the tokenizer's marginal residual distribution; if the split holds, GenEval object-position and attribute scores should stay roughly constant while only texture degrades, and if those scores drop sharply, the residuals carry structure.

Watch

Extended reading notes

Core claim

DC-AR establishes that a MaskGIT-style masked autoregressive generator, operating on 2D spatially structured tokens at 32x compression, can reach leading text-to-image quality if the discrete path carries structure and a residual continuous path carries refinement. The paper's quantitative claim is concrete: gFID 5.49 on MJHQ-30K and GenEval 0.69, with 12 sampling steps, 10.3 images per second at batch 16, and 0.4 seconds latency at batch 1 for 512x512 output, compared with Sana-0.6B at gFID 5.67 and 6.7 images per second and Show-o at gFID 14.59 and 1.3 images per second. The discovery is that hybrid tokenization, quantized tokens Zq plus residuals Zr = Z − Zq summed and decoded, lets the transformer predict all coarse content in a few steps while the diffusion head adds texture, so the coarse-to-fine split is what reconciles quality and speed.

Load-bearing premise

The premise is that the leftover between the continuous image code and its compressed discrete version contains only fine texture, not layout or meaning, so the transformer can generate all discrete tokens first and the diffusion head merely refines.

Editorial extensions

If this is right

  • Discrete-token masked autoregressive generation can match or beat diffusion text-to-image quality at 12 steps, so autoregressive pipelines need not remain confined to low-compression tokenizers.
  • A 2D spatial tokenizer retains cross-resolution generalization, letting a 512x512 generator be obtained by fine-tuning a 256x256 model and cutting training cost by at least 1.9x versus training from scratch.
  • Hybrid residual refinement adds only about 10 percent inference overhead over the discrete-only baseline while improving gFID from 6.71 to 5.50 and GenEval from 0.66 to 0.69.
  • The optimal sampling budget shifts from 64 steps for continuous-token autoregressive models to 12 steps for DC-AR, making masked autoregressive models competitive on latency for interactive text-to-image systems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If residual tokens truly carry no layout structure, the same discrete-first split could be applied to other discrete generators, such as next-token autoregressive or VQ diffusion models, to cut their sampling steps without losing composition; the paper only demonstrates the masked autoregressive setting.
  • The 32x compression plus coarse-to-fine split suggests inference cost scales with the number of structural tokens rather than the number of refinement steps; pushing to 64x compression or adaptive per-image token budgets would be a natural stress test the paper does not run.
  • Because DC-HT is a 2D tokenizer trained only at 256x256, the claimed cross-resolution generalization predicts that it should transfer to 1024x1024 generation with little fine-tuning, which would extend the reported results beyond 512x512.
  • A direct check of the structure-versus-detail split is to measure how much zeroing the residual tokens changes object layout versus texture; if layout changes substantially, the coarse-to-fine decomposition would need revision.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. DC-AR introduces a masked autoregressive text-to-image generation framework built on a new 32x-compression hybrid tokenizer (DC-HT). The tokenizer combines a discrete VQ path with a continuous path through a three-stage training strategy. For generation, a MaskGIT-style transformer first predicts 64 discrete tokens, and a lightweight 6-layer MLP diffusion head then predicts the residual tokens Zr = Z - Zq conditioned on the transformer's final hidden states. The paper reports gFID 5.49 on MJHQ-30K and GenEval 0.69, with 1.5-7.9x higher throughput and 2.0-3.5x lower latency than selected diffusion and masked-AR baselines. It also reports tokenizer rFID 1.60 on ImageNet 256x256 and demonstrates cross-resolution generalization to 512x512.

Significance. If the central claims hold, DC-AR would be a meaningful step toward closing the quality gap between masked autoregressive and diffusion text-to-image models while retaining the sampling efficiency of masked AR. The paper's strengths include a concrete three-stage adaptation training recipe for the tokenizer (with ablations in Tables 4 and 5), a resolution-generalizable 2D tokenizer that supports pre-training then fine-tuning (Table 6), and a detailed hyperparameter appendix. The code link and the reported ablations are valuable for reproducibility. However, the core conceptual premise -- that residual tokens only carry fine-grained refinement and not structure -- is asserted rather than demonstrated, and the headline SOTA comparisons are not controlled for training data or run-to-run variance. These issues currently limit the strength of the claims.

major comments (3)
  1. [Sec. 3.2, Fig. 5, Table 4] The central premise that residual tokens Zr = Z - Zq "exclusively serve a refining function without altering the overall structure" is asserted without direct evidence. With a 32x spatial compression, each of the 64 latent vectors summarizes a 32x32 pixel region, and nothing in the paper measures the information content of Zr. Table 4 demonstrates only that the hybrid model outperforms the discrete-only baseline; it does not show that the improvement comes from fine-grained detail rather than structural or semantic content. The paper should report at least (i) the norm ratio ||Zr||/||Zq||, (ii) reconstruction metrics for Dec(Zq) alone versus Dec(Zq+Zr), and (iii) some probing of layout or semantic information in Zr (e.g., linear probes for object class or position). Without such evidence, the coarse-to-fine interpretation, the 12-step optimality, and the scaling claims are not established.
  2. [Sec. 4.1, Tables 2 and 3] The headline "state-of-the-art" comparisons are uncontrolled. DC-AR is trained on JourneyDB plus an internal MidJourney-style synthetic dataset with VILA1.5-generated captions, while the baselines are trained on different, mostly public data distributions. On MJHQ-30K, which is derived from MidJourney images, a model trained on a MidJourney-style synthetic set has an inherent advantage. Moreover, all gFID and GenEval numbers are reported as single runs with no error bars or multiple seeds; differences of 0.01 in GenEval (DC-AR 0.69 versus Show-o 0.68) and differences of roughly 0.1-0.5 in gFID are within typical run-to-run noise. To support the SOTA claim, the authors should either release the synthetic dataset, provide a controlled comparison on a shared training set at matched compute, or at least report variance across seeds.
  3. [Sec. 3.2, Figs. 4-5] The diffusion head's conditioning is underspecified at a load-bearing point. During training, the transformer's hidden states are computed from a randomly masked input. At inference, the "final hidden states" after the unmasking schedule are used to condition residual prediction; however, the paper does not state whether an additional unmasked forward pass is performed after all discrete tokens are generated. If the final hidden states are taken from the last masked forward pass, the conditioning distribution differs between train and test, and the diffusion head may be solving a different problem than intended. If an extra unmasked forward pass is used, the reported 12-step discrete sampling count omits one transformer pass and the efficiency/latency numbers in Table 2 change. Please clarify the exact inference procedure and, if relevant, update the efficiency accounting.
minor comments (5)
  1. [Appendix A.5] The word "clarfiy" in the appendix title should be corrected to "clarify".
  2. [Table 4] The table header "Througput" is missing an 'h'; it should be "Throughput".
  3. [Figures 6, 10, and 11] These figures contain garbled legend text that appears to be a leftover from another plot template ("w/Structured Latent Spacew/oStructured Latent Space 32 channels64 channels128 channels16 channels1"). The unrelated legends should be removed so the figures match their captions.
  4. [Sec. 4.1] The paper states that T5-base contains 109M parameters; T5-base usually refers to a 220M-parameter encoder-decoder model. If only the encoder is used as the text encoder, please state this explicitly to avoid confusion.
  5. [Sec. 3.1] The sentence "We adopt the same model architecture [5] as DC-AE [11]" refers to reference [5] (EfficientViT) for the architecture; since the actual architecture is from DC-AE, citing DC-AE directly or explaining how EfficientViT is used would improve clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found: DC-AR's results are direct benchmark measurements, and its self-citations (DC-AE, HART, PixArt-α, VILA) are independently validated artifacts.

full rationale

The paper's headline outputs—gFID 5.49 on MJHQ-30K, GenEval 0.69, throughput 10.3 img/s, latency 0.4 s—are measured benchmark results, not quantities derived from fitted parameters, so the 'fitted input called prediction' pattern does not apply. The coarse-to-fine split in Sec. 3.2 (discrete tokens first, residual tokens Zr = Z − Zq second) is introduced as a design principle: 'residual tokens should exclusively serve a refining function without altering the overall structure.' This is a stated assumption used to justify the architecture, not a result derived from the definition of Zr; the paper validates the design empirically (Table 4: hybrid beats discrete-only; Fig. 9 shows detail improvements), which is evidence of effectiveness, not a circular reduction. The claim that discrete-token-dominated pipelines require few steps is supported by external works (MaskGIT [7], MAR [35]), not by the authors' own citations. The paper cites several same-group works—EfficientViT [5] and DC-AE [11] for the tokenizer backbone, HART [53] for hybrid tokenization, VILA1.5-13B [37] for captioning, PixArt-α [14] for the transformer block—but each of these is an independently validated published artifact (DC-AE on ImageNet reconstruction rFID; PixArt-α on text-to-image benchmarks; VILA on multimodal benchmarks), and the present paper re-evaluates its own tokenizer directly (Tables 1 and 5). Thus the citations are real evidence and do not constitute a load-bearing self-citation chain. The 12-step sampling claim is backed by an explicit step sweep (Fig. 6) in which 12 steps is the empirical optimum, not a number forced by construction. Two weaknesses identified by the skeptic—the untested premise that Zr carries no structural information at 32× compression, and the '#Steps 12' label that omits the 20 residual-diffusion steps—are correctness/fairness concerns about evidence strength and reporting transparency, not instances of a derivation reducing to its own inputs. No equation is defined in terms of its target, no parameter is renamed as a prediction, and no uniqueness theorem is imported from the authors' prior work.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The central machinery rests on standard VQ autoencoding plus a decomposition of latents into discrete and residual parts. The most consequential assumptions are the residual-only-detail hypothesis and the cross-resolution validity of a 32x 2D tokenizer. The free parameters are model and sampling constants chosen by hand or by benchmark grids; they tighten the efficiency claims but do not by themselves make the quality results circular.

free parameters (7)
  • codebook_size = 16384
    Chosen for the VQ tokenizer in Sec 4.1; affects reconstruction fidelity and generation difficulty.
  • latent_channels = 32
    Adopted from the DC-AE-f32c32 architecture; determines the capacity of continuous and residual latents.
  • cfg_scale = 4.5
    Selected sampling hyperparameter in Appendix A.2; no sensitivity analysis is provided.
  • sampling_temperature = 4.5
    Selected sampling hyperparameter in Appendix A.2; affects diversity and quality.
  • discrete_sampling_steps = 12
    Chosen from the grid in Fig 6 (8, 12, 16, 32, 64); the efficiency claim depends on this tuned value.
  • residual_diffusion_steps = 20
    Chosen sampling hyperparameter in Appendix A.2; balances refinement quality and speed.
  • three_stage_training_epochs = 10 / 40 / 10
    Stage durations in Table 7, selected by validation; they affect tokenizer reconstruction quality.
assumptions (4)
  • domain assumption Residual tokens Zr = Z - Zq carry only fine detail and can be modeled by a lightweight MLP diffusion head conditioned on final transformer hidden states without altering image structure.
    Invoked in Sec 3.2 to justify using discrete tokens alone for the transformer forward pass and the 12-step sampling schedule.
  • domain assumption A single 2D tokenizer trained at 256x256 remains valid at 512x512 and produces a shared latent space that enables generator fine-tuning.
    Invoked in Sec 3.2 and Table 6; supported by experiments but not proven by construction.
  • domain assumption The DC-AE architecture, previously validated for continuous latents, remains a good encoder-decoder foundation under VQ discretization at 32x compression.
    Adopted in Sec 3.1 from prior work; the paper adapts it but assumes the architecture transfers to discrete tokens.
  • domain assumption FID and GenEval scores are treated as sufficiently reliable measures of generation quality and prompt alignment.
    Used in Tables 2 and 3 without discussion of the metrics' limitations or variance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DC-AR: Efficient Masked Autoregressive Image Generation with Deep Compression Hybrid Tokenizer." pith.science (2026). https://pith.science/paper/VFSIMW2D

@misc{pith2026250704947,
  author       = {Pith},
  title        = {Pith review of: DC-AR: Efficient Masked Autoregressive Image Generation with Deep Compression Hybrid Tokenizer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VFSIMW2D}},
  note         = {Machine review of arXiv:2507.04947}
}
read the original abstract

We introduce DC-AR, a novel masked autoregressive (AR) text-to-image generation framework that delivers superior image generation quality with exceptional computational efficiency. Due to the tokenizers' limitations, prior masked AR models have lagged behind diffusion models in terms of quality or efficiency. We overcome this limitation by introducing DC-HT - a deep compression hybrid tokenizer for AR models that achieves a 32x spatial compression ratio while maintaining high reconstruction fidelity and cross-resolution generalization ability. Building upon DC-HT, we extend MaskGIT and create a new hybrid masked autoregressive image generation framework that first produces the structural elements through discrete tokens and then applies refinements via residual tokens. DC-AR achieves state-of-the-art results with a gFID of 5.49 on MJHQ-30K and an overall score of 0.69 on GenEval, while offering 1.5-7.9x higher throughput and 2.0-3.5x lower latency compared to prior leading diffusion and autoregressive models.

Figures

Figures reproduced from arXiv: 2507.04947 by the authors.

Figure 1
Figure 1. DC-AR is a cutting-edge masked autoregressive text-to-image generation framework. In comparison to other leading masked autoregressive models and diffusion models, DC-AR delivers 1.5-7.9× higher throughput and 2.0-3.5× lower latency, all while achieving state-of-the-art quality on text-to-image generation benchmarks. Abstract We introduce DC-AR, a novel masked autoregressive (AR) text-to-image generation framework t… view at source ↗
Figure 2
Figure 2. Qualitative Comparison of Text-to-Image Generation Results Between DC-AR and Other Generative Models. 2. Related Work Image Tokenizer. Since directly learning representation and generation in pixel space is computationally expensive and challenging, modern methods employ image tokeniza￾tion to convert images into a latent space. These approaches primarily fall into two categories: strategies based on con￾tinuous lat… view at source ↗
Figure 3
Figure 3. Illustration of Our Three-Stage Adaptation Training Strategy for DC-HT. decent reconstruction performance. Next, we present DC￾AR, an efficient masked autoregressive text-to-image gen￾eration framework built upon DC-HT. 3.1. Deep Compression Hybrid Tokenizer While existing 1D tokenizers used in autoregressive mod￾eling can achieve high compression ratios, they discard the 2D spatial correspondence between pixel patc… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Training Workflow of DC-AR. Our visual tokenizer first decomposes the input image into discrete and residual continuous tokens. As a hybrid generation framework, we design DC-AR to model both types of tokens effectively. We use a cross-entropy loss through the mask-pre…
Figure 5
Figure 5. Figure 5: Inference Workflow of DC-AR. The process begins with a fully masked state, where we progressively predict discrete tokens using an unmasking schedule. Once we generate all discrete tokens, the final hidden states from the transformer are utilized as conditions for the …
Figure 6
Figure 6. Figure 6: Results of gFID under Different Sampling Steps. Our hybrid generation pipeline, dominated by discrete tokens, al￾lows DC-AR to achieve exceptional quality with just 12 sampling steps. In contrast to MAR-based methods, which require numer￾ous steps to attain optimal per…
Figure 7
Figure 7. Figure 7: Additional text-to-image generation results of DC-AR. A.3. Additional Text-to-image Examples [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Additional text-to-image generation results of DC-AR [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Qualitative Comparison: Images Generated by DC-AR vs. the Discrete-Only Baseline. For each pair of images, the left image is produced by the discrete-only baseline, while the right image is generated by DC-AR. A.4. Qualitative Comparison of DC-AR and discrete-only base…
Figure 11
Figure 11. Figure 11: gFID Results on ImageNet 256×256 for MAR-B at Different Sampling Steps. MAR-B requires 64 sampling steps to achieve its best performance, significantly lagging be￾hind our method, which attains optimal performance in just 12 steps. Training Loss Curve: Fine-Tuning vs.…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. HPSv3: Towards Wide-Spectrum Human Preference Score

    cs.CV 2025-08 conditional novelty 6.0 of 10

    HPSv3, trained on the new 1.08M-pair HPDv3 dataset, reaches 76.9% pairwise preference accuracy on its own test set and Spearman 0.94 against human model rankings, and is used to iteratively refine generated images (CoHP).

Reference graph

Works this paper leans on

75 extracted references · 30 canonical work pages · cited by 1 Pith paper

  1. [1]

    Flextok: Re- sampling images into 1d token sequences of flexible length

    Roman Bachmann, Jesse Allardice, David Mizrahi, Enrico Fini, O ˘guzhan Fatih Kar, Elmira Amirloo, Alaaeldin El- Nouby, Amir Zamir, and Afshin Dehghan. Flextok: Re- sampling images into 1d token sequences of flexible length. arXiv preprint arXiv:2502.13967, 2025. 3

    arXiv 2025

  2. [2]

    Meissonic: Revitalizing masked generative trans- formers for efficient high-resolution text-to-image synthesis

    Jinbin Bai, Tian Ye, Wei Chow, Enxin Song, Qing-Guo Chen, Xiangtai Li, Zhen Dong, Lei Zhu, and Shuicheng Yan. Meissonic: Revitalizing masked generative trans- formers for efficient high-resolution text-to-image synthesis. arXiv preprint arXiv:2410.08261, 2024. 2, 3, 7

    arXiv

  3. [3]

    All are worth words: A vit backbone for diffusion models

    Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu. All are worth words: A vit backbone for diffusion models. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 22669–22679, 2023. 3

    work page 2023

  4. [4]

    Flux, 2024

    BlackForest Labs. Flux, 2024. 2

    work page 2024

  5. [5]

    Efficientvit: Lightweight multi-scale attention for high- resolution dense prediction

    Han Cai, Junyan Li, Muyan Hu, Chuang Gan, and Song Han. Efficientvit: Lightweight multi-scale attention for high- resolution dense prediction. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 17302– 17313, 2023. 4

    work page 2023

  6. [6]

    Condition-aware neural network for controlled image generation

    Han Cai, Muyang Li, Qinsheng Zhang, Ming-Yu Liu, and Song Han. Condition-aware neural network for controlled image generation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 7194–7203, 2024. 3

    work page 2024

  7. [7]

    Maskgit: Masked generative image transformer

    Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. Maskgit: Masked generative image transformer. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 11315–11325, 2022. 2, 3, 5

    work page 2022

  8. [8]

    Muse: Text-to-image generation via masked generative transform- ers

    Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jose Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Mur- phy, William T Freeman, Michael Rubinstein, et al. Muse: Text-to-image generation via masked generative transform- ers. arXiv preprint arXiv:2301.00704, 2023. 2, 3

    arXiv 2023

Show all 75 references
  1. [9]

    Softvq-vae: Efficient 1-dimensional con- tinuous tokenizer

    Hao Chen, Ze Wang, Xiang Li, Ximeng Sun, Fangyi Chen, Jiang Liu, Jindong Wang, Bhiksha Raj, Zicheng Liu, and Emad Barsoum. Softvq-vae: Efficient 1-dimensional con- tinuous tokenizer. arXiv preprint arXiv:2412.10958 , 2024. 3

    2024 arXiv

  2. [10]

    Masked autoencoders are effective tokenizers for diffusion models

    Hao Chen, Yujin Han, Fangyi Chen, Xiang Li, Yidong Wang, Jindong Wang, Ze Wang, Zicheng Liu, Difan Zou, and Bhik- sha Raj. Masked autoencoders are effective tokenizers for diffusion models. arXiv preprint arXiv:2502.03444, 2025. 3

    2025 arXiv

  3. [11]

    Deep compression autoencoder for efficient high-resolution diffu- sion models

    Junyu Chen, Han Cai, Junsong Chen, Enze Xie, Shang Yang, Haotian Tang, Muyang Li, Yao Lu, and Song Han. Deep compression autoencoder for efficient high-resolution diffu- sion models. arXiv preprint arXiv:2410.10733, 2024. 2, 3, 4, 6

    2024 arXiv

  4. [12]

    Pixart- σ: Weak-to-strong training of diffusion transformer for 4k text-to-image generation

    Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart- σ: Weak-to-strong training of diffusion transformer for 4k text-to-image generation. In European Conference on Computer Vision , pages 74–91. Spri...

    2024

  5. [13]

    Pixart- {\delta}: Fast and controllable image generation with latent consis- tency models

    Junsong Chen, Yue Wu, Simian Luo, Enze Xie, Sayak Paul, Ping Luo, Hang Zhao, and Zhenguo Li. Pixart- {\delta}: Fast and controllable image generation with latent consis- tency models. arXiv preprint arXiv:2401.05252, 2024

    2024 arXiv

  6. [14]

    Pixart- α: Fast training of diffusion transformer for photorealistic text-to-image synthesis

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Zhongdao Wang, James T Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart- α: Fast training of diffusion transformer for photorealistic text-to-image synthesis. In ICLR, 2024. 3, 6, 7

    2024

  7. [15]

    Maskmamba: A hybrid mamba-transformer model for masked image generation

    Wenchao Chen, Liqiang Niu, Ziyao Lu, Fandong Meng, and Jie Zhou. Maskmamba: A hybrid mamba-transformer model for masked image generation. arXiv preprint arXiv:2409.19937, 2024. 2, 3

    2024 arXiv

  8. [16]

    Janus- pro: Unified multimodal understanding and generation with data and model scaling

    Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus- pro: Unified multimodal understanding and generation with data and model scaling. arXiv preprint arXiv:2501.17811 ,

    arXiv

  9. [17]

    Collaborative decoding makes visual auto-regressive modeling efficient

    Zigeng Chen, Xinyin Ma, Gongfan Fang, and Xinchao Wang. Collaborative decoding makes visual auto-regressive modeling efficient. arXiv preprint arXiv:2411.17787, 2024. 3

    2024 arXiv

  10. [18]

    Vqgan-clip: Open domain image generation and editing with natural language guidance

    Katherine Crowson, Stella Biderman, Daniel Kornis, Dashiell Stander, Eric Hallahan, Louis Castricato, and Ed- ward Raff. Vqgan-clip: Open domain image generation and editing with natural language guidance. In European con- ference on computer vision , pages 88–105. Springer, 2022. 3

    2022

  11. [19]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 2, 6

    2009

  12. [20]

    Bert: Pre-training of deep bidirectional trans- formers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional trans- formers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the asso- ciation for computational linguistics: human l...

    2019

  13. [21]

    Cogview: Mastering text-to-image generation via transformers

    Ming Ding, Zhuoyi Yang, Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, et al. Cogview: Mastering text-to-image generation via transformers. Advances in neural information processing systems, 34:19822–19835, 2021. 3

    2021

  14. [22]

    Cogview2: Faster and better text-to-image generation via hierarchical transformers

    Ming Ding, Wendi Zheng, Wenyi Hong, and Jie Tang. Cogview2: Faster and better text-to-image generation via hierarchical transformers. Advances in Neural Information Processing Systems, 35:16890–16902, 2022. 3

    2022

  15. [23]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021. 3

    2021

  16. [24]

    Scaling recti- fied flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. In Forty-first international conference on mach...

  17. [25]

    Fluid: Scaling autoregressive text-to-image generative models with continuous tokens

    Lijie Fan, Tianhong Li, Siyang Qin, Yuanzhen Li, Chen Sun, Michael Rubinstein, Deqing Sun, Kaiming He, and Yonglong Tian. Fluid: Scaling autoregressive text-to-image generative models with continuous tokens. arXiv preprint arXiv:2410.13863, 2024. 2, 3, 7

    arXiv

  18. [26]

    Make-a-scene: Scene- based text-to-image generation with human priors

    Oran Gafni, Adam Polyak, Oron Ashual, Shelly Sheynin, Devi Parikh, and Yaniv Taigman. Make-a-scene: Scene- based text-to-image generation with human priors. In Eu- ropean Conference on Computer Vision , pages 89–106. Springer, 2022. 3

    2022

  19. [27]

    Geneval: An object-focused framework for evaluating text- to-image alignment

    Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text- to-image alignment. Advances in Neural Information Pro- cessing Systems, 36:52132–52152, 2023. 6

    2023

  20. [28]

    Infinity: Scaling bit- wise autoregressive modeling for high-resolution image syn- thesis

    Jian Han, Jinlai Liu, Yi Jiang, Bin Yan, Yuqi Zhang, Zehuan Yuan, Bingyue Peng, and Xiaobing Liu. Infinity: Scaling bit- wise autoregressive modeling for high-resolution image syn- thesis. arXiv preprint arXiv:2412.04431, 2024. 2, 3

    2024 arXiv

  21. [29]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 6

    2017

  22. [30]

    Lantern: Accelerating visual autoregressive mod- els with relaxed speculative decoding

    Doohyuk Jang, Sihwan Park, June Yong Yang, Yeonsung Jung, Jihun Yun, Souvik Kundu, Sung-Yub Kim, and Eunho Yang. Lantern: Accelerating visual autoregressive mod- els with relaxed speculative decoding. arXiv preprint arXiv:2410.03355, 2024. 2, 3

    arXiv

  23. [31]

    Democra- tizing text-to-image masked generative models with com- pact text-aware one-dimensional tokens

    Dongwon Kim, Ju He, Qihang Yu, Chenglin Yang, Xiao- hui Shen, Suha Kwak, and Liang-Chieh Chen. Democra- tizing text-to-image masked generative models with com- pact text-aware one-dimensional tokens. arXiv preprint arXiv:2501.07730, 2025. 2, 3, 6, 7, 5

    2025 arXiv

  24. [32]

    Videopoet: A large language model for zero-shot video gen- eration

    Dan Kondratyuk, Lijun Yu, Xiuye Gu, Jos ´e Lezama, Jonathan Huang, Grant Schindler, Rachel Hornung, Vigh- nesh Birodkar, Jimmy Yan, Ming-Chang Chiu, et al. Videopoet: A large language model for zero-shot video gen- eration. In ICML, 2024. 2

    2024

  25. [33]

    Playground v2

    Daiqing Li, Aleks Kamko, Ehsan Akhgari, Ali Sabet, Lin- miao Xu, and Suhail Doshi. Playground v2. 5: Three in- sights towards enhancing aesthetic quality in text-to-image generation. arXiv preprint arXiv:2402.17245, 2024. 3, 6, 7

    2024 arXiv

  26. [34]

    Mage: Masked generative encoder to unify representation learning and image synthe- sis

    Tianhong Li, Huiwen Chang, Shlok Mishra, Han Zhang, Dina Katabi, and Dilip Krishnan. Mage: Masked generative encoder to unify representation learning and image synthe- sis. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 2142–2152,

  27. [35]

    Autoregressive image generation without vec- tor quantization

    Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vec- tor quantization. Advances in Neural Information Processing Systems, 37:56424–56445, 2025. 3, 5, 8

    2025

  28. [36]

    Controlvar: Exploring con- trollable visual autoregressive modeling

    Xiang Li, Kai Qiu, Hao Chen, Jason Kuen, Zhe Lin, Rita Singh, and Bhiksha Raj. Controlvar: Exploring con- trollable visual autoregressive modeling. arXiv preprint arXiv:2406.09750, 2024. 3

    2024 arXiv

  29. [37]

    Vila: On pre-training for vi- sual language models

    Ji Lin, Hongxu Yin, Wei Ping, Pavlo Molchanov, Moham- mad Shoeybi, and Song Han. Vila: On pre-training for vi- sual language models. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 26689–26699, 2024. 6

    2024

  30. [38]

    Playground v3: Im- proving text-to-image alignment with deep-fusion large lan- guage models

    Bingchen Liu, Ehsan Akhgari, Alexander Visheratin, Aleks Kamko, Linmiao Xu, Shivam Shrirao, Chase Lambert, Joao Souza, Suhail Doshi, and Daiqing Li. Playground v3: Im- proving text-to-image alignment with deep-fusion large lan- guage models. arXiv preprint arXiv:2409.10695, 2024. 2, 3

    2024 arXiv

  31. [39]

    Lumina-mgpt: Illuminate flexible photorealistic text-to-image generation with multimodal generative pretraining

    Dongyang Liu, Shitian Zhao, Le Zhuo, Weifeng Lin, Yu Qiao, Hongsheng Li, and Peng Gao. Lumina-mgpt: Illuminate flexible photorealistic text-to-image generation with multimodal generative pretraining. arXiv preprint arXiv:2408.02657, 2024. 3

    2024 arXiv

  32. [40]

    World model on million-length video and language with blockwise ringattention

    Hao Liu, Wilson Yan, Matei Zaharia, and Pieter Abbeel. World model on million-length video and language with blockwise ringattention. arXiv preprint arXiv:2402.08268 ,

    arXiv

  33. [41]

    Exploring the role of large language models in prompt encoding for diffusion models

    Bingqi Ma, Zhuofan Zong, Guanglu Song, Hongsheng Li, and Yu Liu. Exploring the role of large language models in prompt encoding for diffusion models. In The Thirty-eighth Annual Conference on Neural Information Processing Sys- tems, 2024. 3

    2024

  34. [42]

    Star: Scale-wise text-to- image generation via auto-regressive representations

    Xiaoxiao Ma, Mohan Zhou, Tao Liang, Yalong Bai, Tiejun Zhao, Huaian Chen, and Yi Jin. Star: Scale-wise text-to- image generation via auto-regressive representations. arXiv preprint arXiv:2406.10797, 2024. 3

    2024 arXiv

  35. [43]

    Hello gpt-4o, 2024

    OpenAI. Hello gpt-4o, 2024. 2

    2024

  36. [44]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 4195–4205,

  37. [45]

    W ¨urstchen: An ef- ficient architecture for large-scale text-to-image diffusion models

    Pablo Pernias, Dominic Rampas, Mats Leon Richter, Christopher Pal, and Marc Aubreville. W ¨urstchen: An ef- ficient architecture for large-scale text-to-image diffusion models. In The Twelfth International Conference on Learn- ing Representations, 2024. 2

    2024

  38. [46]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. In The Twelfth Inter- national Conference on Learning Representations, 2023. 3, 7

    2023

  39. [47]

    Tokenflow: Unified image tokenizer for multimodal understanding and generation

    Liao Qu, Huichao Zhang, Yiheng Liu, Xu Wang, Yi Jiang, Yiming Gao, Hu Ye, Daniel K Du, Zehuan Yuan, and Xinglong Wu. Tokenflow: Unified image tokenizer for multimodal understanding and generation. arXiv preprint arXiv:2412.03069, 2024. 3

    2024 arXiv

  40. [48]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67, 2020. 6

    2020

  41. [49]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International confer- ence on machine learning, pages 8821–8831. Pmlr, 2021. 3

    2021

  42. [50]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 3

    2022

  43. [51]

    Journeydb: A benchmark for generative im- age understanding

    Keqiang Sun, Junting Pan, Yuying Ge, Hao Li, Haodong Duan, Xiaoshi Wu, Renrui Zhang, Aojun Zhou, Zipeng Qin, Yi Wang, et al. Journeydb: A benchmark for generative im- age understanding. Advances in neural information process- ing systems, 36:49659–49678, 2023. 6

    2023

  44. [52]

    Autoregressive model beats diffusion: Llama for scalable image generation

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525, 2024. 3

    2024 arXiv

  45. [53]

    Hart: Efficient visual generation with hybrid au- toregressive transformer

    Haotian Tang, Yecheng Wu, Shang Yang, Enze Xie, Junsong Chen, Junyu Chen, Zhuoyang Zhang, Han Cai, Yao Lu, and Song Han. Hart: Efficient visual generation with hybrid au- toregressive transformer. arXiv preprint arXiv:2410.10812,

    arXiv

  46. [54]

    Introducing auraflow v0

    Auraflow Team. Introducing auraflow v0. 1, an open explo- ration of large rectified flow models, 2024.URL https://blog. fal. ai/auraflow, 4, 2024. 3

    2024

  47. [55]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023. 2

    2023 arXiv

  48. [56]

    Kolors: Effective training of diffusion model for photorealistic text-to-image synthesis

    K Team. Kolors: Effective training of diffusion model for photorealistic text-to-image synthesis. arXiv preprint, 2024. 3

    2024

  49. [57]

    Visual autoregressive modeling: Scalable image generation via next-scale prediction

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Li- wei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. Advances in neural in- formation processing systems, 37:84839–84865, 2025. 3

    2025

  50. [58]

    Neural discrete representation learning

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information pro- cessing systems, 30, 2017. 3

    2017

  51. [59]

    Phenaki: Variable length video generation from open domain textual description

    Ruben Villegas, Mohammad Babaeizadeh, Pieter-Jan Kin- dermans, Hernan Moraldo, Han Zhang, Mohammad Taghi Saffar, Santiago Castro, Julius Kunze, and Dumitru Erhan. Phenaki: Variable length video generation from open domain textual description. arXiv preprint arXiv:2210.02399, 2...

    arXiv

  52. [60]

    Emu3: Next-token prediction is all you need

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869, 2024. 2, 3

    2024 arXiv

  53. [61]

    Parallelized autoregressive visual generation

    Yuqing Wang, Shuhuai Ren, Zhijie Lin, Yujin Han, Haoyuan Guo, Zhenheng Yang, Difan Zou, Jiashi Feng, and Xihui Liu. Parallelized autoregressive visual generation. arXiv preprint arXiv:2412.15119, 2024. 3

    2024 arXiv

  54. [62]

    Maskbit: Embedding-free image generation via bit tokens

    Mark Weber, Lijun Yu, Qihang Yu, Xueqing Deng, Xiao- hui Shen, Daniel Cremers, and Liang-Chieh Chen. Maskbit: Embedding-free image generation via bit tokens. Transac- tions on Machine Learning Research, 2024. 2

    2024

  55. [63]

    Janus: Decoupling visual encoding for unified multimodal understanding and generation

    Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, et al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. arXiv preprint arXiv:2410.13848, 2024. 2

    arXiv

  56. [64]

    Vila-u: a unified foundation model inte- grating visual understanding and generation

    Yecheng Wu, Zhuoyang Zhang, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu Yin, Li Yi, et al. Vila-u: a unified foundation model inte- grating visual understanding and generation. arXiv preprint arXiv:2409.04429, 2024. 2, 3

    2024 arXiv

  57. [65]

    Sana: Efficient high-resolution image syn- thesis with linear diffusion transformers

    Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, et al. Sana: Efficient high-resolution image syn- thesis with linear diffusion transformers. arXiv preprint arXiv:2410.10629, 2024. 2, 3, 7

    arXiv

  58. [66]

    Sana 1.5: Efficient scaling of training-time and inference-time compute in linear diffusion transformer

    Enze Xie, Junsong Chen, Yuyang Zhao, Jincheng Yu, Ligeng Zhu, Yujun Lin, Zhekai Zhang, Muyang Li, Junyu Chen, Han Cai, et al. Sana 1.5: Efficient scaling of training-time and inference-time compute in linear diffusion transformer. arXiv preprint arXiv:2501.18427, 2025. 3

    2025 arXiv

  59. [67]

    Show-o: One single transformer to unify multimodal understanding and generation

    Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408.12528 , 2024. 2, 3, 7

    2024 arXiv

  60. [68]

    Car: Controllable autoregressive modeling for visual generation

    Ziyu Yao, Jialin Li, Yifeng Zhou, Yong Liu, Xi Jiang, Chengjie Wang, Feng Zheng, Yuexian Zou, and Lei Li. Car: Controllable autoregressive modeling for visual generation. arXiv preprint arXiv:2410.04671, 2024. 3

    2024 arXiv

  61. [69]

    Scaling autoregres- sive models for content-rich text-to-image generation.Trans

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gun- jan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yin- fei Yang, Burcu Karagol Ayan, et al. Scaling autoregres- sive models for content-rich text-to-image generation.Trans. Mach. Learn. Res., 2022. 2, 3

    2022

  62. [70]

    Magvit: Masked generative video transformer

    Lijun Yu, Yong Cheng, Kihyuk Sohn, Jos ´e Lezama, Han Zhang, Huiwen Chang, Alexander G Hauptmann, Ming- Hsuan Yang, Yuan Hao, Irfan Essa, et al. Magvit: Masked generative video transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p...

    2023

  63. [71]

    An image is worth 32 tokens for reconstruction and generation

    Qihang Yu, Mark Weber, Xueqing Deng, Xiaohui Shen, Daniel Cremers, and Liang-Chieh Chen. An image is worth 32 tokens for reconstruction and generation. Advances in Neural Information Processing Systems , 37:128940– 128966, 2025. 2, 3, 6

    2025

  64. [72]

    Shield- gemma: Generative ai content moderation based on gemma

    Wenjun Zeng, Yuchi Liu, Ryan Mullins, Ludovic Peran, Joe Fernandez, Hamza Harkous, Karthik Narasimhan, Drew Proud, Piyush Kumar, Bhaktipriya Radharapu, et al. Shield- gemma: Generative ai content moderation based on gemma. arXiv preprint arXiv:2407.21772, 2024. 1

    2024 arXiv

  65. [73]

    Language- guided image tokenization for generation

    Kaiwen Zha, Lijun Yu, Alireza Fathi, David A Ross, Cordelia Schmid, Dina Katabi, and Xiuye Gu. Language- guided image tokenization for generation. arXiv preprint arXiv:2412.05796, 2024. 3, 6

    2024 arXiv

  66. [74]

    Var-clip: Text-to-image gen- erator with visual auto-regressive modeling

    Qian Zhang, Xiangzi Dai, Ninghua Yang, Xiang An, Ziy- ong Feng, and Xingyu Ren. Var-clip: Text-to-image gen- erator with visual auto-regressive modeling. arXiv preprint arXiv:2408.01181, 2024. 3

    2024 arXiv

  67. [75]

    A red heart

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 6 DC-AR: Efficient M...

    2018

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.