Pith. sign in

REVIEW 4 major objections 5 minor 10 cited by

RandAR: Decoder-only Autoregressive Visual Generation in Random Orders

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A decoder-only transformer can generate images in arbitrary token orders without loss of quality, if each image token is preceded by a position instruction token telling it where in the image it lives.

desk verdict Solid empirical design with real applications, but novelty is overstated: RAR already did random-order decoder-only generation, and the paper needs to confront that overlap. read the letter →

arxiv 2412.01827 v2 pith:TRQ4K3GN submitted 2024-12-02 cs.CV cs.AI

Ziqi Pang , Tianyuan Zhang , Fujun Luan , Yunze Man , Hao Tan , Kai Zhang , William T. Freeman , Yu-Xiong Wang This is my paper · ORCID
classification cs.CVcs.AI
keywords random-orderautoregressivegenerationdecoder-onlytransformerpositioninstructiontokenimageparalleldecodingzero-shotinpaintingoutpaintingresolutionextrapolation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a GPT-style decoder-only image generator really needs the fixed raster order (top-left to bottom-right) that nearly all such models use. It claims the answer is no: inserting a small position instruction token before each image token lets the same architecture be trained to predict tokens in any order, and the resulting model matches its raster-order counterpart on ImageNet 256x256 generation (RandAR-XL FID 2.25 vs 2.16). The payoff is a set of zero-shot abilities that fixed-order models lack, including parallel decoding for roughly 2.5x faster sampling, inpainting, outpainting, resolution extrapolation to 512x512, and bi-directional feature extraction. The paper positions this as evidence that the order bias is not a necessary cost of decoder-only generation, and that plain next-token prediction can model 2D images with bidirectional context.

What carries the argument

The position instruction token is the load-bearing object: a single shared learnable embedding rotated by 2D rotary position embeddings (2D-RoPE) to the coordinates $(h_i, w_i)$ of the next image token, and inserted immediately before that token in the shuffled sequence. It tells the causal transformer where in the image the token being predicted lives, so the model can be trained on fully random permutations of the 256 tokens instead of a fixed order. Because the context and target positions are arbitrary, the same mechanism directly supports parallel decoding (several position instructions appended at once), inpainting and outpainting (any visible tokens as context, any target positions to fill), and two-pass bidirectional encoding (run the sequence twice and read the second-pass features).

What would settle it

Evaluate a trained RandAR model on inference orders deliberately constructed to be far from the uniform random-order training distribution (for example, fixed adversarial permutations, or orders whose first tokens all come from one corner of the image) and compare FID with the reported random-order FID of 2.25.

Watch

Extended reading notes

Core claim

The paper's central claim is that random-order next-token prediction, not a predefined raster sequence, is what makes decoder-only visual autoregressive models rigid, and that removing the order constraint is possible without sacrificing generation quality. RandAR does this by training a standard causal transformer on randomly permuted image-token sequences with a position instruction token before each token, making every prediction condition on an arbitrary subset of the image. On ImageNet 256x256, RandAR-XL reaches FID 2.25 with parallel decoding in 88 steps, essentially matching its raster-order counterpart's 2.16, and then reuses the same weights for zero-shot inpainting, outpainting, high-resolution synthesis, and bidirectional encoding. The paper interprets these results as showing that a plain decoder-only transformer can acquire the bi-directional context modeling that previously required masked encoder-decoder designs.

Load-bearing premise

The load-bearing premise is that training on only about $3\times10^8$ of the roughly $8\times10^{506}$ possible $256$-token orders teaches the model to generate well on arbitrary unseen orders at inference.

Editorial extensions

If this is right

  • RandAR reaches FID 2.25 at 88 steps with parallel decoding, essentially matching its raster-order counterpart's 2.16, so random-order training costs little generation quality.
  • Parallel decoding cuts latency roughly 2.5x (from 16.8s to 6.6s for a 1.4B model on A100), because the model can predict multiple tokens in one forward pass.
  • The same 256x256-trained model performs zero-shot inpainting, outpainting to 256x1024, and resolution extrapolation to 512x512 without fine-tuning.
  • Passing the token sequence through the causal transformer twice yields bi-directional features that improve SPair-71k semantic correspondence (PCK 31.3 vs 24.5 for the raster-order model) and ImageNet linear probing.
  • Inference-order ablations show fully random orders outperform fixed orders (random FID 2.25 vs raster FID 4.82), indicating the model learns to exploit broad context rather than memorize a sequence.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the permutation-generalization assumption holds, the same position-instruction mechanism could be dropped into existing decoder-only multimodal models that currently hard-code raster order for images, giving them the same zero-shot editing and parallel-decoding abilities without retraining from scratch.
  • The two-pass bidirectional encoding result suggests random-order decoder-only transformers could serve as generative vision encoders, but the paper only evaluates linear probing and semantic correspondence; broader transfer tasks such as detection and segmentation remain untested.
  • A direct test of the mechanism would be to train RandAR on a restricted family of permutations (for example, only hierarchical or locality-preserving orders) and measure how much of the zero-shot capability disappears; the paper does not vary permutation coverage during training.
  • The resolution-extrapolation recipe of generating even coordinates first and then filling details with NTK-style high-frequency RoPE looks like a general coordinate-sampling schedule that might transfer to arbitrary aspect ratios or video frames, though the paper demonstrates it only for square 256-to-512 synthesis.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. RandAR trains a decoder-only LLaMAGen-style transformer on randomly permuted VQGAN token sequences, inserting a shared 'position instruction token' (rotated by 2D-RoPE) before each image token and supervising prediction of that image token from the instruction token. On ImageNet 256x256, RandAR-XL reaches FID 2.25 at 88 parallel-decoding steps versus 2.16 for an identically trained raster-order counterpart at 256 steps, which supports the paper's 'comparable quality' claim. The paper further reports 2.5x latency reduction through parallel decoding and demonstrates zero-shot inpainting, outpainting, 512x512 resolution extrapolation, and second-pass bi-directional feature encoding. It includes ablations of the position-instruction design (Table 2), inference-time orders (Table 3), and an honest Limitations section noting that random-order learning is less data-efficient than fixed-order training.

Significance. If the claims hold, RandAR is a conceptually useful step: it gives decoder-only autoregressive image models the flexibility usually associated with masked or bi-directional models while retaining plain next-token training and KV-cache compatibility. The central empirical comparison is well designed: the raster-order counterpart is trained with the same architecture and setup, FID-50K follows the standard ADM protocol, and latency is measured on the same hardware. The paper also provides pseudo-code for training and parallel decoding, and several negative results are reported honestly (e.g., the raster-order model collapses under the second-pass encoding in Table 5, and the 'Merge' position design hurts parallel decoding in Table 2). The main reservations are that the parallel-decoding procedure is described as exact when it is in fact an approximation, the 'arbitrary order' claim rests on generalization from a tiny fraction of the permutation space without direct analysis, and some headline zero-shot applications are evaluated only qualitatively. The work is worth publishing after these points are addressed.

major comments (4)
  1. [Sec. 3.3, Eq. (6)-(7), Algorithm B] Parallel decoding introduces a train/inference mismatch that the paper does not acknowledge. In training (Eq. (4)) every image token is immediately preceded by its own position instruction token, and consecutive position instruction tokens never occur. In Eq. (6), however, the model is given the context [..., P_n, P_{n+1}] and asked to predict x_n from a history that already contains the future instruction token P_{n+1}. The rearrangement in Eq. (7) fixes the final sequence format but does not change the conditioning under which x_n was sampled. Thus the parallel procedure is not an exact sampler of the Eq. (4) distribution; the small FID gap in Fig. 4(b) and Table 1 shows that the approximation is empirically benign, but the text should say so explicitly and, ideally, compare one-token-at-a-time decoding with the parallel schedule at equal step budgets to quantify the approximation error.
  2. [Sec. 3.2 'Training'; Table 3] The paper acknowledges that 300 epochs cover at most 3x10^8 of roughly 8x10^506 possible 256-token permutations. All of the zero-shot claims (inpainting, outpainting, resolution extrapolation, second-pass encoding) involve conditioning patterns and sequence lengths that are extremely unlikely to have appeared in training, so the claim of 'arbitrary orders' needs direct support. Table 3 tests a handful of hand-picked inference orders, but it does not relate performance to permutation coverage, order statistics, or the distance between an inference order and the training distribution. I would like to see either an experiment that varies the number or type of permutations seen during training and measures FID and downstream zero-shot quality, or an explicit caveat that 'arbitrary' means 'orders statistically similar to the uniform random training distribution.' Without one of these, the zero-shot framing overreaches.
  3. [Sec. 4.4.3, Fig. 6, Sec. G.3] Resolution extrapolation is presented as a headline capability, but the only evidence is qualitative. The paper itself concedes in Sec. 4.4.3 that the model struggles with intricate structures and small high-frequency patterns. Since the claim is 'zero-shot generalization to 512x512', some quantitative measure is needed before this contribution can be assessed: for example, FID of the 512x512 outputs against appropriately scaled real data, a distribution-distance metric, or a perceptual/user study relative to the 256x256 baseline. As written, the reader cannot tell how often the extrapolation succeeds or how severe the observed failures are.
  4. [Table 1 caption; Related Work [58]] The caption of Table 1 states that RandAR is 'the only decoder-only method capable of generating images in random token orders,' yet the same table and related work cite RAR [58], whose title is 'Randomized Autoregressive Visual Generation.' The paper should clarify the precise technical difference between RandAR and RAR (is it the position-instruction-token mechanism, the use of full random permutations, or something else?) and soften the uniqueness claim if RAR already performs random-order decoding. As written, the novelty claim is not supported by the paper's own references.
minor comments (5)
  1. [Sec. 3.2, Eq. (2)-(3)] The notation is inconsistent: Eq. (2) drops the last image token, but Eq. (3) ends with P_N without the corresponding x_N, while Algorithm A and Eq. (4) use all N tokens. Please fix the sequence definition so that the indices and sequence lengths match the training code.
  2. [Table 1 and Fig. 5] The word 'casual' appears in the 'Casual AR' table headings and in 'full casual attention' (Sec. 4.4.2, Fig. 5); this should be 'causal' throughout.
  3. [Supplementary, Eq. (A)] The formula for spatial contextual guidance contains an extra unmatched parenthesis in the term -(e_theta(x^phi_{1:n}, c)); please correct the typesetting.
  4. [Sec. 3.3 and Algorithm B] The description of the KV-cache rewrite is terse and relies on an example inside a code comment. A short formal invariant stating that the effective KV-cache always matches the training-time interleaved format would help readers verify the correctness of the cache manipulation.
  5. [Sec. 4.1] The text says '300 epochs ... 360K iterations' with batch size 1024; for ImageNet this is approximately 375K iterations, so the stated number should be reconciled with the actual training schedule.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: RandAR is an empirical system validated against external benchmarks and measured inference-time procedures, with no fitted quantity masquerading as a prediction.

full rationale

The paper's central claims are empirical rather than derivational: random-order training quality is measured by ImageNet FID/IS against a same-setup raster-order counterpart (Table 1), acceleration is measured by A100 latency (Table 4), and zero-shot capabilities are evaluated on SPair71k, linear probing, and visual extrapolation tasks. No core result is obtained by fitting a parameter to the quantity it later 'predicts'; CFG weights, cosine schedules, and SCG weights are standard model-selection choices and are ablated or swept rather than loaded into the claim as predictions. The self-citations that exist are not load-bearing: MaskGIT [5] (an author-overlapping citation) is used only as an external comparison point and a source of the cosine sampling schedule, and [34] (same first/last authors) is cited for the general observation that different transformer layers encode different feature levels, with the paper's own Table C ablation selecting the layer. The paper's Limitations section and Sec. 4.4.3 openly acknowledge that random-order training gives only comparable, not superior, quality and that zero-shot resolution extrapolation struggles on high-frequency details; these are honest empirical limitations, not circular steps. The parallel-decoding procedure (Sec. 3.3) conditions on position-instruction tokens not seen in exactly that configuration during teacher forcing, so it is an approximate inference scheme, but the paper validates it with FID and latency measurements rather than asserting it as an exact distribution by construction. Overall, no predicted quantity reduces to an input by definition, and no load-bearing argument is carried solely by self-citation.

Assumptions & free parameters 6 free parameters · 5 assumptions · 1 invented entities

The central claims rest on the inherited VQGAN tokenizer, on empirical generalization from a tiny sample of the 256! permutation space, and on the transfer of RoPE as a global position signal. The position instruction token is the paper's invented mechanism, validated only by the paper's own experiments.

free parameters (6)
  • CFG weight = swept with step size 0.1 per model
    Classifier-free guidance weight is tuned per model and per experiment; reported FID depends on this choice (Supplementary Sec. B).
  • Parallel decoding step schedule = cosine schedule, default 88 steps for 256 tokens
    The number of decoding steps and the cosine schedule are hand-picked; FID varies with step count (Fig. 4b, Table 4).
  • SCG weight w_scg = 2.5 for resolution extrapolation, 1.2 for 256x256 sFID
    Spatial Contextual Guidance strength is chosen per use case (Supplementary Sec. F.2).
  • SCG token dropout rate = 25%
    Random token dropout for the secondary sequence is set by hand (Supplementary Sec. F.1).
  • RoPE base = 10000
    Standard LLaMA RoPE base reused; affects position instruction token encoding (Sec. 3.2, Algorithm A).
  • Position instruction shared embedding = learned on ImageNet
    The single trainable embedding e is learned during training; it is the method's added parameter (Sec. 3.2).
assumptions (5)
  • domain assumption The VQGAN tokenizer from LLaMAGen provides adequate discrete tokens for generation.
    RandAR inherits the tokenizer and its 16x16 grid, so its 256 token positions and quality ceiling depend on this external component (Sec. 4.1).
  • ad hoc to paper A model trained on a tiny fraction of all permutations generalizes to arbitrary unseen orders.
    Sec. 3.2 Training: at most 3x10^8 orders seen vs 8x10^506 possible; the entire zero-shot flexibility claim rests on this empirical generalization.
  • ad hoc to paper RoPE, a relative positional scheme, works as a global position embedding when applied to the shared instruction embedding.
    Sec. 3.2: 'We empirically find that it also works well as a global position embedding.'
  • ad hoc to paper Parallel decoding with two position tokens reproduces the training-time conditioning well enough to preserve quality.
    Sec. 3.3: the second token in a pair is predicted without seeing the first sampled token, a pattern absent from the interleaved training format; quality held only empirically (Fig. 4b).
  • domain assumption RoPE interpolation and high-frequency replacement transfer from language models to 2D image positions.
    Sec. 3.4.3 uses interpolated RoPE and NTK-style high-frequency components for 512x512 extrapolation.
invented entities (1)
  • Position instruction token
    purpose: Informs the decoder of the spatial location of the next image token to be predicted, enabling random-order generation.
    A new token type whose validity is demonstrated only by the paper's own experiments; no external falsifiable prediction (Sec. 3.2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of RandAR: Decoder-only Autoregressive Visual Generation in Random Orders." pith.science (2026). https://pith.science/paper/TRQ4K3GN

@misc{pith2026241201827,
  author       = {Pith},
  title        = {Pith review of: RandAR: Decoder-only Autoregressive Visual Generation in Random Orders},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TRQ4K3GN}},
  note         = {Machine review of arXiv:2412.01827}
}
read the original abstract

We introduce RandAR, a decoder-only visual autoregressive (AR) model capable of generating images in arbitrary token orders. Unlike previous decoder-only AR models that rely on a predefined generation order, RandAR removes this inductive bias, unlocking new capabilities in decoder-only generation. Our essential design enables random order by inserting a "position instruction token" before each image token to be predicted, representing the spatial location of the next image token. Trained on randomly permuted token sequences -- a more challenging task than fixed-order generation, RandAR achieves comparable performance to its conventional raster-order counterpart. More importantly, decoder-only transformers trained from random orders acquire new capabilities. For the efficiency bottleneck of AR models, RandAR adopts parallel decoding with KV-Cache at inference time, enjoying 2.5x acceleration without sacrificing generation quality. Additionally, RandAR supports inpainting, outpainting and resolution extrapolation in a zero-shot manner. We hope RandAR inspires new directions for decoder-only visual generation models and broadens their applications across diverse scenarios. Our project page is at https://rand-ar.github.io/.

Figures

Figures reproduced from arXiv: 2412.01827 by the authors.

Figure 1
Figure 1. Our RandAR enables GPT-style causal decoder-only transformers to generate images via random-order next-token prediction, which entirely removes the raster-order sequencing inductive bias of previous decoder-only models. RandAR not only (a) generates images of comparable quality, but also shows multiple zero-shot capabilities, including (b) parallel decoding for acceleration, (c) inpainting, (d) outpainting, and (e) … view at source ↗
Figure 2
Figure 2. Overview of our RandAR. (a) Conventional autoregressive generation typically enforces a fixed order, e.g., raster order, allowing the model to memorize token orders. (b) Our RandAR enables random order generation by inserting a position instruction token before each image token to be predicted. This design seamlessly integrates with the next-token prediction framework using decoder-only transformer. decoder-only mod… view at source ↗
Figure 3
Figure 3. Zero-shot capabilities of RandAR. (a) RandAR directly enables parallel decoding to accelerate AR generation (Sec. 3.3). (b) Without the order constraint, our RandAR can support inpainting (Sec. 3.4.1) and outpainting (Sec. 3.4.2). (c) RandAR trained on 256×256 can also zero-shot generalize to synthesize 512×512 higher-resolution images with customized order (Sec. 3.4.3). (d) The random order training enables to extr… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: (a) FID-50K over training iterations for RandAR in three different sizes. (b) Effect of inference steps on FID-50K for Ran￾dAR and the raster-order counterpart (775M models). ply parallel decoding to both RandAR and a raster-order model trained under the same setup, re…
Figure 6
Figure 6. Figure 6: RandAR, trained on 256×256 images, can generate 512×512 images with finer details in zero-shot (Sec 3.4.3). This is achieved using a hierarchical decoding order, benefiting the uni￾fied layout, and SCG (Spatial Context Guidance) enhancing the visual quality by refining…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. D-AR: Diffusion via Autoregressive Models

    cs.CV 2025-05 conditional novelty 7.0 of 10

    D-AR recasts pixel-space diffusion as vanilla autoregressive next-token prediction using a diffusion-ordered discrete tokenizer, reaching 2.09 FID on ImageNet 256x256 with a 775M Llama backbone.

  2. Next-Dense-Stride Prediction for Multimodal Autoregressive Visual Modeling

    eess.IV 2026-07 conditional novelty 6.5 of 10

    Next-dense-stride prediction enables coarse-to-fine autoregressive image generation on a single-scale grid and unifies multi-contrast MRI translation, generation, and segmentation in one model.

  3. Progressive Checkerboards for Autoregressive Multiscale Image Generation

    cs.CV 2026-02 conditional novelty 6.0 of 10

    A balanced multiscale checkerboard sampling order for autoregressive image generation allows large scale-up factors without quality loss, because only the total number of serial steps matters.

  4. Scalable Training for Vector-Quantized Networks with 100% Codebook Utilization

    cs.CV 2025-09 conditional novelty 6.0 of 10

    A training-only ViT-based projector, VQBridge, combined with learning annealing, achieves full codebook utilization in vector-quantized networks at large codebook sizes, improving reconstruction and autoregressive ima...

  5. NeoBabel: A Multilingual Open Tower for Visual Generation

    cs.CL 2025-07 conditional novelty 6.0 of 10

    A 2B multilingual text-to-image model trained on 124M translated pairs matches or beats larger English-only baselines on English while scoring higher on the authors' multilingual benchmark extensions.

  6. Rethinking Discrete Tokens: Treating Them as Conditions for Continuous Autoregressive Image Synthesis

    cs.CV 2025-07 conditional novelty 6.0 of 10

    DisCon treats discrete image tokens as conditioning signals rather than targets, letting a continuous autoregressive model refine details and reach gFID 1.38 on ImageNet-256.

  7. Multi-Token Prediction Needs Registers

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Interleaved register tokens with offset-encoded position ids add a training-only multi-token prediction objective that improves fine-tuning, PEFT, and image-generation pretraining over next-token baselines.

  8. Next Patch Prediction for Autoregressive Visual Generation

    cs.CV 2024-12 conditional novelty 6.0 of 10

    Averaging neighboring image tokens into patches during training lets autoregressive image models train faster and generate higher-quality images, with inference unchanged.

  9. Parallelized Autoregressive Visual Generation

    cs.CV 2024-12 conditional novelty 6.0 of 10

    Grouping spatially distant visual tokens into parallel prediction steps reduces autoregressive generation steps by 3.9x to 11.3x with modest FID/FVD loss.

  10. Any-Order GPT as Masked Diffusion Model: Decoupling Formulation and Architecture

    cs.LG 2025-06 conditional novelty 5.0 of 10

    A decoder-only masked diffusion model (AO-GPT) enables a fairer comparison of autoregressive and diffusion language models, showing that uniform random token order is suboptimal and that decoder-only diffusion offers ...

Reference graph

Works this paper leans on

64 extracted references · 30 canonical work pages · cited by 10 Pith papers

  1. [58]

    Randomized autoregressive visual generation

    Qihang Yu, Ju He, Xueqing Deng, Xiaohui Shen, and Liang- Chieh Chen. Randomized autoregressive visual generation. arXiv preprint arXiv:2411.00776, 2024. 1, 2, 6

    arXiv 2024

  2. [1]

    Large scale gan training for high fidelity natural image synthesis

    Andrew Brock. Large scale gan training for high fidelity natural image synthesis. arXiv preprint arXiv:1809.11096 ,

    arXiv

  3. [2]

    In- structPix2Pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. In- structPix2Pix: Learning to follow image editing instructions. In CVPR, 2023. 12

    work page 2023

  4. [3]

    Language models are few-shot learners

    Tom B Brown. Language models are few-shot learners. arXiv preprint arXiv:2005.14165, 2020. 2, 3

    arXiv 2005

  5. [4]

    Medusa: Simple llm inference acceleration framework with multiple decod- ing heads

    Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Ja- son D Lee, Deming Chen, and Tri Dao. Medusa: Simple llm inference acceleration framework with multiple decod- ing heads. In ICML, 2024. 4

    work page 2024

  6. [5]

    Maskgit: Masked generative image transformer

    Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. Maskgit: Masked generative image transformer. In CVPR, 2022. 2, 4, 6, 7, 8

    work page 2022

  7. [6]

    Muse: Text- to-image generation via masked generative transformers

    Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jos´e Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Murphy, William T Freeman, Michael Rubinstein, et al. Muse: Text- to-image generation via masked generative transformers. In ICML, 2023. 2

    work page 2023

  8. [7]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009. 2, 4, 6, 9

    work page 2009

Show all 64 references
  1. [8]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In NeurIPS, 2021. 6, 7

    2021

  2. [9]

    Patched denoising diffusion models for high-resolution im- age synthesis

    Zheng Ding, Mengqi Zhang, Jiajun Wu, and Zhuowen Tu. Patched denoising diffusion models for high-resolution im- age synthesis. In ICLR, 2023. 8

    2023

  3. [10]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In CVPR,

  4. [11]

    Fluid: Scaling autoregressive text-to-image generative models with continuous tokens

    Lijie Fan, Tianhong Li, Siyang Qin, Yuanzhen Li, Chen Sun, Michael Rubinstein, Deqing Sun, Kaiming He, and Yonglong Tian. Fluid: Scaling autoregressive text-to-image generative models with continuous tokens. arXiv preprint arXiv:2410.13863, 2024. 2

    arXiv

  5. [12]

    Data engineer- ing for scaling language models to 128k context

    Yao Fu, Rameswar Panda, Xinyao Niu, Xiang Yue, Han- naneh Hajishirzi, Yoon Kim, and Hao Peng. Data engineer- ing for scaling language models to 128k context. In ICML,

  6. [13]

    Masked diffusion transformer is a strong image synthesizer

    Shanghua Gao, Pan Zhou, Ming-Ming Cheng, and Shuicheng Yan. Masked diffusion transformer is a strong image synthesizer. In ICCV, 2023. 2

    2023

  7. [14]

    Olmo: Ac- celerating the science of language models

    Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, et al. Olmo: Ac- celerating the science of language models. arXiv preprint arXiv:2402.00838, 2024. 2

    2024 arXiv

  8. [15]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. In CVPR, 2022. 9

    2022

  9. [16]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. In NeurIPS, 2017. 7

    2017

  10. [17]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 5, 6, 7

    2022 arXiv

  11. [18]

    Scaling up gans for text-to-image synthesis

    Minguk Kang, Jun-Yan Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, and Taesung Park. Scaling up gans for text-to-image synthesis. In CVPR, 2023. 6

    2023

  12. [19]

    BERT: Pre-training of deep bidirectional trans- formers for language understanding

    Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. BERT: Pre-training of deep bidirectional trans- formers for language understanding. In NAACL, 2019. 2

    2019

  13. [20]

    SqueezeLLM: Dense-and-sparse quantization

    Sehoon Kim, Coleman Richard Charles Hooper, Amir Gho- lami, Zhen Dong, Xiuyu Li, Sheng Shen, Michael W Ma- honey, and Kurt Keutzer. SqueezeLLM: Dense-and-sparse quantization. In ICML, 2024. 4

    2024

  14. [21]

    Adam: A method for stochastic opti- mization

    Diederik P Kingma. Adam: A method for stochastic opti- mization. arXiv preprint arXiv:1412.6980, 2014. 6

    2014 arXiv

  15. [22]

    Improved precision and recall met- ric for assessing generative models

    Tuomas Kynk ¨a¨anniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall met- ric for assessing generative models. In NeurIPS, 2019. 7

    2019

  16. [23]

    Autoregressive image generation using residual quantization

    Doyup Lee, Chiheon Kim, Saehoon Kim, Minsu Cho, and Wook-Shin Han. Autoregressive image generation using residual quantization. In CVPR, 2022. 2, 6

    2022

  17. [24]

    Mage: Masked generative encoder to unify representation learning and image synthe- sis

    Tianhong Li, Huiwen Chang, Shlok Mishra, Han Zhang, Dina Katabi, and Dilip Krishnan. Mage: Masked generative encoder to unify representation learning and image synthe- sis. In CVPR, 2023. 2

    2023

  18. [25]

    Autoregressive image generation without vec- tor quantization

    Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vec- tor quantization. In NeurIPS, 2024. 2, 3, 4, 6, 7, 8

    2024

  19. [26]

    Solving masked jigsaw puzzles with diffusion vision transformers

    Jinyang Liu, Wondmgezahu Teshome, Sandesh Ghimire, Mario Sznaier, and Octavia Camps. Solving masked jigsaw puzzles with diffusion vision transformers. In CVPR, 2024. 2

    2024

  20. [27]

    Compositional visual generation with composable diffusion models

    Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum. Compositional visual generation with composable diffusion models. In ECCV, 2022. 12

    2022

  21. [28]

    Customize your visual autoregressive recipe with set autoregressive modeling

    Wenze Liu, Le Zhuo, Yi Xin, Sheng Xia, Peng Gao, and Xiangyu Yue. Customize your visual autoregressive recipe with set autoregressive modeling. arXiv preprint arXiv:2410.10511, 2024. 2, 6

    2024 arXiv

  22. [29]

    Roberta: A robustly optimized bert pretraining approach

    Yinhan Liu. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 364, 2019. 2

    1907 arXiv

  23. [30]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019. 6

    2019

  24. [31]

    Open-magvit2: An open-source project toward democratizing auto-regressive visual gener- ation

    Zhuoyan Luo, Fengyuan Shi, Yixiao Ge, Yujiu Yang, Limin Wang, and Ying Shan. Open-magvit2: An open-source project toward democratizing auto-regressive visual gener- ation. arXiv preprint arXiv:2409.04410, 2024. 2, 6

    2024 arXiv

  25. [32]

    SIT: Explor- ing flow and diffusion-based generative models with scalable interpolant transformers

    Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijnden, and Saining Xie. SIT: Explor- ing flow and diffusion-based generative models with scalable interpolant transformers. arXiv preprint arXiv:2401.08740,

    arXiv

  26. [33]

    Spair-71k: A large-scale benchmark for semantic correspon- dence

    Juhong Min, Jongmin Lee, Jean Ponce, and Minsu Cho. Spair-71k: A large-scale benchmark for semantic correspon- dence. arXiv preprint arXiv:1908.10543, 2019. 9, 11, 12

    1908 arXiv

  27. [34]

    Frozen transformers in language models are effective visual encoder layers

    Ziqi Pang, Ziyang Xie, Yunze Man, and Yu-Xiong Wang. Frozen transformers in language models are effective visual encoder layers. In ICLR, 2024. 11 18

    2024

  28. [35]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, 2023. 6

    2023

  29. [36]

    Improving language understanding with unsuper- vised learning

    Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding with unsuper- vised learning. Technical report, OpenAI, 2018. 2

    2018

  30. [37]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67, 2020. 2

    2020

  31. [38]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In ICML, 2021. 2

    2021

  32. [39]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, 2022. 6

    2022

  33. [40]

    Improved techniques for training gans

    Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. In NeurIPS, 2016. 7

    2016

  34. [41]

    Stylegan- xl: Scaling stylegan to large diverse datasets

    Axel Sauer, Katja Schwarz, and Andreas Geiger. Stylegan- xl: Scaling stylegan to large diverse datasets. InSIGGRAPH,

  35. [42]

    Fast transformer decoding: One write-head is all you need

    Noam Shazeer. Fast transformer decoding: One write-head is all you need. arXiv preprint arXiv:1911.02150, 2019. 4

    1911 arXiv

  36. [43]

    Glu variants improve transformer

    Noam Shazeer. Glu variants improve transformer. arXiv preprint arXiv:2002.05202, 2020. 6

    2002 arXiv

  37. [44]

    Roformer: Enhanced transformer with rotary position embedding

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063,

  38. [45]

    Autoregressive model beats diffusion: Llama for scalable image generation

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525, 2024. 1, 2, 3, 4, 6, 7, 8, 10, 12

    2024 arXiv

  39. [46]

    Emergent correspondence from image diffusion

    Luming Tang, Menglin Jia, Qianqian Wang, Cheng Perng Phoo, and Bharath Hariharan. Emergent correspondence from image diffusion. In NeurIPS, 2023. 9, 11

    2023

  40. [47]

    Chameleon: Mixed-modal early-fusion foundation models

    Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models. arXiv preprint arXiv:2405.09818, 2024. 1, 2, 9

    2024 arXiv

  41. [48]

    Visual autoregressive modeling: Scalable image generation via next-scale prediction

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Li- wei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. In NeurIPS, 2024. 2, 6, 8

    2024

  42. [49]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 2, 6

    2023 arXiv

  43. [50]

    Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023. 2

    2023 arXiv

  44. [51]

    Conditional image gener- ation with pixelcnn decoders

    Aaron Van den Oord, Nal Kalchbrenner, Lasse Espeholt, Oriol Vinyals, and Alex Graves. Conditional image gener- ation with pixelcnn decoders. In NeurIPS, 2016. 3

    2016

  45. [52]

    Emu3: Next-token prediction is all you need

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, Yingli Zhao, Yulong Ao, Xuebin Min, Tao Li, Boya Wu, Bo Zhao, Bowen Zhang, Liangdong Wang, Guang Liu, Zheqi He, Xi Yang, Jingjing Liu, Yonghua Lin, Tie...

    arXiv

  46. [53]

    Maskbit: Embedding-free image generation via bit tokens

    Mark Weber, Lijun Yu, Qihang Yu, Xueqing Deng, Xiao- hui Shen, Daniel Cremers, and Liang-Chieh Chen. Maskbit: Embedding-free image generation via bit tokens. arXiv preprint arXiv:2409.16211, 2024. 2

    2024 arXiv

  47. [54]

    Show-o: One single transformer to unify multimodal understanding and generation

    Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408.12528, 2024. 2

    2024 arXiv

  48. [55]

    Carbonell, Ruslan Salakhutdinov, and Quoc V

    Zhilin Yang, Zihang Dai, Yiming Yang, Jaime G. Carbonell, Ruslan Salakhutdinov, and Quoc V . Le. Xlnet: General- ized autoregressive pretraining for language understanding. In NeurIPS, 2019. 2, 3

    2019

  49. [56]

    Hauptmann, Ming- Hsuan Yang, Yuan Hao, Irfan Essa, and Lu Jiang

    Lijun Yu, Yong Cheng, Kihyuk Sohn, Jos ´e Lezama, Han Zhang, Huiwen Chang, Alexander G. Hauptmann, Ming- Hsuan Yang, Yuan Hao, Irfan Essa, and Lu Jiang. Magvit: Masked generative video transformer. In CVPR, 2023. 2

    2023

  50. [57]

    Gundavarapu, Luca Ver- sari, Kihyuk Sohn, David Minnen, Yong Cheng, Vigh- nesh Birodkar, Agrim Gupta, Xiuye Gu, Alexander G

    Lijun Yu, Jos ´e Lezama, Nitesh B. Gundavarapu, Luca Ver- sari, Kihyuk Sohn, David Minnen, Yong Cheng, Vigh- nesh Birodkar, Agrim Gupta, Xiuye Gu, Alexander G. Hauptmann, Boqing Gong, Ming-Hsuan Yang, Irfan Essa, David A. Ross, and Lu Jiang. Language model beats diffusion–toke...

  51. [59]

    An image is worth 32 tokens for reconstruction and generation

    Qihang Yu, Mark Weber, Xueqing Deng, Xiaohui Shen, Daniel Cremers, and Liang-Chieh Chen. An image is worth 32 tokens for reconstruction and generation. arXiv preprint arXiv:2406.07550, 2024. 2, 6

    2024 arXiv

  52. [60]

    Root mean square layer nor- malization

    Biao Zhang and Rico Sennrich. Root mean square layer nor- malization. In NeurIPS, 2019. 6

    2019

  53. [61]

    Diffcollage: Parallel generation of large content with diffusion models

    Qinsheng Zhang, Jiaming Song, Xun Huang, Yongxin Chen, and Ming-Yu Liu. Diffcollage: Parallel generation of large content with diffusion models. In CVPR, 2023. 8

    2023

  54. [62]

    Transfusion: Pre- dict the next token and diffuse images with one multi-modal model

    Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Pre- dict the next token and diffuse images with one multi-modal model. arXiv preprint arXiv:2408.11039, 2024. 1, 2, 9

    2024 arXiv

  55. [63]

    Segment everything everywhere all at once

    Xueyan Zou, Jianwei Yang, Hao Zhang, Feng Li, Linjie Li, Jianfeng Wang, Lijuan Wang, Jianfeng Gao, and Yong Jae Lee. Segment everything everywhere all at once. InNeurIPS,

  56. [2021]

    2, 3, 5, 6, 7, 8, 10

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.