Pith. sign in

REVIEW 4 major objections 6 minor 5 cited by

Hita: Holistic Tokenizer for Autoregressive Image Generation

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A tokenizer that writes global 'holistic' tokens before patch tokens lifts a 2B autoregressive image model to 2.59 FID and 281.9 IS on ImageNet, trains faster, and unlocks zero-shot style transfer and inpainting.

desk verdict Solid tokenizer paper with real gains, but the headline mechanism—holistic-first ordering—isn't isolated from DINOv2 injection and the k-drop bottleneck; still worth a careful referee. read the letter →

arxiv 2507.02358 v4 pith:OF5CSHHU submitted 2025-07-03 cs.CV cs.AI

Anlin Zheng , Haochen Wang , Yucheng Zhao , Weipeng Deng , Tiancai Wang , Xiangyu Zhang , Xiaojuan Qi This is my paper
classification cs.CVcs.AI
keywords holistic-to-localtokenizationautoregressiveimagegenerationvectorquantizationNetclass-conditionalzero-shotstyletransferinpaintingDINOv2semanticinjectioncausaltokenfusion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that autoregressive image models are limited not just by their architecture but by their tokenizers: with patch-only visual tokens, the generator must rediscover global structure one token at a time, which hurts coherence and slows learning. Hita proposes a holistic-to-local tokenization scheme in which 128 learnable queries summarize whole-image properties such as texture, material, shape, and color into discrete holistic tokens placed at the start of the token sequence, ahead of the raster-scanned patch tokens. Semantic features from a frozen pretrained vision model are injected into the queries, and a causal token fusion module with a small 'drop the first k patch tokens' trick forces the decoder to actually use the holistic tokens. The resulting 2B autoregressive model reaches 2.59 FID and 281.9 IS on ImageNet 256x256 class-conditional generation, the smaller Hita models beat same-size LlamaGen baselines under equal training budgets, and the tokenizer alone enables zero-shot style transfer and image inpainting. The deeper claim is that a tokenizer whose latent sequence is ordered to match causal next-token prediction can make vanilla GPT-style transformers competitive with diffusion and masked models for image synthesis.

What carries the argument

The central object is the holistic-to-local token sequence with a causal prefix structure. Learnable queries interact with patch embeddings and injected DINOv2 features through a transformer to produce 128 holistic features, while separate codebooks of size 16,384 quantize holistic and patch features (dimensions 12 and 8). Before decoding, a causal token fusion module drops the first k=4 patch tokens and uses the last 4 holistic tokens in their place, which forces the decoder to rely on the holistic summary and prevents holistic codebook collapse. This ordered, causal latent space is what lets a standard autoregressive transformer treat the holistic tokens as a global prompt.

What would settle it

Train the same Hita pipeline but replace the 128 holistic tokens with a fixed learned vector or with the pretrained model's class token at AR generation time, keeping the tokenizer otherwise identical; if the ImageNet 256x256 FID stays near 2.59 the holistic summary is not doing the work, while a large FID jump would confirm that it is.

Watch

Extended reading notes

Core claim

Hita's central claim is that global information can be moved into the token sequence itself, as a prefix, without changing the autoregressive generator. The tokenizer encodes an image into 128 holistic tokens followed by patch-level tokens, quantizes the two groups with separate codebooks, and feeds the de-quantized tokens through a causal transformer before decoding. To keep patch tokens from bypassing the holistic ones, the fusion module discards the first k=4 patch tokens and replaces them with the last 4 holistic tokens, so the decoder has to draw on holistic information. The AR transformer then first predicts holistic tokens as a prefix prompt and uses them to guide the subsequent generation of patch tokens. With this scheme a 2B Llama-style model achieves 2.59 FID and 281.9 IS on ImageNet 256x256, reaches FID 4.22 about 2.1x faster during training than LlamaGen, and the holistic tokens demonstrably capture global properties through style transfer, inpainting, and linear probing experiments.

Load-bearing premise

The design assumes that a quantized 128-token global summary, built from local image features plus features from a frozen pretrained vision model and passed through a small causal transformer, really captures the information an autoregressive generator needs as its prefix, but the evidence is indirect because a training-time trick that drops the first few patch tokens before decoding forces the decoder to rely on the global summary, so part of the measured gain could come from the pretrained vision model's semantics or from added capacity rather than from the global-then-local ordering itself.

Editorial extensions

If this is right

  • AR image generators trained on Hita tokens converge faster: the time to reach an FID of 4.22 is reduced by a factor of 2.1 compared with the same training setup on LlamaGen tokens.
  • A 2B-parameter Hita model reaches 2.59 FID and 281.9 IS on ImageNet 256x256 class-conditional generation, surpassing the LlamaGen-3B baseline with one third fewer parameters.
  • At equal size and training budget, Hita-B achieves 5.85 FID versus LlamaGen-B's 8.31 at 50 epochs, and the advantage persists at 300 epochs (Hita-B 4.33, Hita-L 2.86).
  • The holistic tokens support zero-shot style transfer by swapping the holistic tokens between images, and zero-shot inpainting by feeding a partially masked image's tokens as a prefix.
  • The method leaves the autoregressive transformer unchanged, so GPT-style next-token models can be used as-is without bidirectional-attention modifications.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: Because the k=4 replacement trick forces the decoder to depend on the holistic tokens, the measured gains are partly guaranteed by architecture; ablating the DINOv2 feature injection at test time would show how much of the prefix's usefulness is genuinely learned by the holistic codebook rather than inherited from the pretrained model.
  • Inference: The same holistic-prefix design should transfer to text-conditioned and video autoregressive models, where a compact block of global tokens could play the role of a content summary or frame-level condition while keeping the next-token paradigm intact.
  • Inference: A direct test of whether the holistic codebook stores a compact global prior is to vary the number of holistic tokens (for example, 32, 64, 128, 256) and measure generation FID; if quality is flat, the tokens are mostly copying pretrained semantics rather than compressing global image structure.
  • Inference: The paper's own limitation note that Hita was trained only on ImageNet with basic optimization suggests the reported numbers are conservative lower bounds; scaling data, text conditioning, and training time are the obvious next axes of improvement.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Hita, an image tokenizer for autoregressive (AR) generation that augments standard patch tokens with a set of 128 learnable holistic tokens. The holistic tokens are derived from patch embeddings plus DINOv2 semantic features, quantized in a separate codebook, and placed at the beginning of the token sequence so that a causal AR model first generates holistic tokens as a prefix and then generates patch tokens. A causal fusion transformer with a k-token selection trick (k=4) is used before decoding. The authors report that Hita-based AR models outperform LlamaGen at matched model sizes and training budgets on ImageNet 256x256 class-conditional generation, with a 2B model reaching 2.59 FID and 281.9 IS, and they demonstrate zero-shot style transfer and inpainting.

Significance. If the reported gains are robust, Hita would be a useful step toward giving AR image generators an explicit global prior while preserving the standard next-token paradigm. The paper is thorough in ablating several design choices (queries, semantic injection, k, attention modules, fusion variants) and compares fairly with LlamaGen at matched sizes and epochs. The code is released, which strengthens reproducibility. The two most attractive properties—the holistic-to-local prefix ordering and the zero-shot abilities—are interesting and potentially transferable to other AR visual generation settings. However, the central mechanism is not yet cleanly isolated from confounds, and the headline numbers are single-run point estimates.

major comments (4)
  1. [Sec. 3.3, Tables 2 and 3] The central claim that the holistic-to-local prefix ordering is load-bearing is not isolated by any experiment. The comparisons against LlamaGen change the token content (adding a 128-query holistic branch and DINOv2 semantic injection), the reconstruction bottleneck (the k=4 trick of Eq. 5), and the sequence order simultaneously. The ablations in Table 2 add queries and DINOv2 while keeping the order fixed, and Table 3 varies k without varying order. No run uses a reversed order (patch tokens first, holistic tokens last) with the same tokenizer, token count, and architecture. Without such a run, the observed FID gains could be caused by DINOv2 semantic leakage, the k=4 regularizing bottleneck, or extra capacity rather than by the proposed ordering. A reversed-order AR training run, or a run with the same tokenizer but shuffled token order, is necessary to support the paper's main conceptual contribution.
  2. [Sec. 3.2.3, Eq. (5)] Eq. (5) states that the decoder receives only the last k=4 of the 128 holistic tokens together with the first N-patch tokens (dropping the last k patch tokens). This means that 124 of the 128 holistic tokens that the AR model generates are never used in the final reconstruction. The manuscript does not discuss or justify this substantial discarding of generated tokens. It raises the question of why the AR model should spend capacity predicting 128 holistic tokens if only 4 matter for the image, and whether the reported gains are due to the k=4 bottleneck rather than to the holistic-to-local ordering. The authors should either justify the choice of M=128 given k=4, include an ablation of M with k fixed, or explicitly address what information, if any, the discarded holistic tokens contribute.
  3. [Sec. 4.3, Table 5] The linear probing result (Hita 36.6% vs VQGAN 6.9% top-1 accuracy) is cited as evidence that holistic tokens carry semantic content, but the holistic features are computed with DINOv2 features directly injected (Sec. 3.2.1, Eq. 4). The table lacks a control: linear probing on DINOv2 features alone, or on holistic tokens produced without semantic injection. As reported, the accuracy gain could largely reflect the injected DINOv2 representation rather than a property of the learned holistic tokenizer. This weakens the supporting argument in Sec. 3.4 and the 'What do holistic tokens capture?' discussion.
  4. [Sec. 4.2, Tables 1-4] All FID and IS numbers in the paper are single-run point estimates with no error bars or multiple seeds. Given that several comparisons are close (e.g., Hita-L 2.86 vs LlamaGen-L 3.07 at 300 epochs; Hita-XL 2.98 vs LlamaGen-XL 3.24 at 50 epochs), it is not possible to assess whether the reported improvements are statistically significant. The authors should report at least 2-3 seeds with mean and variance for the main comparisons, or justify why single runs are sufficient in this setting.
minor comments (6)
  1. [Sec. 1] The word 'autogressive' is a typo for 'autoregressive' in the first sentence of the Introduction.
  2. [Sec. 3.2.1, Eq. (4)] In the text following Eq. (4), 'path-level embeddings' should be 'patch-level embeddings'.
  3. [Sec. 3.3] The section title 'Autoreggresive Image Generation' has a typo; it should be 'Autoregressive'.
  4. [Sec. 5] The Conclusion contains 'zeros-shot image in-painting' and 'incoporate'; these should be 'zero-shot' and 'incorporate'.
  5. [Sec. 4.1] The sentence 'we set the codebook vector dimension of the patch-level image token and holistic token to 8 and 12' should be 'patch-level image tokens and holistic tokens'.
  6. [Fig. 2c] The training-speed plot lacks clear axis labels and definitions. Please specify whether the x-axis is wall-clock time or training iterations, and describe how the FID=4.22 threshold was chosen.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Hita's generation gains are measured against external ImageNet FID/IS benchmarks, and the semantic-injection analysis is an attribution confound rather than a self-referential derivation.

full rationale

The paper's central claim is that a holistic-to-local tokenizer improves autoregressive image generation. The supporting evidence is external: FID, IS, precision, and recall on ImageNet 256x256, plus ablations that independently vary learnable queries, DINOv2 injection, fusion length k, and attention modules. None of these numbers is fitted by construction to the reported FID, and no parameter is renamed as a prediction. The holistic-to-local ordering itself is tested through the k-ablation and the token-fusion variants, even though the ordering is not isolated from the simultaneous DINOv2 injection and the k=4 bottleneck; that is a confound in attribution, not circularity. The linear-probing analysis of holistic tokens does inherit semantics from the injected DINOv2 features, but the paper explicitly acknowledges this ('DINOv2, a pre-trained model integrated in Hita, injects necessary semantic-aware features'), so the evidence is not presented as independent of its input. The only self-citations (Refs. 28 and 52) appear in general related-work lists and are not load-bearing for the method or its evaluation. No uniqueness theorem, ansatz, or fitted quantity is imported from the authors' prior work. Accordingly, the derivation chain is self-contained with respect to the reported benchmarks, and there is no circular step to flag.

Assumptions & free parameters 5 free parameters · 4 assumptions · 1 invented entities

The central claim depends on a handful of hand-set or ablation-selected hyperparameters, chiefly k=4, 128 holistic queries, codebook dimensions, and transformer depth. The method also assumes DINOv2 semantics survive quantization, that the k-drop fusion prevents codebook collapse, and that ImageNet metrics measure holistic coherence. These are reasonable engineering assumptions, but they are not first-principles derivations.

free parameters (5)
  • Holistic token selection length k = 4
    The k parameter controls how many patch tokens are excluded from the direct reconstruction path, forcing reliance on holistic tokens. It was chosen by comparing k=0, 1, 2, and 4 on ImageNet reconstruction and generation metrics (Table 3).
  • Number of holistic queries M = 128
    The number of learnable holistic queries is fixed at 128 without an ablation study for query count.
  • Codebook sizes and vector dimensions = patch: 16384 x 8; holistic: 16384 x 12
    Codebook size 16,384 and vector dimensions 8 and 12 are adopted following LlamaGen and TiTok, and they directly affect reconstruction fidelity and codebook utilization.
  • Transformer depth in tokenizer = 3 layers
    The paper states the depth of each transformer is set to 3 to preserve tokenizer simplicity; this depth is not ablated.
  • Tokenizer loss weights = alpha=1, lambda=1
    The combined loss is L = alpha * L_vq + lambda * L_AE with both weights set to 1, following VQGAN-style training without tuning.
assumptions (4)
  • domain assumption DINOv2 features carry holistic semantics that remain informative after being transformed and quantized into a 12-dimensional holistic codebook.
    The semantic injection step in Sec. 3.2.1 relies on this assumption, and the linear-probing results in Sec. 4.3 are used as evidence for it.
  • ad hoc to paper Removing the first k patch tokens from the direct reconstruction path forces the fusion transformer to use holistic tokens and prevents holistic codebook collapse.
    This is an engineered mechanism introduced in Sec. 3.2.3 and validated only empirically in Table 3; it is not derived from a theoretical principle.
  • standard math Standard VQGAN training losses and transformer attention are sufficient to learn the intended holistic and local latent spaces.
    The paper reuses quantized autoencoder losses, perceptual loss, and adversarial loss from VQGAN as described in Sec. 3.2.4.
  • domain assumption FID, IS, precision, recall, linear-probing accuracy, and CLIP score on ImageNet validation are valid proxies for generation quality and holistic coherence.
    All headline claims are evaluated with these metrics, as specified in Sec. 4.1 and Sec. C.4 of the appendix.
invented entities (1)
  • Holistic latent tokens
    purpose: To encode global image properties, such as color, material, texture, and shape, as a discrete prefix sequence for autoregressive generation.
    The holistic token is a new latent construct introduced by the paper. Its usefulness is demonstrated through the authors' own style-transfer, inpainting, and linear-probing experiments, but no external benchmark or independently predicted observable exists, and no verified code release accompanies the preprint.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hita: Holistic Tokenizer for Autoregressive Image Generation." pith.science (2026). https://pith.science/paper/OF5CSHHU

@misc{pith2026250702358,
  author       = {Pith},
  title        = {Pith review of: Hita: Holistic Tokenizer for Autoregressive Image Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OF5CSHHU}},
  note         = {Machine review of arXiv:2507.02358}
}
read the original abstract

Vanilla autoregressive image generation models generate visual tokens step-by-step, limiting their ability to capture holistic relationships among token sequences. Moreover, because most visual tokenizers map local image patches into latent tokens, global information is limited. To address this, we introduce \textit{Hita}, a novel image tokenizer for autoregressive (AR) image generation. It introduces a holistic-to-local tokenization scheme with learnable holistic queries and local patch tokens. Hita incorporates two key strategies to better align with the AR generation process: 1) {arranging} a sequential structure with holistic tokens at the beginning, followed by patch-level tokens, and using causal attention to maintain awareness of previous tokens; and 2) adopting a lightweight fusion module before feeding the de-quantized tokens into the decoder to control information flow and prioritize holistic tokens. Extensive experiments show that Hita accelerates the training speed of AR generators and outperforms those trained with vanilla tokenizers, achieving \textbf{2.59 FID} and \textbf{281.9 IS} on the ImageNet benchmark. Detailed analysis of the holistic representation highlights its ability to capture global image properties, such as textures, materials, and shapes. Additionally, Hita also demonstrates effectiveness in zero-shot style transfer and image in-painting. The code is available at \href{https://github.com/CVMI-Lab/Hita}{https://github.com/CVMI-Lab/Hita}.

Figures

Figures reproduced from arXiv: 2507.02358 by the authors.

Figure 1
Figure 1. The concept of holistic tokenizer. A set of learnable queries that capture global properties, such as color, texture, ma￾terial, etc, from pixels, with semantic-level feature injected, is uti￾lized to reconstruct the image along with image patches. as DALL-E [38], Parti [55, 56], VAR [47], and LlamaGen [46]. In AR-based image generation, VQVAE [12] is em￾ployed to encode images into discrete tokens. These tokens are… view at source ↗
Figure 2
Figure 2. Holistic tokens capture the global style and content information, as demonstrated by their ability to enable (a) style transfer by replacing the holistic tokens of input images with those of reference images, and (b) zero-shot image inpainting using pre-trained AR generation models . Additionally, the introduction of these new latent tokens accelerates the training speed of the GPT-style AR image generation model by… view at source ↗
Figure 3
Figure 3. The flowchart of the proposed holistic image tokenizer, which includes: [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Token fusion and decoding procedure The the codebook significantly impacts the performance of the image tokenizer. Following [46, 54], we apply ℓ2- normalization to the codebook vectors, opting for a lower vector dimension and a larger codebook size to improve the reco…
Figure 5
Figure 5. Figure 5: Class-conditional image generation with our proposed Hita. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Token fusion module comparison. (a).Token fusion [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Quantitative zero-shot in-painting analysis with Hita-L conducted on ImageNet [ [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Some zero-shot style-transfer samples by Hita’s holistic tokenizer. Best viewed with zoom-in. [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Zero-shot in-painting examples by Hita’s AR generation model. Compared with the baseline. Best viewed with zoom-in. [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Visualization of class-conditional samples generated by Hita. Best viewed with zoom-in. [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Autoregressive Visual Generation Needs a Prologue

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    Prologue introduces dedicated prologue tokens to decouple generation and reconstruction in AR visual models, significantly improving generation FID scores on ImageNet while maintaining reconstruction quality.

  2. Autoregressive Visual Generation Needs a Prologue

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    Prologue adds a small set of learnable tokens trained exclusively with AR cross-entropy loss to decouple generation from reconstruction in autoregressive visual models, yielding lower gFID on ImageNet 256x256.

  3. Beyond Patches: Global-aware Autoregressive Model for Multimodal Few-Shot Font Generation

    cs.CV 2026-01 unverdicted novelty 7.0 of 10

    GAR-Font is a global-aware autoregressive framework for multimodal few-shot font generation that adds global tokenization, a language-style adapter, and post-refinement to improve style coherence over patch-based methods.

  4. Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipes

    cs.CV 2026-08 conditional novelty 6.0 of 10

    Multimodal pretraining transfers asymmetrically: language boosts vision, understanding boosts generation, generation is mostly neutral, and early unified training prevents vision laziness.

  5. Mutual Enhancement Between Global Tokens and Patch Tokens: From Theory to Practice

    cs.CV 2026-05 unverdicted novelty 5.0 of 10

    TaTok is a theoretically grounded adaptive tokenization method that uses global tokens and cumulative conditional entropy filtering to reduce redundancy while improving reconstruction quality over fixed-rate patch tok...

Reference graph

Works this paper leans on

62 extracted references · 27 canonical work pages · cited by 4 Pith papers

  1. [1]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023. 1

    arXiv 2023

  2. [2]

    Estimating or propagating gradients through stochastic neurons for conditional computation

    Yoshua Bengio, Nicholas L ´eonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432, 2013. 3

    arXiv 2013

  3. [3]

    Paligemma: A versatile 3b vlm for trans- fer

    Lucas Beyer, Andreas Steiner, Andr ´e Susano Pinto, Alexan- der Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, et al. Paligemma: A versatile 3b vlm for trans- fer. arXiv preprint arXiv:2407.07726, 2024. 3

    arXiv 2024

  4. [4]

    Language models are few-shot learners

    Tom B Brown. Language models are few-shot learners. arXiv preprint arXiv:2005.14165, 2020. 1

    arXiv 2005

  5. [5]

    Maskgit: Masked generative image transformer

    Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. Maskgit: Masked generative image transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11315–11325, 2022. 6, 7, 12

    work page 2022

  6. [6]

    Generative pre- training from pixels

    Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Hee- woo Jun, David Luan, and Ilya Sutskever. Generative pre- training from pixels. In International conference on machine learning, pages 1691–1703. PMLR, 2020. 1, 3

    work page 2020

  7. [7]

    Vision transformers need registers, 2023

    Timoth ´ee Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers, 2023. 2, 4, 6, 7, 8

    work page 2023

  8. [8]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 6, 7, 8, 12, 13, 14

    work page 2009

Show all 62 references
  1. [9]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. 1

    2018 arXiv

  2. [10]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural informa- tion processing systems, 34:8780–8794, 2021. 1, 6

    2021

  3. [11]

    Scalable pre- training of large autoregressive image models.arXiv preprint arXiv:2401.08541, 2024

    Alaaeldin El-Nouby, Michal Klein, Shuangfei Zhai, Miguel Angel Bautista, Alexander Toshev, Vaishaal Shankar, Joshua M Susskind, and Armand Joulin. Scalable pre- training of large autoregressive image models.arXiv preprint arXiv:2401.08541, 2024. 2, 3

    2024 arXiv

  4. [12]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021. 1, 2, 3, 4, 5, 6, 7, 12, 14

    2021

  5. [13]

    Bard, 2023

    Google. Bard, 2023. 1

    2023

  6. [14]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 8, 14

    2016

  7. [15]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 16000– 16009, 2022. 8, 13

    2022

  8. [16]

    Scaling laws for autoregressive generative modeling

    Tom Henighan, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Heewoo Jun, Tom B Brown, Prafulla Dhariwal, Scott Gray, et al. Scaling laws for autoregressive generative modeling. arXiv preprint arXiv:2010.14701, 2020. 1

    2010 arXiv

  9. [17]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 6, 12

    2017

  10. [18]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 6

    2022 arXiv

  11. [19]

    Cascaded diffu- sion models for high fidelity image generation

    Jonathan Ho, Chitwan Saharia, William Chan, David J Fleet, Mohammad Norouzi, and Tim Salimans. Cascaded diffu- sion models for high fidelity image generation. Journal of Machine Learning Research, 23(47):1–33, 2022. 1, 6

    2022

  12. [20]

    Image-to-image translation with conditional adver- sarial networks

    Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adver- sarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1125–1134,

  13. [21]

    Scal- ing up gans for text-to-image synthesis

    Minguk Kang, Jun-Yan Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, and Taesung Park. Scal- ing up gans for text-to-image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10124–10134, 2023. 6, 7

    2023

  14. [22]

    Scaling laws for neural language models

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361,

    2001 arXiv

  15. [23]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 2

    2013 arXiv

  16. [24]

    The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale

    Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Ui- jlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, et al. The open images dataset v4: Unified image classification, object detection, and visual relationship detection...

    1956

  17. [25]

    Improved precision and recall met- ric for assessing generative models

    Tuomas Kynk ¨a¨anniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall met- ric for assessing generative models. Advances in neural in- formation processing systems, 32, 2019. 6

    2019

  18. [26]

    Autoregressive image generation using residual quantization

    Doyup Lee, Chiheon Kim, Saehoon Kim, Minsu Cho, and Wook-Shin Han. Autoregressive image generation using residual quantization. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 11523–11532, 2022. 2, 3, 6, 7, 12

    2022

  19. [27]

    Autoregressive image generation without vec- tor quantization

    Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vec- tor quantization. arXiv preprint arXiv:2406.11838, 2024. 2, 3

    2024 arXiv

  20. [28]

    Can ood object detectors learn from founda- tion models? In European Conference on Computer Vision, pages 213–231

    Jiahui Liu, Xin Wen, Shizhen Zhao, Yingxian Chen, and Xiaojuan Qi. Can ood object detectors learn from founda- tion models? In European Conference on Computer Vision, pages 213–231. Springer, 2024. 3

    2024

  21. [29]

    Open-magvit2: An open-source project toward democratizing auto-regressive visual gener- ation

    Zhuoyan Luo, Fengyuan Shi, Yixiao Ge, Yujiu Yang, Limin Wang, and Ying Shan. Open-magvit2: An open-source project toward democratizing auto-regressive visual gener- ation. arXiv preprint arXiv:2409.04410, 2024. 2, 3

    2024 arXiv

  22. [30]

    Finite scalar quantization: Vq-vae made simple

    Fabian Mentzer, David Minnen, Eirikur Agustsson, and Michael Tschannen. Finite scalar quantization: Vq-vae made simple. arXiv preprint arXiv:2309.15505, 2023. 3

    2023 arXiv

  23. [31]

    Chatgpt, 2022

    OpenAI. Chatgpt, 2022. 1

    2022

  24. [32]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 4195–4205,

  25. [33]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 1

    2023 arXiv

  26. [34]

    Improving language understanding by gener- ative pre-training

    Alec Radford. Improving language understanding by gener- ative pre-training. 2018. 3

    2018

  27. [35]

    Language models are unsu- pervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsu- pervised multitask learners. OpenAI blog, 1(8):9, 2019. 1

    2019

  28. [36]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  29. [37]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67, 2020. 1

    2020

  30. [38]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International confer- ence on machine learning, pages 8821–8831. Pmlr, 2021. 1

    2021

  31. [39]

    Gener- ating diverse high-fidelity images with vq-vae-2

    Ali Razavi, Aaron Van den Oord, and Oriol Vinyals. Gener- ating diverse high-fidelity images with vq-vae-2. Advances in neural information processing systems, 32, 2019. 6

    2019

  32. [40]

    Gener- ating diverse high-fidelity images with vq-vae-2

    Ali Razavi, Aaron Van den Oord, and Oriol Vinyals. Gener- ating diverse high-fidelity images with vq-vae-2. Advances in neural information processing systems, 32, 2019. 2, 3

    2019

  33. [41]

    High-resolution image syn- thesis with latent diffusion models, 2021

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models, 2021. 1, 2, 6, 7

    2021

  34. [42]

    Improved techniques for training gans

    Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016. 6, 12

    2016

  35. [43]

    Scaling stylegan to large diverse datasets

    A Sauer, K Schwarz, and A StyleGAN-XL Geiger. Scaling stylegan to large diverse datasets. InProceedings of the SIG- GRAPH Conference. ACM, pages 1–10. 6, 7

  36. [44]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 1

    2010 arXiv

  37. [45]

    Roformer: Enhanced transformer with rotary position embedding

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063,

  38. [46]

    Autoregressive model beats diffusion: Llama for scalable image generation

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525, 2024. 1, 2, 3, 5, 6, 7, 8, 12, 13, 14

    2024 arXiv

  39. [47]

    Visual autoregressive modeling: Scalable im- age generation via next-scale prediction

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Li- wei Wang. Visual autoregressive modeling: Scalable im- age generation via next-scale prediction. arXiv preprint arXiv:2404.02905, 2024. 1, 2, 3, 12

    2024 arXiv

  40. [48]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 1, 2, 5

    2023 arXiv

  41. [49]

    Conditional image genera- tion with pixelcnn decoders

    Aaron Van den Oord, Nal Kalchbrenner, Lasse Espeholt, Oriol Vinyals, Alex Graves, et al. Conditional image genera- tion with pixelcnn decoders. Advances in neural information processing systems, 29, 2016. 1, 3

    2016

  42. [50]

    Neural discrete representation learning

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information pro- cessing systems, 30, 2017. 2, 3

    2017

  43. [51]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017. 1

    2017

  44. [52]

    Recon- structive visual instruction tuning

    Haochen Wang, Anlin Zheng, Yucheng Zhao, Tiancai Wang, Ge Zheng, Xiangyu Zhang, and Zhaoxiang Zhang. Recon- structive visual instruction tuning. In International Confer- ence on Learning Representations, 2025. 1

    2025

  45. [53]

    Show-o: One single transformer to unify multimodal understanding and generation

    Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408.12528, 2024. 2, 3

    2024 arXiv

  46. [55]

    Vector-quantized image modeling with improved vqgan

    Jiahui Yu, Xin Li, Jing Yu Koh, Han Zhang, Ruoming Pang, James Qin, Alexander Ku, Yuanzhong Xu, Jason Baldridge, and Yonghui Wu. Vector-quantized image modeling with improved vqgan. arXiv preprint arXiv:2110.04627, 2021. 1, 3

    2021 arXiv

  47. [56]

    Scaling autoregres- sive models for content-rich text-to-image generation

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gun- jan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yin- fei Yang, Burcu Karagol Ayan, et al. Scaling autoregres- sive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2(3):5, 2022. 1, 3

    2022 arXiv

  48. [57]

    Magvit: Masked generative video transformer

    Lijun Yu, Yong Cheng, Kihyuk Sohn, Jos ´e Lezama, Han Zhang, Huiwen Chang, Alexander G Hauptmann, Ming- Hsuan Yang, Yuan Hao, Irfan Essa, et al. Magvit: Masked generative video transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p...

    2023

  49. [58]

    Language model beats diffusion–tokenizer is key to visual generation

    Lijun Yu, Jos ´e Lezama, Nitesh B Gundavarapu, Luca Ver- sari, Kihyuk Sohn, David Minnen, Yong Cheng, Vighnesh Birodkar, Agrim Gupta, Xiuye Gu, et al. Language model beats diffusion–tokenizer is key to visual generation. arXiv preprint arXiv:2310.05737, 2023. 1

    arXiv

  50. [59]

    An image is worth 32 tokens for reconstruction and generation

    Qihang Yu, Mark Weber, Xueqing Deng, Xiaohui Shen, Daniel Cremers, and Liang-Chieh Chen. An image is worth 32 tokens for reconstruction and generation. arXiv preprint arXiv:2406.07550, 2024. 3, 6, 7, 12, 13

    2024 arXiv

  51. [60]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 3

    2018

  52. [61]

    Opt: Open pre-trained trans- former language models

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained trans- former language models. arXiv preprint arXiv:2205.01068,

    arXiv

  53. [62]

    Movq: Modulating quantized vectors for high- fidelity image generation

    Chuanxia Zheng, Tung-Long Vuong, Jianfei Cai, and Dinh Phung. Movq: Modulating quantized vectors for high- fidelity image generation. Advances in Neural Information Processing Systems, 35:23412–23425, 2022. 2

    2022

  54. [63]

    Scaling the codebook size of vqgan to 100,000 with a utilization rate of 99%

    Lei Zhu, Fangyun Wei, Yanye Lu, and Dong Chen. Scaling the codebook size of vqgan to 100,000 with a utilization rate of 99%. arXiv preprint arXiv:2406.11837, 2024. 3 11 First, we compare Hita with other vanilla tokenizers and further discuss the token fusion module. Then, we e...

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.