Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

Quantize-then-Rectify: Efficient VQ-VAE Training

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper establishes that a pretrained continuous VAE can be converted into a discrete VQ-VAE at a fraction of the usual training cost by freezing the VAE and training only a channel multi-group quantizer and a small post rectifier with…

desk verdict A genuinely useful recipe for converting a frozen VAE into a 512-token VQ-VAE, but the headline efficiency claim omits the pre-encoding cost and compares unequal token counts. read the letter →

arxiv 2507.10547 v1 pith:PMVYEBDG submitted 2025-07-14 cs.CV cs.LG

Borui Zhang , Qihang Rao , Wenzhao Zheng , Jie Zhou , Jiwen Lu This is my paper
classification cs.CVcs.LG
keywords visualtokenizationvectorquantizationVQ-VAEimagereconstructiontrainingefficiencycodebooklearninglatentspacediscreterepresentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ReVQ aims to show that training a vector-quantized variational autoencoder (VQ-VAE) from scratch is unnecessary: a pre-trained continuous autoencoder (VAE) can be converted into a discrete tokenizer by freezing the VAE and training only a multi-group codebook plus a small post-quantization rectifier. The paper reports that this recipe compresses ImageNet images into at most 512 tokens with an rFID of 1.06 (a perceptual reconstruction metric), and that full training takes about 22 hours on a single RTX 4090, two orders of magnitude less than comparable from-scratch methods. This matters because VQ-VAEs are the standard bridge between images and discrete language-model tokens, and their current training cost puts them out of reach for many research groups. If the recipe transfers, discrete tokenizers become a cheap add-on to any existing VAE rather than a large-scale training project.

What carries the argument

The load-bearing objects are the channel multi-group quantizer and the post rectifier. Channel multi-group quantization splits the latent feature dimension into B groups, each with its own codebook, raising the effective degrees of freedom from N to N×B; the paper argues channel-wise splits yield more independent feature distributions than spatial splits. The post rectifier $g$ is a small EfficientViT block trained to invert quantization error on the frozen VAE's latent, giving $\mathbf{Z}'_e = g(q(\mathbf{Z}_e, \mathbf{C}))$ and trained by minimizing $\lVert \mathbf{Z}_e - g(q(\mathbf{Z}_e))\rVert_2^2$. A support module, non-activation reset, reinitializes unused codes near highly activated ones at the end of each epoch to prevent codebook collapse. The whole design is justified by an empirical noise-tolerance threshold: the DC-AE reconstruction survives Gaussian latent noise up to variance about 0.3, so ReVQ only needs to keep quantization error below a comparable level.

What would settle it

Re-run the Fig. 3b tolerance test using the actual distribution of quantization errors produced by the trained ReVQ codebook instead of Gaussian noise; if reconstructed images degrade at latent MSE below the 0.1 threshold used in the paper, the load-bearing premise fails. A second check is to swap in a different pre-trained VAE and see whether the $\ell^2$-only rectifier still keeps rFID competitive.

Watch

Extended reading notes

Core claim

ReVQ's central claim is that vector-quantized autoencoders do not have to be trained from scratch: a pre-trained continuous VAE already contains a decoder whose tolerance for latent noise can absorb the error introduced by quantization. The paper identifies a boundary condition — quantization noise must stay below the VAE's tolerance threshold — and then constructs a quantizer that stays within it. The quantizer splits latent channels into groups, giving each group its own codebook so the effective capacity grows from N to N×B, and resets unused codes each epoch to prevent index collapse. A post rectifier, a small EfficientViT block, then learns to map the quantized features back toward the original latent under a pure $\ell^2$ loss (Eq. 5) with the VAE frozen. On ImageNet the 512-token configuration reaches rFID 1.06, and the paper reports completing training in about 22 hours on one RTX 4090, compared with 3456 GPU-hours for MaskBit.

Load-bearing premise

The load-bearing premise is that minimizing $\ell^2$ error on the frozen VAE's latent features keeps the quantizer's noise inside the VAE's tolerance region, so the decoder produces perceptually good images without any perceptual or adversarial training signal.

Editorial extensions

If this is right

  • High-compression discrete tokenizers could be produced in about a day on a single consumer GPU, cutting reported GPU-hours by 40x to 150x.
  • The 512-token ReVQ variant beats prior frozen-VAE tokenizers (TokenBridge at 4096 tokens, CODA at 2560 tokens) on rFID while using far fewer tokens.
  • At 256 tokens, ReVQ needs large codebooks (up to 262,144) and still trails MaskBit, so the method does not yet displace from-scratch training at the highest compression ratios.
  • Channel-axis multi-group splitting consistently beats spatial splitting in rFID, supporting the claim that latent features are more independent across channels.
  • Non-activation reset keeps codebook utilization above 97% even at 262,144 codes; without it, utilization drops to about 65%.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The l2-only training objective suggests the same quantize-then-rectify recipe could be lifted to other frozen autoencoders for video, audio, or 3D data, which the paper leaves untested.
  • The fitted exponential relation between token length and required codebook size implies that codebook scaling alone cannot reach TiTok-style 32-token compression; the rectifier architecture, not the codebook, is the lever the paper itself flags for future work.
  • A natural extension, not tested in the paper, would be to add a lightweight perceptual or adversarial loss on the decoded output while keeping the VAE frozen, which would test whether the l2-only target is the bottleneck or the speed advantage is preserved.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes ReVQ, a method for converting a pre-trained continuous VAE into a discrete VQ-VAE by freezing the VAE encoder and decoder and training only a channel multi-group quantizer plus a post-quantization rectifier. The quantized latents are mapped back through the frozen decoder, and all training is done with an ℓ2 loss on the normalized latent features (Eq. 5). On ImageNet, ReVQ reports rFID 1.06 with at most 512 tokens, and the paper claims that full training takes about 22 hours on a single RTX 4090, a 40–150x reduction in GPU-hours compared with MaskBit and TiTok-S-128. The paper includes ablations on spatial vs. channel multi-group quantization, on the non-activation reset strategy, and on the rectifier architecture, as well as additional training curves and visualizations in the appendix.

Significance. The central idea — reusing a strong pre-trained VAE by training only a lightweight quantizer/rectifier — is simple and potentially valuable for making high-compression tokenizers accessible. The component-wise ablations are clear and support the design choices: channel splitting beats spatial splitting, the reset strategy improves codebook utilization, and a ViT rectifier outperforms CNN and MLP counterparts. If the efficiency claim is accurate after accounting for all preprocessing costs, the method represents a real step toward cheap discrete visual tokenization. However, the headline training-cost comparison omits the one-time pre-encoding of all 1.28M images and the pretraining cost of the frozen VAE, which is load-bearing because the main contribution is the claimed two-orders-of-magnitude speedup. The 'boundary condition' linking VAE noise tolerance to quantization error is also empirical and heuristic, rather than a formal condition, which limits the strength of the general framework claim.

major comments (3)
  1. [Appendix C.1, Table 3, Abstract] The claimed 'full training' time of 22 hours on one RTX 4090 (Abstract, Section 1, Table 3) excludes the one-time DC-AE encoding of all 1.28M ImageNet training images into 2048-dimensional latent vectors described in Appendix C.1. Because the paper's headline result is a two-orders-of-magnitude reduction in training cost, the total wall-clock comparison is incomplete unless the pre-encoding pass time is reported and included, and the amortized pretraining cost of DC-AE is either included or explicitly excluded with justification. The comparison also mixes 4090 GPU-hours with A100 GPU-hours without normalization, so the '40×∼150×' reduction stated in Section 4.2 is not yet established.
  2. [Section 3.2, Fig. 3b, Eq. (5), Appendix D.1] The 'boundary condition' for converting a VAE into a VQ-VAE is supported only by a Gaussian-noise tolerance experiment (Fig. 3b) and an empirically chosen MSE threshold of 0.1 (Appendix D.1). The paper provides no formal relation between the distribution of quantization error produced by channel multi-group quantization and Gaussian noise, and no argument that an ℓ2-optimal latent-space rectifier (Eq. 5) preserves perceptual quality after decoding. Since the method is presented as a general framework, this gap should be addressed either by analysis or by explicitly framing the threshold as an empirical observation specific to DC-AE, rather than a derived boundary condition.
  3. [Table 3, Section 4.2] The training-efficiency comparison is made against MaskBit and TiTok-S-128, which use different token counts (256 and 128) and are trained from scratch. TokenBridge and CODA, which also start from pre-trained VAEs and are the natural baselines for the 'frozen VAE' setting, are omitted from the training-time comparison. Reporting their training times, or otherwise restricting the efficiency claim to the from-scratch comparison, would make the speedup claim more credible and avoid the implication that ReVQ is the fastest option among all pre-trained-VAE-based tokenizers.
minor comments (5)
  1. [Table 2] The configuration 'ReVQ256T' is listed twice with different codebook sizes (65,536 and 262,144); the names should distinguish these two variants, e.g., ReVQ256T-S and ReVQ256T-L.
  2. [Table 3] The row 'ReVQ512T 262,144' appears inconsistent with Table 2 and Section 4.1, where the 262,144-codebook configuration is associated with 256 tokens; this is likely a typo and should be corrected.
  3. [Sections 4.1, 4.4, Table 5] The text and Table 5 interchangeably use 'decoder' and 'rectifier' for the post-quantization network, which is confusing because the frozen VAE decoder is a different module; the manuscript should consistently call the trained post-quantization network the rectifier.
  4. [Appendix A] The analysis of the reset strategy shows that the lower bound on quantization error can decrease after splitting a cluster, but it does not show that the specific reset operation in Eq. (3) achieves this lower bound; the empirical results in Fig. 8 partially compensate, but the wording 'This analysis demonstrates that the Reset operation can effectively reduce quantization error' is stronger than the derivation supports.
  5. [Section 5, Abstract] Section 5 acknowledges that ReVQ cannot reach very high compression ratios such as 32 tokens, yet the abstract and introduction emphasize 'at most 512 tokens' without this scope limitation; stating the compression-range limitation earlier would give readers a more accurate picture of the method's boundaries.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: measured rFID benchmarks and ablations carry the claims; the only self-citation (OptVQ) is motivational, not load-bearing.

full rationale

The central claims—512-token compression with rFID 1.06 and 22-hour training on one RTX 4090—are empirical and are compared against external, published rFID and GPU-hour numbers (Tables 2–3). The training objective (Eq. 5) minimizes l2 latent error; rFID is then measured on validation images through the frozen DC-AE decoder, so the reported quality is not forced by construction. The 'tolerance threshold' argument (Fig. 3b) is used only as motivation and is not a derivation: the paper never claims to compute rFID from the Gaussian-noise threshold, and the quantizer/rectifier are not trained to a threshold. The only self-citation, OptVQ [Zhang et al., 2024], motivates codebook-collapse awareness in Sec. 3.2.2, but the non-activation reset is independently proposed, theoretically analyzed in Appendix A, and ablated in Fig. 8; the load-bearing evidence is the paper's own experiments, not the cited result. The framing that 'quantization noise within the VAE's tolerance threshold' yields acceptable reconstruction is close to tautological, but it is not used to derive the measured results. Separately, the efficiency comparison omits the one-time DC-AE pre-encoding pass (Appendix C.1), which is an accounting/correctness issue, not a circular reduction: the 22-hour figure applies to the quantizer/rectifier training stage, and the quality numbers stand independently of the cost framing.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method adds no physical or conceptual entities; it reuses a pretrained DC-AE and introduces a quantizer and rectifier. The main quantities chosen by hand are the codebook sizes, token lengths, the noise tolerance threshold, the reset perturbation, and the fitted exponential coefficients that appear only in the motivation. The unstated premises are that the frozen VAE is a reliable reconstruction backbone, that Gaussian noise tolerance transfers to quantizer error, and that l2 latent loss is enough to train a rectifier that preserves perceptual quality.

free parameters (5)
  • Codebook size N = 16384 (512 tokens), 65536 or 262144 (256 tokens)
    Chosen by hand for each token configuration; directly controls representational capacity and training memory/time, and is central to the reported trade-offs.
  • Number of channel groups B (token length) = 512 or 256
    Design choice that sets the codebook dimension (2048/B) and the compression ratio; the paper compares these two settings but does not derive an optimal choice.
  • Noise tolerance threshold = variance at most 0.3
    Empirically estimated from Fig. 3b by adding Gaussian noise to DC-AE latents; used to justify that quantization error can be kept within the VAE's tolerance, but the connection to quantizer error is not quantified.
  • Exponential fit coefficients for codebook size vs token length = a = -3.6, b = 12.82 in M = 10^(a log10 B + b)
    Fitted to the empirical curve in Fig. 10 (Eq. 9); used only to motivate the need for a rectifier at low token lengths, not as part of the training objective.
  • Reset perturbation epsilon = small random noise (not specified)
    Added in Eq. (3) to avoid identical codes after resetting; the magnitude is not reported or ablated.
assumptions (4)
  • domain assumption The pre-trained DC-AE is a sufficiently good autoencoder for ImageNet reconstruction, and its decoder can reconstruct images from modified latents.
    The entire method freezes the DC-AE encoder and decoder (Section 4.1) and relies on its reconstruction ability after quantization and rectification; this is only validated empirically.
  • ad hoc to paper Quantization error can be treated as analogous to Gaussian noise for the purpose of the VAE's tolerance threshold.
    Section 3.2 uses Gaussian noise injection (Fig. 3b) to measure the VAE's tolerance, but the actual quantization error from nearest-neighbor search has a different distribution; the equivalence is not demonstrated.
  • domain assumption Minimizing l2 loss in latent space is sufficient to train a rectifier that improves perceptual reconstruction quality.
    Equation (5) optimizes only squared error on latent features, with no perceptual or adversarial loss; the paper's rFID results depend on this assumption holding.
  • standard math Independent codebooks per channel group yield N^B effective combinations.
    The capacity argument in Section 3.2.1 counts the product of codebook sizes, but feature correlations and training dynamics may make actual usable capacity lower.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Quantize-then-Rectify: Efficient VQ-VAE Training." pith.science (2026). https://pith.science/paper/PMVYEBDG

@misc{pith2026250710547,
  author       = {Pith},
  title        = {Pith review of: Quantize-then-Rectify: Efficient VQ-VAE Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PMVYEBDG}},
  note         = {Machine review of arXiv:2507.10547}
}
read the original abstract

Visual tokenizers are pivotal in multimodal large models, acting as bridges between continuous inputs and discrete tokens. Nevertheless, training high-compression-rate VQ-VAEs remains computationally demanding, often necessitating thousands of GPU hours. This work demonstrates that a pre-trained VAE can be efficiently transformed into a VQ-VAE by controlling quantization noise within the VAE's tolerance threshold. We present \textbf{Quantize-then-Rectify (ReVQ)}, a framework leveraging pre-trained VAEs to enable rapid VQ-VAE training with minimal computational overhead. By integrating \textbf{channel multi-group quantization} to enlarge codebook capacity and a \textbf{post rectifier} to mitigate quantization errors, ReVQ compresses ImageNet images into at most 512 tokens while sustaining competitive reconstruction quality (rFID = 1.06). Significantly, ReVQ reduces training costs by over two orders of magnitude relative to state-of-the-art approaches: ReVQ finishes full training on a single NVIDIA 4090 in approximately 22 hours, whereas comparable methods require 4.5 days on 32 A100 GPUs. Experimental results show that ReVQ achieves superior efficiency-reconstruction trade-offs.

Figures

Figures reproduced from arXiv: 2507.10547 by the authors.

Figure 1
Figure 1. ReVQ achieves the optimal trade-off between training efficiency (1 day on a single [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Comparison of Vanilla VQ and ReVQ. (Top) Vanilla VQ trains encoder, decoder, and [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Research Motivation. (a) Computational statistics reveal that shallow layers dominate com [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: (a) Three different quantizers. (b) Shows feature correlations under spatial/channel spliting, [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Reconstruction results on ImageNet validation set (details marked in red boxes). [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Single/multi-group strategy. w/ o Reset w / Reset [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 8
Figure 8. Figure 8: The results show that without the reset strategy, codebook utilization decreases rapidly as the codebook size increases, with only 65.3% of the codes utilized. In contrast, with the reset strategy, codebook utilization remains above 97% without significant decline as t…
Figure 10
Figure 10. Figure 10: Relationship between the token length B and the number of codebooks M required to keep the quantization error below 0.1. We found that when the quantization error (MSE) of the latent vector is below 0.1, the visual results of the reconstructed images are basically acc…
Figure 11
Figure 11. Figure 11: Details training statistics. D.3 More Visualizations on ImageNet In this section, we present additional reconstruction results, as shown in [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Additional reconstructed results on ImageNet dataset. [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. ChannelTok: Efficient Flexible-Length Vision Tokenization

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    ChannelTok introduces channel-wise tokenization with stochastic tail-dropping to achieve rFID 2.92 on ImageNet at 8.6x faster decoding and 2.1x smaller size than prior flexible tokenizers.

  2. NSVQ: Mitigating Codebook Collapse by Stabilizing Encoder Drift in Vector Quantization

    cs.CV 2026-06 unverdicted novelty 5.0 of 10

    NSVQ mitigates codebook collapse in large-codebook VQ by addressing encoder drift via non-stationary loss, replacement, and staged freezing, improving rFID from 2.39 to 2.10 on ImageNet-1k while achieving 100% utilization.

Reference graph

Works this paper leans on

45 extracted references · 26 canonical work pages · cited by 2 Pith papers

  1. [1]

    Cosmos world foundation model platform for physical ai

    Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, et al. Cosmos world foundation model platform for physical ai. arXiv, abs/2501.03575, 2025

    arXiv 2025

  2. [2]

    Sequential modeling enables scalable learning for large vision models

    Yutong Bai, Xinyang Geng, Karttikeya Mangalam, Amir Bar, Alan L Yuille, Trevor Darrell, Jitendra Malik, and Alexei A Efros. Sequential modeling enables scalable learning for large vision models. In CVPR, pages 22861--22872, 2024

    work page 2024

  3. [3]

    Beit: Bert pre-training of image transformers

    Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image transformers. In ICLR, 2022

    work page 2022

  4. [4]

    Estimating or propagating gradients through stochastic neurons for conditional computation

    Yoshua Bengio, Nicholas L \'e onard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv, abs/1308.3432, 2013

    arXiv 2013

  5. [5]

    Language models are few-shot learners

    Tom B Brown. Language models are few-shot learners. arXiv, abs/2005.14165, 2020

    arXiv 2005

  6. [6]

    Efficientvit: Lightweight multi-scale attention for high-resolution dense prediction

    Han Cai, Junyan Li, Muyan Hu, Chuang Gan, and Song Han. Efficientvit: Lightweight multi-scale attention for high-resolution dense prediction. In ICCV, pages 17302--17313, 2023

    work page 2023

  7. [7]

    Efficient-vqgan: Towards high-resolution image generation with efficient vision transformers

    Shiyue Cao, Yueqin Yin, Lianghua Huang, Yu Liu, Xin Zhao, Deli Zhao, and Kaigi Huang. Efficient-vqgan: Towards high-resolution image generation with efficient vision transformers. In ICCV, pages 7368--7377, 2023

    work page 2023

  8. [8]

    Maskgit: Masked generative image transformer

    Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. Maskgit: Masked generative image transformer. In CVPR, pages 11315--11325, 2022

    work page 2022

Show all 45 references
  1. [9]

    Deep compression autoencoder for efficient high-resolution diffusion models

    Junyu Chen, Han Cai, Junsong Chen, Enze Xie, Shang Yang, Haotian Tang, Muyang Li, Yao Lu, and Song Han. Deep compression autoencoder for efficient high-resolution diffusion models. arXiv, abs/2410.10733, 2024

    2024 arXiv

  2. [10]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248--255, 2009

    2009

  3. [11]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In CVPR, pages 12873--12883, 2021

    2021

  4. [12]

    Making llama see and draw with seed tokenizer

    Yuying Ge, Sijie Zhao, Ziyun Zeng, Yixiao Ge, Chen Li, Xintao Wang, and Ying Shan. Making llama see and draw with seed tokenizer. In ICLR, 2024

    2024

  5. [13]

    Generative adversarial nets

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. NeurIPS, 27, 2014

    2014

  6. [14]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. NeurIPS, 30, 2017

    2017

  7. [15]

    Reducing the dimensionality of data with neural networks

    Geoffrey E Hinton and Ruslan R Salakhutdinov. Reducing the dimensionality of data with neural networks. science, 313 0 (5786): 0 504--507, 2006

    2006

  8. [16]

    Straightening out the straight-through estimator: Overcoming optimization challenges in vector quantized networks

    Minyoung Huh, Brian Cheung, Pulkit Agrawal, and Phillip Isola. Straightening out the straight-through estimator: Overcoming optimization challenges in vector quantized networks. In ICML, pages 14096--14113, 2023

    2023

  9. [17]

    Image-to-image translation with conditional adversarial networks

    Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In CVPR, pages 1125--1134, 2017

    2017

  10. [18]

    Unified language-vision pretraining in llm with dynamic discrete visual tokenization

    Yang Jin, Kun Xu, Liwei Chen, Chao Liao, Jianchao Tan, Quzhe Huang, CHEN Bin, Chengru Song, Di ZHANG, Wenwu Ou, et al. Unified language-vision pretraining in llm with dynamic discrete visual tokenization. In ICLR, 2024

    2024

  11. [19]

    Perceptual losses for real-time style transfer and super-resolution

    Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In ECCV, pages 694--711, 2016

    2016

  12. [20]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv, abs/1312.6114, 2013

    2013 arXiv

  13. [21]

    Autoencoding beyond pixels using a learned similarity metric

    Anders Boesen Lindbo Larsen, S ren Kaae S nderby, Hugo Larochelle, and Ole Winther. Autoencoding beyond pixels using a learned similarity metric. In ICML, pages 1558--1566, 2016

    2016

  14. [22]

    Autoregressive image generation using residual quantization

    Doyup Lee, Chiheon Kim, Saehoon Kim, Minsu Cho, and Wook-Shin Han. Autoregressive image generation using residual quantization. In CVPR, pages 11523--11532, 2022

    2022

  15. [23]

    Imagefolder: Autoregressive image generation with folded tokens

    Xiang Li, Kai Qiu, Hao Chen, Jason Kuen, Jiuxiang Gu, Bhiksha Raj, and Zhe Lin. Imagefolder: Autoregressive image generation with folded tokens. In ICLR, 2025

    2025

  16. [24]

    Coda: Repurposing continuous vaes for discrete tokenization

    Zeyu Liu, Zanlin Ni, Yeguo Hua, Xin Deng, Xiao Ma, Cheng Zhong, and Gao Huang. Coda: Repurposing continuous vaes for discrete tokenization. arXiv, abs/2503.17760, 2025

    2025

  17. [25]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019

    2019

  18. [26]

    Unitok: A unified tokenizer for visual generation and understanding

    Chuofan Ma, Yi Jiang, Junfeng Wu, Jihan Yang, Xin Yu, Zehuan Yuan, Bingyue Peng, and Xiaojuan Qi. Unitok: A unified tokenizer for visual generation and understanding. arXiv, abs/2502.20321, 2025

    2025

  19. [27]

    Finite scalar quantization: Vq-vae made simple

    Fabian Mentzer, David Minnen, Eirikur Agustsson, and Michael Tschannen. Finite scalar quantization: Vq-vae made simple. arXiv, abs/2309.15505, 2023

    2023 arXiv

  20. [28]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth \'e e Darcet, Th \'e o Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv, abs/2304.07193, 2023

    2023 arXiv

  21. [29]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. NeurIPS, 32, 2019

    2019

  22. [30]

    Generating diverse high-fidelity images with vq-vae-2

    Ali Razavi, Aaron Van den Oord, and Oriol Vinyals. Generating diverse high-fidelity images with vq-vae-2. NeurIPS, 32, 2019

    2019

  23. [31]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj \"o rn Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, pages 10684--10695, 2022

    2022

  24. [32]

    Autoregressive model beats diffusion: Llama for scalable image generation

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv, abs/2406.06525, 2024

    2024 arXiv

  25. [33]

    Visual autoregressive modeling: Scalable image generation via next-scale prediction

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. In NeurIPS, 2024

    2024

  26. [34]

    Neural discrete representation learning

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. NeurIPS, 30, 2017

    2017

  27. [35]

    Bridging continuous and discrete tokens for autoregressive visual generation

    Yuqing Wang, Zhijie Lin, Yao Teng, Yuanzhi Zhu, Shuhuai Ren, Jiashi Feng, and Xihui Liu. Bridging continuous and discrete tokens for autoregressive visual generation. arXiv, abs/2503.16430, 2025

    2025 arXiv

  28. [36]

    Image quality assessment: from error visibility to structural similarity

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE TIP, 13 0 (4): 0 600--612, 2004

    2004

  29. [37]

    Maskbit: Embedding-free image generation via bit tokens

    Mark Weber, Lijun Yu, Qihang Yu, Xueqing Deng, Xiaohui Shen, Daniel Cremers, and Liang-Chieh Chen. Maskbit: Embedding-free image generation via bit tokens. arXiv, abs/2409.16211, 2024

    2024 arXiv

  30. [38]

    Vector-quantized image modeling with improved vqgan

    Jiahui Yu, Xin Li, Jing Yu Koh, Han Zhang, Ruoming Pang, James Qin, Alexander Ku, Yuanzhong Xu, Jason Baldridge, and Yonghui Wu. Vector-quantized image modeling with improved vqgan. In ICLR, 2022

    2022

  31. [39]

    Language model beats diffusion-tokenizer is key to visual generation

    Lijun Yu, Jose Lezama, Nitesh Bharadwaj Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Agrim Gupta, Xiuye Gu, Alexander G Hauptmann, et al. Language model beats diffusion-tokenizer is key to visual generation. In ICLR, 2024 a

    2024

  32. [40]

    An image is worth 32 tokens for reconstruction and generation

    Qihang Yu, Mark Weber, Xueqing Deng, Xiaohui Shen, Daniel Cremers, and Liang-Chieh Chen. An image is worth 32 tokens for reconstruction and generation. In NeurIPS, 2024 b

    2024

  33. [41]

    Preventing local pitfalls in vector quantization via optimal transport

    Borui Zhang, Wenzhao Zheng, Jie Zhou, and Jiwen Lu. Preventing local pitfalls in vector quantization via optimal transport. arXiv, abs/2412.15195, 2024

    2024 arXiv

  34. [42]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, pages 586--595, 2018

    2018

  35. [43]

    Movq: Modulating quantized vectors for high-fidelity image generation

    Chuanxia Zheng, Tung-Long Vuong, Jianfei Cai, and Dinh Phung. Movq: Modulating quantized vectors for high-fidelity image generation. NeurIPS, 35: 0 23412--23425, 2022

    2022

  36. [44]

    Scaling the codebook size of vqgan to 100,000 with a utilization rate of 99\ arXiv, abs/2406.11837, 2024

    Lei Zhu, Fangyun Wei, Yanye Lu, and Dong Chen. Scaling the codebook size of vqgan to 100,000 with a utilization rate of 99\ arXiv, abs/2406.11837, 2024

    2024 arXiv

  37. [45]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.